2024-04-24T05:33:06.653 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T05:33:06.653 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671152 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} --name rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi --owner scheduled_rishabh@teuthology --verbose -- /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671152/orig.config.yaml 2024-04-24T05:33:06.699 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T05:33:06.791 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671152 branch: wip-rishabh-testing-20240416.193735-5 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} email: null first_in_suite: false job_id: '7671152' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735-5 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735-5 sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7954 sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735-5 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJZ8f+QA2XGb+7sd1j7CUoN/FV5hrqTKOcg3SNpKfFMdBNWX7SiedWdG2mbVrPra1dFWU7VmpzqSwfB/VO3IRHE= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKhrioUtGsH6Ui2kfSAZASIczHQFe7It+EGUuixVBaf0uPjpauvKx1TVJ9KRiMtkxG7AOBmqFbBRz3DRQRhqszQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_05:22:11 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T05:33:06.791 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa; will attempt to use it 2024-04-24T05:33:06.791 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa/tasks 2024-04-24T05:33:06.792 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T05:33:06.793 INFO:teuthology.task.internal:Checking packages... 2024-04-24T05:33:06.818 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411' 2024-04-24T05:33:06.818 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T05:33:06.818 INFO:teuthology.packaging:ref: None 2024-04-24T05:33:06.818 INFO:teuthology.packaging:tag: None 2024-04-24T05:33:06.818 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735-5 2024-04-24T05:33:06.818 INFO:teuthology.packaging:sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T05:33:06.818 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735-5 2024-04-24T05:33:07.040 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3124.ga3c8a22b 2024-04-24T05:33:07.042 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T05:33:07.049 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T05:33:07.049 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T05:33:07.089 INFO:teuthology.task.internal:Saving configuration 2024-04-24T05:33:07.106 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T05:33:07.118 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T05:33:07.141 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671152', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 05:27:49.598534', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJZ8f+QA2XGb+7sd1j7CUoN/FV5hrqTKOcg3SNpKfFMdBNWX7SiedWdG2mbVrPra1dFWU7VmpzqSwfB/VO3IRHE='} 2024-04-24T05:33:07.165 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671152', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 05:27:49.599755', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKhrioUtGsH6Ui2kfSAZASIczHQFe7It+EGUuixVBaf0uPjpauvKx1TVJ9KRiMtkxG7AOBmqFbBRz3DRQRhqszQ='} 2024-04-24T05:33:07.165 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T05:33:07.173 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-24T05:33:07.173 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-24T05:33:07.173 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T05:33:07.248 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc6bf159160>, signals=[15]) 2024-04-24T05:33:07.249 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T05:33:07.257 INFO:teuthology.task.internal:Opening connections... 2024-04-24T05:33:07.257 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-04-24T05:33:07.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T05:33:07.332 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-04-24T05:33:07.333 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T05:33:07.405 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T05:33:07.413 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-04-24T05:33:07.430 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-04-24T05:33:07.431 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-04-24T05:33:07.487 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-04-24T05:33:07.487 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-04-24T05:33:07.487 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-04-24T05:33:07.487 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-04-24T05:33:07.487 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-04-24T05:33:07.487 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-04-24T05:33:07.488 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T05:33:07.488 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-04-24T05:33:07.488 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-04-24T05:33:07.488 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T05:33:07.488 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-04-24T05:33:07.488 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T05:33:07.488 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T05:33:07.488 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T05:33:07.489 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-04-24T05:33:07.504 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-04-24T05:33:07.522 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-04-24T05:33:07.522 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-04-24T05:33:07.579 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2024-04-24T05:33:07.579 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="9" 2024-04-24T05:33:07.580 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2024-04-24T05:33:07.580 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2024-04-24T05:33:07.580 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="9" 2024-04-24T05:33:07.580 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el9" 2024-04-24T05:33:07.580 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T05:33:07.580 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2024-04-24T05:33:07.580 INFO:teuthology.orchestra.run.smithi155.stdout:LOGO="fedora-logo-icon" 2024-04-24T05:33:07.580 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T05:33:07.580 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2024-04-24T05:33:07.580 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T05:33:07.580 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T05:33:07.580 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T05:33:07.580 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-04-24T05:33:07.604 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T05:33:07.615 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T05:33:07.634 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T05:33:07.634 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-04-24T05:33:07.637 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-04-24T05:33:07.653 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T05:33:07.660 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T05:33:07.661 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-04-24T05:33:07.693 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-04-24T05:33:07.718 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T05:33:07.775 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T05:33:07.792 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-24T05:33:07.792 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-24T05:33:07.792 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T05:33:07.793 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T05:33:07.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T05:33:07.809 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T05:33:07.809 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T05:33:07.809 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-04-24T05:33:07.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T05:33:07.811 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T05:33:07.811 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T05:33:07.811 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-04-24T05:33:07.866 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-437.el9.x86_64 2024-04-24T05:33:07.866 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T05:33:07.867 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T05:33:07.867 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T05:33:07.871 INFO:teuthology.orchestra.run.smithi155.stdout:5.14.0-437.el9.x86_64 2024-04-24T05:33:07.871 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T05:33:07.871 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T05:33:07.871 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T05:33:08.017 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum upgrade -y linux-firmware 2024-04-24T05:33:08.101 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum upgrade -y linux-firmware 2024-04-24T05:33:10.671 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 8.0 MB 00:00 2024-04-24T05:33:13.030 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-24T05:33:17.489 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 6.6 MB/s | 6.2 MB 00:00 2024-04-24T05:33:19.241 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 854 kB/s | 8.0 MB 00:09 2024-04-24T05:33:20.758 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 7.5 kB/s | 16 kB 00:02 2024-04-24T05:33:21.084 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-24T05:33:23.599 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 7.7 MB/s | 21 MB 00:02 2024-04-24T05:33:25.351 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.2 MB 00:00 2024-04-24T05:33:28.045 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-04-24T05:33:28.582 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-04-24T05:33:29.895 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-24T05:33:30.534 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-24T05:33:30.535 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-24T05:33:30.535 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-24T05:33:30.642 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-24T05:33:30.643 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T05:33:30.795 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T05:33:30.796 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T05:33:30.798 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi123.front.sepia.ceph.com to complete... 2024-04-24T05:33:35.053 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-24T05:33:37.043 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-24T05:33:37.044 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-04-24T05:33:37.044 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-24T05:33:37.158 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-24T05:33:37.158 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T05:33:37.308 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T05:33:37.308 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T05:33:37.310 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi155.front.sepia.ceph.com to complete... 2024-04-24T05:33:50.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24 05:33:50 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T05:33:50.366 INFO:teuthology.orchestra.run.smithi123.stderr:FINISHED --2024-04-24 05:33:50-- 2024-04-24T05:33:50.366 INFO:teuthology.orchestra.run.smithi123.stderr:Total wall clock time: 20s 2024-04-24T05:33:50.366 INFO:teuthology.orchestra.run.smithi123.stderr:Downloaded: 1 files, 341M in 19s (17.9 MB/s) 2024-04-24T05:33:50.369 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-24T05:33:50.369 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T05:33:52.464 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying... ######################################## 2024-04-24T05:33:52.744 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing... ######################################## 2024-04-24T05:33:52.847 INFO:teuthology.orchestra.run.smithi123.stdout:Updating / installing... 2024-04-24T05:34:00.843 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T05:34:02.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24 05:34:02 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T05:34:02.288 INFO:teuthology.orchestra.run.smithi155.stderr:FINISHED --2024-04-24 05:34:02-- 2024-04-24T05:34:02.288 INFO:teuthology.orchestra.run.smithi155.stderr:Total wall clock time: 25s 2024-04-24T05:34:02.288 INFO:teuthology.orchestra.run.smithi155.stderr:Downloaded: 1 files, 341M in 25s (13.9 MB/s) 2024-04-24T05:34:02.291 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-24T05:34:02.291 DEBUG:teuthology.orchestra.run.smithi155:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T05:34:04.425 INFO:teuthology.orchestra.run.smithi155.stdout:Verifying... ######################################## 2024-04-24T05:34:04.627 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing... ######################################## 2024-04-24T05:34:04.730 INFO:teuthology.orchestra.run.smithi155.stdout:Updating / installing... 2024-04-24T05:34:12.703 INFO:teuthology.orchestra.run.smithi155.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T05:34:39.498 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi123.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T05:34:39.499 DEBUG:teuthology.orchestra.run.smithi123:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T05:34:39.579 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:39.579 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:39.579 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:39.579 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T05:34:39.579 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T05:34:39.579 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T05:34:39.579 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T05:34:39.580 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T05:34:39.580 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T05:34:39.580 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T05:34:39.580 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T05:34:39.580 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T05:34:39.580 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T05:34:39.580 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T05:34:39.580 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T05:34:39.580 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T05:34:39.580 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T05:34:39.580 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T05:34:39.580 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T05:34:39.580 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T05:34:39.581 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T05:34:39.581 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T05:34:39.581 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T05:34:39.581 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T05:34:39.581 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T05:34:39.581 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T05:34:39.581 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T05:34:39.581 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T05:34:39.581 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T05:34:39.581 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T05:34:39.581 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T05:34:39.581 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T05:34:39.581 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T05:34:39.581 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T05:34:39.582 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T05:34:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T05:34:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T05:34:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T05:34:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T05:34:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T05:34:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T05:34:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T05:34:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T05:34:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T05:34:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T05:34:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T05:34:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T05:34:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T05:34:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T05:34:39.584 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T05:34:39.584 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T05:34:39.584 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T05:34:39.584 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T05:34:39.584 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T05:34:39.584 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T05:34:39.584 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T05:34:39.584 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T05:34:39.584 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T05:34:39.584 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T05:34:39.584 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T05:34:39.584 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T05:34:39.584 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T05:34:39.585 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T05:34:39.585 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T05:34:39.585 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T05:34:39.585 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T05:34:39.585 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T05:34:39.585 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T05:34:39.585 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T05:34:39.585 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T05:34:39.585 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T05:34:39.585 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T05:34:39.585 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T05:34:39.585 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T05:34:39.585 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T05:34:39.585 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T05:34:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T05:34:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T05:34:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T05:34:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T05:34:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T05:34:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T05:34:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T05:34:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T05:34:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T05:34:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T05:34:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T05:34:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T05:34:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T05:34:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T05:34:39.587 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T05:34:39.587 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T05:34:39.587 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T05:34:39.587 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T05:34:39.587 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T05:34:39.587 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T05:34:39.587 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T05:34:39.587 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T05:34:39.587 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T05:34:39.587 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T05:34:39.587 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T05:34:39.587 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T05:34:39.587 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T05:34:39.587 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T05:34:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T05:34:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T05:34:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T05:34:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T05:34:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T05:34:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T05:34:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T05:34:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T05:34:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T05:34:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T05:34:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T05:34:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T05:34:39.588 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T05:34:39.589 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T05:34:39.590 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T05:34:39.590 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T05:34:39.590 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T05:34:39.590 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T05:34:39.590 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T05:34:39.590 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T05:34:39.590 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T05:34:39.590 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T05:34:39.590 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T05:34:39.590 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T05:34:39.590 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T05:34:39.590 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T05:34:39.590 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T05:34:39.590 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T05:34:39.591 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T05:34:39.591 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T05:34:39.591 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T05:34:39.591 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T05:34:39.591 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T05:34:39.591 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T05:34:39.591 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T05:34:39.591 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T05:34:39.591 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T05:34:39.591 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T05:34:39.591 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T05:34:39.591 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T05:34:39.591 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T05:34:39.591 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T05:34:39.592 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T05:34:39.592 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T05:34:39.592 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T05:34:39.592 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T05:34:39.592 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T05:34:39.592 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T05:34:39.592 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T05:34:39.592 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T05:34:39.592 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T05:34:39.592 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T05:34:39.592 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T05:34:39.592 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T05:34:39.592 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T05:34:39.592 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T05:34:39.593 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T05:34:39.593 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T05:34:39.593 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T05:34:39.593 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T05:34:39.593 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T05:34:39.593 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T05:34:39.593 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T05:34:39.593 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T05:34:39.593 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T05:34:39.593 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T05:34:39.593 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T05:34:39.593 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T05:34:39.593 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T05:34:39.593 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T05:34:39.594 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T05:34:39.595 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T05:34:39.596 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T05:34:39.596 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T05:34:39.596 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T05:34:39.596 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T05:34:39.596 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T05:34:39.596 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T05:34:39.596 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T05:34:39.596 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T05:34:39.596 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T05:34:39.596 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T05:34:39.596 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T05:34:39.596 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T05:34:39.596 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T05:34:39.596 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T05:34:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T05:34:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T05:34:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T05:34:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T05:34:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T05:34:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T05:34:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T05:34:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T05:34:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T05:34:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T05:34:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T05:34:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T05:34:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T05:34:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T05:34:39.598 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T05:34:39.598 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T05:34:39.598 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T05:34:39.598 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T05:34:39.598 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T05:34:39.598 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T05:34:39.598 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T05:34:39.598 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T05:34:39.598 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T05:34:39.598 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T05:34:39.598 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T05:34:39.598 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T05:34:39.598 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T05:34:39.599 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T05:34:39.600 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T05:34:39.601 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T05:34:39.602 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T05:34:39.603 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T05:34:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T05:34:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T05:34:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T05:34:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T05:34:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T05:34:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T05:34:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T05:34:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T05:34:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T05:34:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T05:34:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T05:34:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T05:34:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T05:34:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T05:34:39.605 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T05:34:39.606 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T05:34:39.607 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T05:34:39.607 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T05:34:39.607 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T05:34:39.607 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T05:34:39.607 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T05:34:39.607 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T05:34:39.607 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T05:34:39.607 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T05:34:39.607 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T05:34:39.607 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T05:34:39.607 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T05:34:39.607 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T05:34:39.607 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T05:34:39.607 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T05:34:39.608 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T05:34:39.608 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T05:34:39.608 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T05:34:39.608 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T05:34:39.608 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T05:34:39.608 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T05:34:39.608 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T05:34:39.608 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T05:34:39.608 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T05:34:39.608 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T05:34:39.608 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T05:34:39.608 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T05:34:39.608 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T05:34:39.608 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T05:34:39.609 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T05:34:39.609 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T05:34:39.609 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T05:34:39.609 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T05:34:39.609 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T05:34:39.609 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T05:34:39.609 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T05:34:39.609 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T05:34:39.609 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T05:34:39.609 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T05:34:39.609 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T05:34:39.609 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T05:34:39.609 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T05:34:39.609 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T05:34:39.610 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T05:34:39.610 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T05:34:39.610 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T05:34:39.610 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T05:34:39.610 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T05:34:39.610 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T05:34:39.610 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T05:34:39.610 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T05:34:39.610 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T05:34:39.610 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T05:34:39.610 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T05:34:39.610 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T05:34:39.610 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T05:34:39.611 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T05:34:39.611 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T05:34:39.611 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T05:34:39.611 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T05:34:39.611 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T05:34:39.611 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T05:34:39.611 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T05:34:39.611 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T05:34:39.611 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T05:34:39.611 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T05:34:39.611 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T05:34:39.611 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T05:34:39.611 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T05:34:39.611 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T05:34:39.612 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T05:34:39.612 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T05:34:39.612 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T05:34:39.612 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T05:34:39.612 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T05:34:39.612 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T05:34:39.612 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T05:34:39.612 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T05:34:39.612 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T05:34:39.612 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T05:34:39.612 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T05:34:39.612 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T05:34:39.612 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T05:34:39.613 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T05:34:39.613 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T05:34:39.613 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T05:34:39.613 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T05:34:39.613 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T05:34:39.613 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T05:34:39.613 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T05:34:39.613 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T05:34:39.613 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T05:34:39.613 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T05:34:39.613 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T05:34:39.613 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T05:34:39.613 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T05:34:39.613 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T05:34:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T05:34:39.615 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T05:34:39.616 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T05:34:39.616 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T05:34:39.616 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T05:34:39.616 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T05:34:39.616 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T05:34:39.616 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T05:34:39.616 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T05:34:39.616 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T05:34:39.616 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T05:34:39.616 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T05:34:39.616 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T05:34:39.616 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T05:34:39.616 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T05:34:39.616 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T05:34:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T05:34:39.618 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T05:34:39.618 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T05:34:39.618 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T05:34:39.618 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T05:34:39.618 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T05:34:39.618 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T05:34:39.618 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T05:34:39.618 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T05:34:39.618 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T05:34:39.618 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T05:34:39.618 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T05:34:39.618 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T05:34:39.618 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T05:34:39.618 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T05:34:39.619 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T05:34:39.620 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T05:34:39.620 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T05:34:39.620 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T05:34:39.620 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T05:34:39.620 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T05:34:39.620 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T05:34:39.620 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T05:34:39.620 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T05:34:39.620 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T05:34:39.620 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T05:34:39.620 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T05:34:39.620 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T05:34:39.620 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T05:34:39.620 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T05:34:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T05:34:39.622 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T05:34:39.622 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T05:34:39.622 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T05:34:39.622 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T05:34:39.622 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T05:34:39.622 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T05:34:39.622 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T05:34:39.622 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T05:34:39.622 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T05:34:39.622 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T05:34:39.622 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T05:34:39.622 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T05:34:39.622 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T05:34:39.622 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T05:34:39.623 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T05:34:39.624 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T05:34:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T05:34:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T05:34:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T05:34:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T05:34:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T05:34:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T05:34:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T05:34:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T05:34:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T05:34:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T05:34:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T05:34:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T05:34:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T05:34:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T05:34:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T05:34:39.627 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T05:34:39.627 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T05:34:39.627 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T05:34:39.627 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T05:34:39.627 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T05:34:39.627 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T05:34:39.627 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T05:34:39.627 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T05:34:39.627 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T05:34:39.627 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T05:34:39.627 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T05:34:39.627 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T05:34:39.627 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T05:34:39.627 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T05:34:39.628 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T05:34:39.628 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T05:34:39.628 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T05:34:39.628 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T05:34:39.628 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T05:34:39.628 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T05:34:39.628 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T05:34:39.628 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T05:34:39.628 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T05:34:39.628 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T05:34:39.628 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T05:34:39.628 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T05:34:39.628 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T05:34:39.629 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T05:34:39.630 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T05:34:39.631 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T05:34:39.631 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T05:34:39.631 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T05:34:39.631 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T05:34:39.631 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T05:34:39.631 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T05:34:39.631 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T05:34:39.631 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T05:34:39.631 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T05:34:39.631 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T05:34:39.631 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T05:34:39.631 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T05:34:39.631 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T05:34:39.631 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T05:34:39.632 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T05:34:39.632 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T05:34:39.632 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T05:34:39.632 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T05:34:39.632 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T05:34:39.632 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T05:34:39.632 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T05:34:39.632 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T05:34:39.632 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T05:34:39.632 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T05:34:39.632 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T05:34:39.632 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T05:34:39.632 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T05:34:39.633 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T05:34:39.633 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T05:34:39.633 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T05:34:39.633 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T05:34:39.633 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T05:34:39.633 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T05:34:39.633 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T05:34:39.633 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T05:34:39.633 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T05:34:39.633 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T05:34:39.633 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T05:34:39.633 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T05:34:39.633 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T05:34:39.633 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T05:34:39.634 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T05:34:39.634 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T05:34:39.634 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T05:34:39.634 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T05:34:39.634 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T05:34:39.634 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T05:34:39.634 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T05:34:39.634 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T05:34:39.634 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T05:34:39.634 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T05:34:39.634 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T05:34:39.634 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T05:34:39.634 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T05:34:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T05:34:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T05:34:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T05:34:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T05:34:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T05:34:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T05:34:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T05:34:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T05:34:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T05:34:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T05:34:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T05:34:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T05:34:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T05:34:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T05:34:39.636 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T05:34:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T05:34:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T05:34:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T05:34:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T05:34:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T05:34:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T05:34:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T05:34:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T05:34:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T05:34:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T05:34:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T05:34:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T05:34:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T05:34:39.637 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T05:34:39.638 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T05:34:39.638 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T05:34:39.638 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T05:34:39.638 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T05:34:39.638 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T05:34:39.638 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T05:34:39.638 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T05:34:39.638 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T05:34:39.638 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T05:34:39.638 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T05:34:39.638 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T05:34:39.638 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T05:34:39.639 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T05:34:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T05:34:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T05:34:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T05:34:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T05:34:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T05:34:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T05:34:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T05:34:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T05:34:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T05:34:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T05:34:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T05:34:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T05:34:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T05:34:39.641 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T05:34:39.641 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T05:34:39.641 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T05:34:39.641 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T05:34:39.641 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T05:34:39.641 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T05:34:39.641 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T05:34:39.641 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T05:34:39.641 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T05:34:39.641 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T05:34:39.641 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T05:34:39.641 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T05:34:39.641 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T05:34:39.641 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T05:34:39.642 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T05:34:39.642 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T05:34:39.642 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T05:34:39.642 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T05:34:39.642 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T05:34:39.642 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T05:34:39.642 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T05:34:39.642 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T05:34:39.642 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T05:34:39.642 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T05:34:39.642 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T05:34:39.642 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T05:34:39.642 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T05:34:39.642 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T05:34:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T05:34:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T05:34:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T05:34:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T05:34:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T05:34:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T05:34:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T05:34:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T05:34:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T05:34:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T05:34:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T05:34:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T05:34:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T05:34:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T05:34:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T05:34:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T05:34:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T05:34:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T05:34:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T05:34:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T05:34:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T05:34:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T05:34:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T05:34:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T05:34:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T05:34:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T05:34:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T05:34:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T05:34:39.645 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T05:34:39.645 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T05:34:39.645 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T05:34:39.645 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T05:34:39.645 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T05:34:39.645 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T05:34:39.645 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T05:34:39.645 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T05:34:39.645 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T05:34:39.645 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T05:34:39.645 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T05:34:39.645 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T05:34:39.645 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T05:34:39.645 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T05:34:39.646 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T05:34:39.646 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T05:34:39.646 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T05:34:39.646 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T05:34:39.646 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T05:34:39.646 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T05:34:39.646 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T05:34:39.646 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T05:34:39.646 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T05:34:39.646 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T05:34:39.646 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T05:34:39.646 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T05:34:39.646 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T05:34:39.647 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T05:34:39.647 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T05:34:39.647 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T05:34:39.647 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T05:34:39.647 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T05:34:39.647 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T05:34:39.647 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T05:34:39.647 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T05:34:39.647 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T05:34:39.647 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T05:34:39.647 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T05:34:39.647 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T05:34:39.647 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T05:34:39.647 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T05:34:39.648 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T05:34:39.649 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T05:34:39.650 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T05:34:39.650 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T05:34:39.650 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T05:34:39.650 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T05:34:39.650 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T05:34:39.650 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T05:34:39.650 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T05:34:39.650 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T05:34:39.650 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T05:34:39.650 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T05:34:39.650 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T05:34:39.650 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T05:34:39.650 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T05:34:39.651 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T05:34:39.651 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T05:34:39.651 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T05:34:39.651 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T05:34:39.651 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T05:34:39.651 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T05:34:39.651 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T05:34:39.651 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T05:34:39.651 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T05:34:39.651 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T05:34:39.651 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T05:34:39.651 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T05:34:39.651 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T05:34:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T05:34:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T05:34:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T05:34:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T05:34:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T05:34:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T05:34:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T05:34:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T05:34:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T05:34:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T05:34:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T05:34:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T05:34:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T05:34:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T05:34:39.653 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T05:34:39.653 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T05:34:39.653 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T05:34:39.653 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T05:34:39.653 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T05:34:39.653 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T05:34:39.653 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T05:34:39.653 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T05:34:39.653 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T05:34:39.653 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T05:34:39.653 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T05:34:39.653 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T05:34:39.653 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T05:34:39.654 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T05:34:39.655 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T05:34:39.655 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T05:34:39.655 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T05:34:39.655 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T05:34:39.655 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T05:34:39.655 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T05:34:39.655 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T05:34:39.655 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T05:34:39.655 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T05:34:39.655 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T05:34:39.655 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T05:34:39.655 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T05:34:39.655 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T05:34:39.655 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T05:34:39.656 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T05:34:39.657 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T05:34:39.658 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T05:34:39.659 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T05:34:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T05:34:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T05:34:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T05:34:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T05:34:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T05:34:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T05:34:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T05:34:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T05:34:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T05:34:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T05:34:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T05:34:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T05:34:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T05:34:39.660 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T05:34:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T05:34:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T05:34:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T05:34:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T05:34:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T05:34:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T05:34:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T05:34:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T05:34:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T05:34:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T05:34:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T05:34:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T05:34:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T05:34:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T05:34:39.662 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T05:34:39.662 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T05:34:39.662 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T05:34:39.662 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T05:34:39.662 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T05:34:39.662 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T05:34:39.662 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T05:34:39.662 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T05:34:39.662 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T05:34:39.662 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T05:34:39.662 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T05:34:39.662 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T05:34:39.663 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T05:34:39.663 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T05:34:39.663 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T05:34:39.663 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T05:34:39.663 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T05:34:39.663 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T05:34:39.663 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T05:34:39.663 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T05:34:39.663 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T05:34:39.663 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T05:34:39.663 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T05:34:39.663 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T05:34:39.663 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T05:34:39.663 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T05:34:39.664 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T05:34:39.664 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T05:34:39.664 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T05:34:39.664 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T05:34:39.664 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T05:34:39.664 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T05:34:39.664 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T05:34:39.664 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T05:34:39.664 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T05:34:39.664 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T05:34:39.664 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T05:34:39.664 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T05:34:39.664 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T05:34:39.664 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T05:34:39.665 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T05:34:39.665 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T05:34:39.665 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T05:34:39.665 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T05:34:39.665 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T05:34:39.665 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T05:34:39.665 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T05:34:39.665 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T05:34:39.665 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T05:34:39.665 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T05:34:39.665 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T05:34:39.665 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T05:34:39.665 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T05:34:39.665 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T05:34:39.666 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T05:34:39.666 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T05:34:39.666 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T05:34:39.666 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T05:34:39.666 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T05:34:39.666 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T05:34:39.666 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T05:34:39.666 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T05:34:39.666 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T05:34:39.666 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T05:34:39.666 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T05:34:39.666 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T05:34:39.666 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T05:34:39.666 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T05:34:39.667 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T05:34:39.667 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T05:34:39.667 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T05:34:39.667 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T05:34:39.667 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T05:34:39.667 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T05:34:39.667 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T05:34:39.667 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T05:34:39.667 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T05:34:39.667 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T05:34:39.667 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T05:34:39.667 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T05:34:39.667 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T05:34:39.667 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T05:34:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T05:34:39.669 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T05:34:39.669 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T05:34:39.669 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T05:34:39.669 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T05:34:39.669 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T05:34:39.669 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T05:34:39.669 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T05:34:39.669 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T05:34:39.669 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T05:34:39.669 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T05:34:39.669 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T05:34:39.669 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T05:34:39.669 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T05:34:39.669 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T05:34:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T05:34:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T05:34:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T05:34:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T05:34:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T05:34:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T05:34:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T05:34:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T05:34:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T05:34:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T05:34:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T05:34:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T05:34:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T05:34:39.671 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T05:34:39.672 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T05:34:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T05:34:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T05:34:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T05:34:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T05:34:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T05:34:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T05:34:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T05:34:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T05:34:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T05:34:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T05:34:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T05:34:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T05:34:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T05:34:39.673 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T05:34:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T05:34:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T05:34:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T05:34:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T05:34:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T05:34:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T05:34:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T05:34:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T05:34:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T05:34:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T05:34:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T05:34:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T05:34:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T05:34:39.674 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T05:34:39.675 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T05:34:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T05:34:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T05:34:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T05:34:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T05:34:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T05:34:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T05:34:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T05:34:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T05:34:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T05:34:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T05:34:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T05:34:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T05:34:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T05:34:39.676 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T05:34:39.677 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T05:34:39.677 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T05:34:39.677 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T05:34:39.677 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T05:34:39.677 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T05:34:39.677 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T05:34:39.677 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T05:34:39.677 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T05:34:39.677 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T05:34:39.677 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T05:34:39.677 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T05:34:39.677 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T05:34:39.677 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T05:34:39.677 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T05:34:39.678 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T05:34:39.678 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T05:34:39.678 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T05:34:39.678 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T05:34:39.678 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T05:34:39.678 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T05:34:39.678 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T05:34:39.678 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T05:34:39.678 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T05:34:39.678 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T05:34:39.678 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T05:34:39.678 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T05:34:39.679 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T05:34:39.679 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T05:34:39.679 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T05:34:39.679 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T05:34:39.679 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T05:34:39.679 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T05:34:39.679 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T05:34:39.679 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T05:34:39.679 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T05:34:39.679 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T05:34:39.679 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T05:34:39.679 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T05:34:39.679 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T05:34:39.679 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T05:34:39.680 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T05:34:39.680 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T05:34:39.682 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T05:34:39.682 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T05:34:39.682 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T05:34:39.682 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T05:34:39.682 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T05:34:39.682 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T05:34:39.682 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T05:34:39.682 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T05:34:39.682 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T05:34:39.682 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T05:34:39.682 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T05:34:39.682 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T05:34:39.683 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T05:34:39.683 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T05:34:39.683 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T05:34:39.683 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T05:34:39.683 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T05:34:39.683 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T05:34:39.683 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T05:34:39.683 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T05:34:39.683 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T05:34:39.683 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T05:34:39.683 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T05:34:39.683 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T05:34:39.683 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T05:34:39.683 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T05:34:39.684 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T05:34:39.684 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T05:34:39.684 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T05:34:39.684 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T05:34:39.684 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T05:34:39.684 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T05:34:39.684 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T05:34:39.684 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T05:34:39.684 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T05:34:39.684 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T05:34:39.684 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T05:34:39.684 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T05:34:39.684 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T05:34:39.684 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T05:34:39.685 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T05:34:39.686 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T05:34:39.686 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T05:34:39.686 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T05:34:39.686 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T05:34:39.686 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T05:34:39.686 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T05:34:39.686 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T05:34:39.686 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T05:34:39.686 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T05:34:39.686 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T05:34:39.686 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T05:34:39.686 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T05:34:39.686 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T05:34:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T05:34:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T05:34:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T05:34:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T05:34:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T05:34:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T05:34:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T05:34:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T05:34:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T05:34:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T05:34:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T05:34:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T05:34:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T05:34:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T05:34:39.688 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T05:34:39.688 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T05:34:39.688 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T05:34:39.688 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T05:34:39.688 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T05:34:39.688 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T05:34:39.688 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T05:34:39.688 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T05:34:39.688 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T05:34:39.688 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T05:34:39.688 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T05:34:39.688 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T05:34:39.688 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T05:34:39.689 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T05:34:39.689 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T05:34:39.689 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T05:34:39.689 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T05:34:39.689 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T05:34:39.689 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T05:34:39.689 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T05:34:39.689 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T05:34:39.689 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T05:34:39.689 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T05:34:39.689 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T05:34:39.689 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T05:34:39.689 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T05:34:39.689 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T05:34:39.690 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T05:34:39.690 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T05:34:39.690 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T05:34:39.690 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T05:34:39.690 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T05:34:39.690 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T05:34:39.690 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T05:34:39.690 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T05:34:39.690 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T05:34:39.690 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T05:34:39.690 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T05:34:39.690 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T05:34:39.690 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T05:34:39.690 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T05:34:39.691 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T05:34:39.691 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T05:34:39.691 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T05:34:39.691 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T05:34:39.691 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T05:34:39.691 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T05:34:39.691 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T05:34:39.691 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T05:34:39.691 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T05:34:39.691 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T05:34:39.691 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T05:34:39.691 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T05:34:39.691 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T05:34:39.691 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T05:34:39.692 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T05:34:39.693 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T05:34:39.693 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T05:34:39.693 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T05:34:39.693 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T05:34:39.693 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T05:34:39.693 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T05:34:39.693 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T05:34:39.693 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T05:34:39.693 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T05:34:39.693 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T05:34:39.693 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T05:34:39.693 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T05:34:39.693 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T05:34:39.693 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T05:34:39.694 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T05:34:39.694 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T05:34:39.694 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T05:34:39.694 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T05:34:39.694 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T05:34:39.694 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T05:34:39.694 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T05:34:39.694 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T05:34:39.694 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T05:34:39.694 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T05:34:39.694 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T05:34:39.694 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T05:34:39.694 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T05:34:39.695 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T05:34:39.695 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T05:34:39.695 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T05:34:39.695 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T05:34:39.695 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T05:34:39.695 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T05:34:39.695 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T05:34:39.695 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T05:34:39.695 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T05:34:39.695 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T05:34:39.695 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T05:34:39.695 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T05:34:39.695 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T05:34:39.696 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T05:34:39.697 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T05:34:39.697 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T05:34:39.697 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T05:34:39.697 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T05:34:39.697 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T05:34:39.697 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T05:34:39.697 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T05:34:39.697 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T05:34:39.697 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T05:34:39.697 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T05:34:39.697 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T05:34:39.697 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T05:34:39.697 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T05:34:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T05:34:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T05:34:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T05:34:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T05:34:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T05:34:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T05:34:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T05:34:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T05:34:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T05:34:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T05:34:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T05:34:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T05:34:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T05:34:39.698 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T05:34:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T05:34:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T05:34:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T05:34:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T05:34:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T05:34:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T05:34:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T05:34:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T05:34:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T05:34:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T05:34:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T05:34:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T05:34:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T05:34:39.699 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T05:34:39.700 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T05:34:39.701 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T05:34:39.702 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T05:34:39.702 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T05:34:39.702 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T05:34:39.702 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T05:34:39.702 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T05:34:39.702 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T05:34:39.702 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T05:34:39.702 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T05:34:39.702 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T05:34:39.702 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T05:34:39.702 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T05:34:39.702 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T05:34:39.702 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T05:34:39.702 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T05:34:39.703 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T05:34:39.704 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T05:34:39.705 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T05:34:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T05:34:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T05:34:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T05:34:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T05:34:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T05:34:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T05:34:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T05:34:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T05:34:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T05:34:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T05:34:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T05:34:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T05:34:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T05:34:39.706 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T05:34:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T05:34:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T05:34:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T05:34:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T05:34:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T05:34:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T05:34:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T05:34:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T05:34:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T05:34:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T05:34:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T05:34:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T05:34:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T05:34:39.707 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T05:34:39.708 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T05:34:39.708 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T05:34:39.708 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T05:34:39.708 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T05:34:39.708 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T05:34:39.708 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T05:34:39.708 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T05:34:39.708 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T05:34:39.708 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T05:34:39.708 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T05:34:39.708 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T05:34:39.708 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T05:34:39.708 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T05:34:39.708 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T05:34:39.709 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T05:34:39.709 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T05:34:39.709 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T05:34:39.709 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T05:34:39.709 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T05:34:39.709 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T05:34:39.709 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T05:34:39.709 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T05:34:39.709 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T05:34:39.709 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T05:34:39.709 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T05:34:39.709 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T05:34:39.709 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T05:34:39.709 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T05:34:39.710 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T05:34:39.710 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T05:34:39.710 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T05:34:39.710 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T05:34:39.710 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T05:34:39.710 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T05:34:39.710 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T05:34:39.710 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T05:34:39.710 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T05:34:39.710 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T05:34:39.710 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T05:34:39.710 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T05:34:39.710 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T05:34:39.710 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T05:34:39.711 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T05:34:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T05:34:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T05:34:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T05:34:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T05:34:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T05:34:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T05:34:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T05:34:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T05:34:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T05:34:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T05:34:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T05:34:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T05:34:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T05:34:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T05:34:39.713 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T05:34:39.714 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T05:34:39.714 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T05:34:39.714 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T05:34:39.714 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T05:34:39.714 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T05:34:39.714 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T05:34:39.714 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T05:34:39.714 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T05:34:39.714 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T05:34:39.714 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T05:34:39.714 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T05:34:39.714 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T05:34:39.714 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T05:34:39.715 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T05:34:39.715 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T05:34:39.715 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T05:34:39.715 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T05:34:39.715 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T05:34:39.715 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T05:34:39.715 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T05:34:39.715 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T05:34:39.715 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T05:34:39.715 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T05:34:39.715 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T05:34:39.715 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T05:34:39.715 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T05:34:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T05:34:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T05:34:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T05:34:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T05:34:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T05:34:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T05:34:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T05:34:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T05:34:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T05:34:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T05:34:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T05:34:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T05:34:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T05:34:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T05:34:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T05:34:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T05:34:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T05:34:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T05:34:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T05:34:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T05:34:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T05:34:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T05:34:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T05:34:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T05:34:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T05:34:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T05:34:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T05:34:39.717 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T05:34:39.718 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T05:34:39.718 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T05:34:39.718 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T05:34:39.718 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T05:34:39.718 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T05:34:39.718 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T05:34:39.722 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:39.722 DEBUG:teuthology.orchestra.run.smithi123:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T05:34:39.744 INFO:teuthology.orchestra.run.smithi123.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T05:34:39.745 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T05:34:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T05:34:39.745 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-24T05:34:39.745 INFO:teuthology.orchestra.run.smithi123.stdout:for file in vmlinuz System.map config; do 2024-04-24T05:34:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T05:34:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T05:34:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: fi 2024-04-24T05:34:39.745 INFO:teuthology.orchestra.run.smithi123.stdout:done 2024-04-24T05:34:39.745 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T05:34:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T05:34:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T05:34:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T05:34:39.746 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-24T05:34:39.746 INFO:teuthology.orchestra.run.smithi123.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T05:34:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:39.746 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-24T05:34:39.746 INFO:teuthology.orchestra.run.smithi123.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T05:34:39.746 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T05:34:39.746 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:39.746 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-24T05:34:39.747 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -qi grub2-tools 2024-04-24T05:34:39.819 INFO:teuthology.orchestra.run.smithi123.stdout:Name : grub2-tools 2024-04-24T05:34:39.819 INFO:teuthology.orchestra.run.smithi123.stdout:Epoch : 1 2024-04-24T05:34:39.819 INFO:teuthology.orchestra.run.smithi123.stdout:Version : 2.06 2024-04-24T05:34:39.819 INFO:teuthology.orchestra.run.smithi123.stdout:Release : 68.el9 2024-04-24T05:34:39.820 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: x86_64 2024-04-24T05:34:39.820 INFO:teuthology.orchestra.run.smithi123.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T05:34:39.820 INFO:teuthology.orchestra.run.smithi123.stdout:Group : Unspecified 2024-04-24T05:34:39.820 INFO:teuthology.orchestra.run.smithi123.stdout:Size : 8264254 2024-04-24T05:34:39.820 INFO:teuthology.orchestra.run.smithi123.stdout:License : GPLv3+ 2024-04-24T05:34:39.820 INFO:teuthology.orchestra.run.smithi123.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T05:34:39.820 INFO:teuthology.orchestra.run.smithi123.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T05:34:39.820 INFO:teuthology.orchestra.run.smithi123.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T05:34:39.820 INFO:teuthology.orchestra.run.smithi123.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T05:34:39.820 INFO:teuthology.orchestra.run.smithi123.stdout:Packager : builder@centos.org 2024-04-24T05:34:39.820 INFO:teuthology.orchestra.run.smithi123.stdout:Vendor : CentOS 2024-04-24T05:34:39.820 INFO:teuthology.orchestra.run.smithi123.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T05:34:39.820 INFO:teuthology.orchestra.run.smithi123.stdout:Summary : Support tools for GRUB. 2024-04-24T05:34:39.820 INFO:teuthology.orchestra.run.smithi123.stdout:Description : 2024-04-24T05:34:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:34:39.821 INFO:teuthology.orchestra.run.smithi123.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T05:34:39.821 INFO:teuthology.orchestra.run.smithi123.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T05:34:39.821 INFO:teuthology.orchestra.run.smithi123.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T05:34:39.821 INFO:teuthology.orchestra.run.smithi123.stdout:hardware devices. 2024-04-24T05:34:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:34:39.821 INFO:teuthology.orchestra.run.smithi123.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T05:34:39.822 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T05:34:39.822 INFO:teuthology.task.kernel:Updating grub on smithi123 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:39.822 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T05:34:40.371 INFO:teuthology.orchestra.run.smithi123.stderr:Generating grub configuration file ... 2024-04-24T05:34:41.758 INFO:teuthology.orchestra.run.smithi123.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T05:34:41.788 INFO:teuthology.orchestra.run.smithi123.stderr:done 2024-04-24T05:34:41.791 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-04-24T05:34:41.810 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.ESUb3IXrLj 2024-04-24T05:34:41.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ESUb3IXrLj 2024-04-24T05:34:41.884 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.ESUb3IXrLj 2024-04-24T05:34:42.021 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.ESUb3IXrLj is 6KB 2024-04-24T05:34:42.041 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.ESUb3IXrLj 2024-04-24T05:34:42.055 DEBUG:teuthology.orchestra.run.smithi123:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T05:34:42.116 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T05:34:42.117 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T05:34:42.117 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T05:34:42.117 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T05:34:42.118 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:42.272 DEBUG:teuthology.orchestra.run.smithi123:> sudo shutdown -r now 2024-04-24T05:34:49.803 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi155.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T05:34:49.803 DEBUG:teuthology.orchestra.run.smithi155:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T05:34:49.888 INFO:teuthology.orchestra.run.smithi155.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:49.888 INFO:teuthology.orchestra.run.smithi155.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:49.889 INFO:teuthology.orchestra.run.smithi155.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:49.889 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T05:34:49.889 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T05:34:49.889 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T05:34:49.889 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T05:34:49.889 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T05:34:49.889 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T05:34:49.889 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T05:34:49.889 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T05:34:49.889 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T05:34:49.889 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T05:34:49.889 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T05:34:49.889 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T05:34:49.890 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T05:34:49.890 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T05:34:49.890 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T05:34:49.890 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T05:34:49.890 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T05:34:49.890 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T05:34:49.890 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T05:34:49.890 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T05:34:49.890 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T05:34:49.890 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T05:34:49.890 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T05:34:49.890 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T05:34:49.890 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T05:34:49.891 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T05:34:49.891 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T05:34:49.891 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T05:34:49.891 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T05:34:49.891 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T05:34:49.891 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T05:34:49.891 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T05:34:49.891 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T05:34:49.891 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T05:34:49.891 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T05:34:49.891 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T05:34:49.891 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T05:34:49.892 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T05:34:49.892 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T05:34:49.892 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T05:34:49.892 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T05:34:49.892 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T05:34:49.892 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T05:34:49.892 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T05:34:49.892 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T05:34:49.892 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T05:34:49.892 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T05:34:49.892 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T05:34:49.892 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T05:34:49.892 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T05:34:49.892 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T05:34:49.893 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T05:34:49.893 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T05:34:49.893 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T05:34:49.893 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T05:34:49.893 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T05:34:49.893 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T05:34:49.893 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T05:34:49.893 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T05:34:49.893 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T05:34:49.893 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T05:34:49.893 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T05:34:49.893 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T05:34:49.893 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T05:34:49.894 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T05:34:49.894 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T05:34:49.894 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T05:34:49.894 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T05:34:49.894 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T05:34:49.894 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T05:34:49.894 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T05:34:49.894 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T05:34:49.894 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T05:34:49.894 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T05:34:49.894 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T05:34:49.894 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T05:34:49.894 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T05:34:49.895 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T05:34:49.896 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T05:34:49.897 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T05:34:49.897 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T05:34:49.897 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T05:34:49.897 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T05:34:49.897 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T05:34:49.897 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T05:34:49.897 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T05:34:49.897 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T05:34:49.897 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T05:34:49.897 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T05:34:49.897 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T05:34:49.897 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T05:34:49.897 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T05:34:49.897 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T05:34:49.898 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T05:34:49.899 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T05:34:49.899 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T05:34:49.899 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T05:34:49.899 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T05:34:49.899 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T05:34:49.899 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T05:34:49.899 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T05:34:49.899 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T05:34:49.899 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T05:34:49.899 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T05:34:49.899 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T05:34:49.899 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T05:34:49.899 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T05:34:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T05:34:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T05:34:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T05:34:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T05:34:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T05:34:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T05:34:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T05:34:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T05:34:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T05:34:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T05:34:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T05:34:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T05:34:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T05:34:49.900 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T05:34:49.901 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T05:34:49.901 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T05:34:49.901 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T05:34:49.901 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T05:34:49.901 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T05:34:49.901 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T05:34:49.901 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T05:34:49.901 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T05:34:49.901 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T05:34:49.901 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T05:34:49.901 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T05:34:49.901 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T05:34:49.901 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T05:34:49.901 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T05:34:49.902 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T05:34:49.902 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T05:34:49.902 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T05:34:49.902 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T05:34:49.902 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T05:34:49.902 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T05:34:49.902 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T05:34:49.902 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T05:34:49.902 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T05:34:49.902 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T05:34:49.902 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T05:34:49.902 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T05:34:49.902 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T05:34:49.903 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T05:34:49.903 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T05:34:49.903 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T05:34:49.903 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T05:34:49.903 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T05:34:49.903 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T05:34:49.903 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T05:34:49.903 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T05:34:49.903 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T05:34:49.903 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T05:34:49.903 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T05:34:49.903 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T05:34:49.903 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T05:34:49.903 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T05:34:49.904 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T05:34:49.905 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T05:34:49.906 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T05:34:49.906 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T05:34:49.906 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T05:34:49.906 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T05:34:49.906 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T05:34:49.906 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T05:34:49.906 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T05:34:49.906 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T05:34:49.906 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T05:34:49.906 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T05:34:49.906 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T05:34:49.906 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T05:34:49.907 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T05:34:49.907 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T05:34:49.907 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T05:34:49.907 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T05:34:49.907 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T05:34:49.907 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T05:34:49.907 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T05:34:49.907 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T05:34:49.907 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T05:34:49.907 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T05:34:49.907 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T05:34:49.907 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T05:34:49.907 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T05:34:49.908 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T05:34:49.909 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T05:34:49.909 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T05:34:49.909 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T05:34:49.909 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T05:34:49.909 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T05:34:49.909 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T05:34:49.909 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T05:34:49.909 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T05:34:49.909 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T05:34:49.909 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T05:34:49.909 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T05:34:49.909 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T05:34:49.909 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T05:34:49.909 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T05:34:49.910 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T05:34:49.910 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T05:34:49.910 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T05:34:49.910 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T05:34:49.910 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T05:34:49.910 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T05:34:49.910 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T05:34:49.910 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T05:34:49.910 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T05:34:49.910 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T05:34:49.910 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T05:34:49.910 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T05:34:49.910 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T05:34:49.910 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T05:34:49.911 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T05:34:49.911 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T05:34:49.911 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T05:34:49.911 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T05:34:49.911 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T05:34:49.911 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T05:34:49.911 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T05:34:49.911 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T05:34:49.911 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T05:34:49.911 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T05:34:49.911 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T05:34:49.911 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T05:34:49.911 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T05:34:49.912 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T05:34:49.912 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T05:34:49.912 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T05:34:49.912 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T05:34:49.912 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T05:34:49.912 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T05:34:49.912 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T05:34:49.912 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T05:34:49.912 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T05:34:49.912 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T05:34:49.912 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T05:34:49.912 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T05:34:49.912 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T05:34:49.912 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T05:34:49.913 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T05:34:49.913 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T05:34:49.913 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T05:34:49.913 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T05:34:49.913 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T05:34:49.913 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T05:34:49.913 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T05:34:49.913 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T05:34:49.913 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T05:34:49.913 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T05:34:49.913 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T05:34:49.913 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T05:34:49.914 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T05:34:49.914 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T05:34:49.914 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T05:34:49.914 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T05:34:49.914 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T05:34:49.914 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T05:34:49.914 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T05:34:49.914 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T05:34:49.914 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T05:34:49.914 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T05:34:49.914 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T05:34:49.914 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T05:34:49.914 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T05:34:49.915 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T05:34:49.916 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T05:34:49.916 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T05:34:49.916 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T05:34:49.916 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T05:34:49.916 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T05:34:49.916 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T05:34:49.916 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T05:34:49.916 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T05:34:49.916 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T05:34:49.916 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T05:34:49.916 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T05:34:49.916 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T05:34:49.916 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T05:34:49.917 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T05:34:49.918 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T05:34:49.918 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T05:34:49.918 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T05:34:49.918 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T05:34:49.918 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T05:34:49.918 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T05:34:49.918 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T05:34:49.918 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T05:34:49.918 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T05:34:49.918 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T05:34:49.918 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T05:34:49.918 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T05:34:49.918 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T05:34:49.919 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T05:34:49.920 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T05:34:49.921 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T05:34:49.922 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T05:34:49.922 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T05:34:49.922 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T05:34:49.922 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T05:34:49.922 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T05:34:49.922 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T05:34:49.922 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T05:34:49.922 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T05:34:49.922 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T05:34:49.922 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T05:34:49.922 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T05:34:49.922 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T05:34:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T05:34:49.924 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T05:34:49.924 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T05:34:49.924 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T05:34:49.924 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T05:34:49.924 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T05:34:49.924 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T05:34:49.924 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T05:34:49.924 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T05:34:49.924 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T05:34:49.924 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T05:34:49.924 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T05:34:49.924 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T05:34:49.924 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T05:34:49.925 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T05:34:49.926 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T05:34:49.926 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T05:34:49.926 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T05:34:49.926 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T05:34:49.926 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T05:34:49.926 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T05:34:49.926 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T05:34:49.926 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T05:34:49.926 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T05:34:49.926 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T05:34:49.926 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T05:34:49.926 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T05:34:49.926 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T05:34:49.926 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T05:34:49.927 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T05:34:49.928 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T05:34:49.929 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T05:34:49.930 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T05:34:49.930 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T05:34:49.930 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T05:34:49.930 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T05:34:49.930 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T05:34:49.930 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T05:34:49.930 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T05:34:49.930 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T05:34:49.930 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T05:34:49.930 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T05:34:49.930 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T05:34:49.930 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T05:34:49.930 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T05:34:49.930 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T05:34:49.931 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T05:34:49.931 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T05:34:49.931 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T05:34:49.931 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T05:34:49.931 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T05:34:49.931 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T05:34:49.931 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T05:34:49.931 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T05:34:49.931 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T05:34:49.931 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T05:34:49.931 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T05:34:49.931 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T05:34:49.931 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T05:34:49.932 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T05:34:49.932 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T05:34:49.932 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T05:34:49.932 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T05:34:49.932 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T05:34:49.932 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T05:34:49.932 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T05:34:49.932 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T05:34:49.932 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T05:34:49.932 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T05:34:49.932 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T05:34:49.932 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T05:34:49.932 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T05:34:49.932 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T05:34:49.933 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T05:34:49.933 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T05:34:49.933 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T05:34:49.933 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T05:34:49.933 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T05:34:49.933 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T05:34:49.933 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T05:34:49.933 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T05:34:49.933 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T05:34:49.933 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T05:34:49.933 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T05:34:49.933 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T05:34:49.933 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T05:34:49.933 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T05:34:49.934 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T05:34:49.934 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T05:34:49.934 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T05:34:49.934 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T05:34:49.934 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T05:34:49.934 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T05:34:49.934 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T05:34:49.934 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T05:34:49.934 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T05:34:49.934 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T05:34:49.934 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T05:34:49.934 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T05:34:49.934 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T05:34:49.934 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T05:34:49.935 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T05:34:49.935 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T05:34:49.935 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T05:34:49.935 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T05:34:49.935 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T05:34:49.935 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T05:34:49.935 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T05:34:49.935 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T05:34:49.935 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T05:34:49.935 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T05:34:49.935 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T05:34:49.935 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T05:34:49.935 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T05:34:49.935 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T05:34:49.936 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T05:34:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T05:34:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T05:34:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T05:34:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T05:34:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T05:34:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T05:34:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T05:34:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T05:34:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T05:34:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T05:34:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T05:34:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T05:34:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T05:34:49.937 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T05:34:49.938 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T05:34:49.938 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T05:34:49.938 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T05:34:49.938 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T05:34:49.938 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T05:34:49.938 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T05:34:49.938 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T05:34:49.938 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T05:34:49.938 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T05:34:49.938 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T05:34:49.938 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T05:34:49.938 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T05:34:49.938 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T05:34:49.939 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T05:34:49.939 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T05:34:49.939 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T05:34:49.939 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T05:34:49.939 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T05:34:49.939 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T05:34:49.939 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T05:34:49.939 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T05:34:49.939 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T05:34:49.939 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T05:34:49.939 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T05:34:49.939 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T05:34:49.939 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T05:34:49.940 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T05:34:49.940 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T05:34:49.940 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T05:34:49.940 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T05:34:49.940 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T05:34:49.940 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T05:34:49.940 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T05:34:49.940 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T05:34:49.940 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T05:34:49.940 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T05:34:49.940 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T05:34:49.940 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T05:34:49.940 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T05:34:49.940 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T05:34:49.941 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T05:34:49.942 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T05:34:49.942 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T05:34:49.942 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T05:34:49.942 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T05:34:49.942 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T05:34:49.942 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T05:34:49.942 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T05:34:49.942 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T05:34:49.942 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T05:34:49.942 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T05:34:49.942 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T05:34:49.942 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T05:34:49.942 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T05:34:49.943 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T05:34:49.943 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T05:34:49.943 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T05:34:49.943 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T05:34:49.943 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T05:34:49.943 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T05:34:49.943 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T05:34:49.943 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T05:34:49.943 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T05:34:49.943 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T05:34:49.943 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T05:34:49.943 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T05:34:49.944 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T05:34:49.945 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T05:34:49.946 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T05:34:49.946 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T05:34:49.946 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T05:34:49.946 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T05:34:49.946 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T05:34:49.946 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T05:34:49.946 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T05:34:49.946 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T05:34:49.946 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T05:34:49.946 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T05:34:49.946 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T05:34:49.946 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T05:34:49.946 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T05:34:49.946 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T05:34:49.947 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T05:34:49.947 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T05:34:49.947 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T05:34:49.947 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T05:34:49.947 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T05:34:49.947 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T05:34:49.947 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T05:34:49.947 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T05:34:49.947 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T05:34:49.947 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T05:34:49.947 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T05:34:49.947 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T05:34:49.947 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T05:34:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T05:34:49.949 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T05:34:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T05:34:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T05:34:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T05:34:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T05:34:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T05:34:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T05:34:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T05:34:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T05:34:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T05:34:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T05:34:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T05:34:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T05:34:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T05:34:49.951 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T05:34:49.952 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T05:34:49.953 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T05:34:49.953 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T05:34:49.953 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T05:34:49.953 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T05:34:49.953 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T05:34:49.953 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T05:34:49.953 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T05:34:49.953 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T05:34:49.953 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T05:34:49.953 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T05:34:49.953 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T05:34:49.953 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T05:34:49.953 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T05:34:49.954 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T05:34:49.954 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T05:34:49.954 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T05:34:49.954 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T05:34:49.954 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T05:34:49.954 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T05:34:49.954 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T05:34:49.954 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T05:34:49.954 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T05:34:49.954 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T05:34:49.954 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T05:34:49.954 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T05:34:49.954 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T05:34:49.954 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T05:34:49.955 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T05:34:49.955 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T05:34:49.955 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T05:34:49.955 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T05:34:49.955 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T05:34:49.955 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T05:34:49.955 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T05:34:49.955 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T05:34:49.955 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T05:34:49.955 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T05:34:49.955 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T05:34:49.955 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T05:34:49.955 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T05:34:49.956 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T05:34:49.956 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T05:34:49.956 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T05:34:49.956 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T05:34:49.956 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T05:34:49.956 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T05:34:49.956 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T05:34:49.956 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T05:34:49.956 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T05:34:49.956 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T05:34:49.956 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T05:34:49.956 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T05:34:49.956 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T05:34:49.957 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T05:34:49.957 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T05:34:49.957 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T05:34:49.957 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T05:34:49.957 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T05:34:49.957 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T05:34:49.957 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T05:34:49.957 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T05:34:49.957 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T05:34:49.957 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T05:34:49.957 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T05:34:49.957 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T05:34:49.957 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T05:34:49.958 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T05:34:49.958 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T05:34:49.958 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T05:34:49.958 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T05:34:49.958 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T05:34:49.958 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T05:34:49.958 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T05:34:49.958 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T05:34:49.958 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T05:34:49.958 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T05:34:49.958 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T05:34:49.958 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T05:34:49.958 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T05:34:49.959 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T05:34:49.959 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T05:34:49.959 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T05:34:49.959 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T05:34:49.959 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T05:34:49.959 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T05:34:49.959 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T05:34:49.959 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T05:34:49.959 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T05:34:49.959 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T05:34:49.959 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T05:34:49.959 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T05:34:49.959 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T05:34:49.959 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T05:34:49.960 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T05:34:49.960 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T05:34:49.960 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T05:34:49.960 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T05:34:49.960 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T05:34:49.960 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T05:34:49.960 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T05:34:49.960 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T05:34:49.960 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T05:34:49.960 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T05:34:49.960 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T05:34:49.960 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T05:34:49.960 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T05:34:49.960 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T05:34:49.961 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T05:34:49.962 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T05:34:49.962 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T05:34:49.962 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T05:34:49.962 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T05:34:49.962 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T05:34:49.962 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T05:34:49.962 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T05:34:49.962 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T05:34:49.962 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T05:34:49.962 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T05:34:49.962 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T05:34:49.962 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T05:34:49.962 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T05:34:49.963 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T05:34:49.963 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T05:34:49.963 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T05:34:49.963 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T05:34:49.963 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T05:34:49.963 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T05:34:49.963 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T05:34:49.963 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T05:34:49.963 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T05:34:49.963 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T05:34:49.963 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T05:34:49.963 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T05:34:49.963 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T05:34:49.963 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T05:34:49.964 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T05:34:49.964 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T05:34:49.964 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T05:34:49.964 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T05:34:49.964 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T05:34:49.964 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T05:34:49.964 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T05:34:49.964 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T05:34:49.964 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T05:34:49.964 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T05:34:49.964 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T05:34:49.964 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T05:34:49.964 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T05:34:49.964 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T05:34:49.965 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T05:34:49.965 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T05:34:49.965 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T05:34:49.965 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T05:34:49.965 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T05:34:49.965 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T05:34:49.965 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T05:34:49.965 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T05:34:49.965 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T05:34:49.965 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T05:34:49.965 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T05:34:49.965 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T05:34:49.965 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T05:34:49.965 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T05:34:49.966 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T05:34:49.966 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T05:34:49.966 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T05:34:49.966 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T05:34:49.966 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T05:34:49.966 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T05:34:49.966 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T05:34:49.966 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T05:34:49.966 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T05:34:49.966 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T05:34:49.966 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T05:34:49.966 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T05:34:49.966 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T05:34:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T05:34:49.968 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T05:34:49.968 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T05:34:49.968 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T05:34:49.968 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T05:34:49.968 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T05:34:49.968 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T05:34:49.968 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T05:34:49.968 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T05:34:49.968 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T05:34:49.968 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T05:34:49.968 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T05:34:49.968 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T05:34:49.968 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T05:34:49.969 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T05:34:49.969 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T05:34:49.969 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T05:34:49.969 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T05:34:49.969 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T05:34:49.969 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T05:34:49.969 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T05:34:49.969 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T05:34:49.969 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T05:34:49.969 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T05:34:49.969 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T05:34:49.969 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T05:34:49.969 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T05:34:49.970 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T05:34:49.970 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T05:34:49.970 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T05:34:49.970 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T05:34:49.970 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T05:34:49.970 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T05:34:49.970 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T05:34:49.970 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T05:34:49.970 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T05:34:49.970 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T05:34:49.970 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T05:34:49.970 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T05:34:49.970 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T05:34:49.970 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T05:34:49.971 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T05:34:49.972 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T05:34:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T05:34:49.974 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T05:34:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T05:34:49.976 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T05:34:49.976 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T05:34:49.976 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T05:34:49.976 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T05:34:49.976 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T05:34:49.976 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T05:34:49.976 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T05:34:49.976 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T05:34:49.976 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T05:34:49.976 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T05:34:49.976 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T05:34:49.976 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T05:34:49.976 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T05:34:49.976 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T05:34:49.977 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T05:34:49.977 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T05:34:49.977 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T05:34:49.977 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T05:34:49.977 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T05:34:49.977 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T05:34:49.977 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T05:34:49.977 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T05:34:49.977 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T05:34:49.977 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T05:34:49.977 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T05:34:49.977 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T05:34:49.977 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T05:34:49.978 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T05:34:49.979 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T05:34:49.979 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T05:34:49.979 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T05:34:49.979 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T05:34:49.979 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T05:34:49.979 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T05:34:49.979 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T05:34:49.979 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T05:34:49.979 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T05:34:49.979 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T05:34:49.979 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T05:34:49.979 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T05:34:49.979 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T05:34:49.979 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T05:34:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T05:34:49.981 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T05:34:49.982 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T05:34:49.982 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T05:34:49.982 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T05:34:49.982 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T05:34:49.982 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T05:34:49.982 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T05:34:49.982 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T05:34:49.982 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T05:34:49.982 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T05:34:49.982 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T05:34:49.982 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T05:34:49.982 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T05:34:49.982 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T05:34:49.982 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T05:34:49.983 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T05:34:49.984 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T05:34:49.985 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T05:34:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T05:34:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T05:34:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T05:34:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T05:34:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T05:34:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T05:34:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T05:34:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T05:34:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T05:34:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T05:34:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T05:34:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T05:34:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T05:34:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T05:34:49.987 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T05:34:49.987 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T05:34:49.987 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T05:34:49.987 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T05:34:49.987 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T05:34:49.987 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T05:34:49.987 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T05:34:49.987 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T05:34:49.987 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T05:34:49.987 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T05:34:49.987 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T05:34:49.987 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T05:34:49.987 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T05:34:49.987 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T05:34:49.988 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T05:34:49.989 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T05:34:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T05:34:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T05:34:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T05:34:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T05:34:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T05:34:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T05:34:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T05:34:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T05:34:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T05:34:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T05:34:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T05:34:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T05:34:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T05:34:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T05:34:49.991 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T05:34:49.992 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T05:34:49.993 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T05:34:49.994 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T05:34:49.994 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T05:34:49.994 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T05:34:49.994 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T05:34:49.994 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T05:34:49.994 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T05:34:49.994 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T05:34:49.994 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T05:34:49.994 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T05:34:49.994 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T05:34:49.994 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T05:34:49.994 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T05:34:49.994 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T05:34:49.995 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T05:34:49.995 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T05:34:49.995 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T05:34:49.995 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T05:34:49.995 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T05:34:49.995 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T05:34:49.995 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T05:34:49.995 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T05:34:49.995 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T05:34:49.995 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T05:34:49.995 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T05:34:49.995 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T05:34:49.995 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T05:34:49.995 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T05:34:49.996 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T05:34:49.997 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T05:34:49.997 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T05:34:49.997 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T05:34:49.997 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T05:34:49.997 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T05:34:49.997 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T05:34:49.997 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T05:34:49.997 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T05:34:49.997 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T05:34:49.997 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T05:34:49.997 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T05:34:49.997 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T05:34:49.997 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T05:34:49.997 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T05:34:49.998 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T05:34:49.998 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T05:34:49.998 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T05:34:49.998 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T05:34:49.998 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T05:34:49.998 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T05:34:49.998 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T05:34:49.998 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T05:34:49.998 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T05:34:49.998 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T05:34:49.998 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T05:34:49.998 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T05:34:49.998 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T05:34:49.999 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T05:34:50.000 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T05:34:50.001 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T05:34:50.001 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T05:34:50.001 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T05:34:50.001 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T05:34:50.001 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T05:34:50.001 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T05:34:50.001 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T05:34:50.001 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T05:34:50.001 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T05:34:50.001 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T05:34:50.001 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T05:34:50.001 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T05:34:50.001 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T05:34:50.001 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T05:34:50.002 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T05:34:50.002 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T05:34:50.002 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T05:34:50.002 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T05:34:50.002 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T05:34:50.002 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T05:34:50.002 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T05:34:50.002 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T05:34:50.002 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T05:34:50.002 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T05:34:50.002 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T05:34:50.002 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T05:34:50.002 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T05:34:50.003 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T05:34:50.003 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T05:34:50.003 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T05:34:50.003 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T05:34:50.003 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T05:34:50.003 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T05:34:50.003 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T05:34:50.003 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T05:34:50.003 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T05:34:50.003 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T05:34:50.003 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T05:34:50.003 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T05:34:50.003 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T05:34:50.003 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T05:34:50.004 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T05:34:50.005 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T05:34:50.006 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T05:34:50.007 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T05:34:50.007 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T05:34:50.007 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T05:34:50.007 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T05:34:50.007 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T05:34:50.007 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T05:34:50.007 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T05:34:50.007 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T05:34:50.007 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T05:34:50.007 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T05:34:50.007 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T05:34:50.007 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T05:34:50.007 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T05:34:50.007 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T05:34:50.008 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T05:34:50.009 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T05:34:50.010 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T05:34:50.010 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T05:34:50.010 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T05:34:50.010 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T05:34:50.010 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T05:34:50.010 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T05:34:50.010 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T05:34:50.010 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T05:34:50.010 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T05:34:50.010 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T05:34:50.010 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T05:34:50.010 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T05:34:50.010 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T05:34:50.010 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T05:34:50.011 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T05:34:50.012 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T05:34:50.013 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T05:34:50.014 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T05:34:50.015 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T05:34:50.016 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T05:34:50.017 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T05:34:50.017 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T05:34:50.017 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T05:34:50.017 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T05:34:50.017 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T05:34:50.017 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T05:34:50.017 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T05:34:50.017 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T05:34:50.017 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T05:34:50.017 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T05:34:50.017 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T05:34:50.017 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T05:34:50.017 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T05:34:50.017 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T05:34:50.018 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T05:34:50.018 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T05:34:50.018 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T05:34:50.018 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T05:34:50.018 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T05:34:50.018 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T05:34:50.018 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T05:34:50.018 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T05:34:50.018 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T05:34:50.018 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T05:34:50.018 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T05:34:50.019 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T05:34:50.019 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T05:34:50.019 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T05:34:50.019 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T05:34:50.019 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T05:34:50.019 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T05:34:50.019 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T05:34:50.019 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T05:34:50.019 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T05:34:50.019 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T05:34:50.019 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T05:34:50.020 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T05:34:50.020 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T05:34:50.020 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T05:34:50.020 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T05:34:50.020 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T05:34:50.020 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T05:34:50.020 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T05:34:50.020 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T05:34:50.020 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T05:34:50.020 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T05:34:50.020 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T05:34:50.020 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T05:34:50.020 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T05:34:50.021 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T05:34:50.021 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T05:34:50.021 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T05:34:50.021 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T05:34:50.021 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T05:34:50.021 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T05:34:50.021 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T05:34:50.021 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T05:34:50.021 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T05:34:50.021 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T05:34:50.021 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T05:34:50.021 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T05:34:50.021 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T05:34:50.022 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T05:34:50.022 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T05:34:50.022 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T05:34:50.022 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T05:34:50.022 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T05:34:50.022 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T05:34:50.022 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T05:34:50.022 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T05:34:50.022 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T05:34:50.022 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T05:34:50.022 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T05:34:50.023 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T05:34:50.024 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T05:34:50.024 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T05:34:50.024 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T05:34:50.024 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T05:34:50.024 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T05:34:50.024 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T05:34:50.024 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T05:34:50.024 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T05:34:50.024 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T05:34:50.024 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T05:34:50.024 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T05:34:50.024 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T05:34:50.024 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T05:34:50.024 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T05:34:50.025 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T05:34:50.025 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T05:34:50.025 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T05:34:50.025 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T05:34:50.025 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T05:34:50.025 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T05:34:50.025 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T05:34:50.025 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T05:34:50.025 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T05:34:50.025 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T05:34:50.025 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T05:34:50.025 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T05:34:50.025 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T05:34:50.026 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T05:34:50.026 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T05:34:50.026 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T05:34:50.026 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T05:34:50.026 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T05:34:50.026 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T05:34:50.026 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T05:34:50.030 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:50.030 DEBUG:teuthology.orchestra.run.smithi155:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T05:34:50.052 INFO:teuthology.orchestra.run.smithi155.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T05:34:50.052 INFO:teuthology.orchestra.run.smithi155.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T05:34:50.052 INFO:teuthology.orchestra.run.smithi155.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T05:34:50.053 INFO:teuthology.orchestra.run.smithi155.stdout:fi 2024-04-24T05:34:50.053 INFO:teuthology.orchestra.run.smithi155.stdout:for file in vmlinuz System.map config; do 2024-04-24T05:34:50.053 INFO:teuthology.orchestra.run.smithi155.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T05:34:50.053 INFO:teuthology.orchestra.run.smithi155.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T05:34:50.053 INFO:teuthology.orchestra.run.smithi155.stdout: fi 2024-04-24T05:34:50.053 INFO:teuthology.orchestra.run.smithi155.stdout:done 2024-04-24T05:34:50.053 INFO:teuthology.orchestra.run.smithi155.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T05:34:50.053 INFO:teuthology.orchestra.run.smithi155.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T05:34:50.053 INFO:teuthology.orchestra.run.smithi155.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T05:34:50.053 INFO:teuthology.orchestra.run.smithi155.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T05:34:50.053 INFO:teuthology.orchestra.run.smithi155.stdout:fi 2024-04-24T05:34:50.053 INFO:teuthology.orchestra.run.smithi155.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T05:34:50.053 INFO:teuthology.orchestra.run.smithi155.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:50.053 INFO:teuthology.orchestra.run.smithi155.stdout:fi 2024-04-24T05:34:50.054 INFO:teuthology.orchestra.run.smithi155.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T05:34:50.054 INFO:teuthology.orchestra.run.smithi155.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T05:34:50.054 INFO:teuthology.orchestra.run.smithi155.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:50.054 INFO:teuthology.orchestra.run.smithi155.stdout:fi 2024-04-24T05:34:50.055 DEBUG:teuthology.orchestra.run.smithi155:> sudo rpm -qi grub2-tools 2024-04-24T05:34:50.127 INFO:teuthology.orchestra.run.smithi155.stdout:Name : grub2-tools 2024-04-24T05:34:50.127 INFO:teuthology.orchestra.run.smithi155.stdout:Epoch : 1 2024-04-24T05:34:50.127 INFO:teuthology.orchestra.run.smithi155.stdout:Version : 2.06 2024-04-24T05:34:50.127 INFO:teuthology.orchestra.run.smithi155.stdout:Release : 68.el9 2024-04-24T05:34:50.127 INFO:teuthology.orchestra.run.smithi155.stdout:Architecture: x86_64 2024-04-24T05:34:50.127 INFO:teuthology.orchestra.run.smithi155.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T05:34:50.128 INFO:teuthology.orchestra.run.smithi155.stdout:Group : Unspecified 2024-04-24T05:34:50.128 INFO:teuthology.orchestra.run.smithi155.stdout:Size : 8264254 2024-04-24T05:34:50.128 INFO:teuthology.orchestra.run.smithi155.stdout:License : GPLv3+ 2024-04-24T05:34:50.128 INFO:teuthology.orchestra.run.smithi155.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T05:34:50.128 INFO:teuthology.orchestra.run.smithi155.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T05:34:50.128 INFO:teuthology.orchestra.run.smithi155.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T05:34:50.128 INFO:teuthology.orchestra.run.smithi155.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T05:34:50.128 INFO:teuthology.orchestra.run.smithi155.stdout:Packager : builder@centos.org 2024-04-24T05:34:50.128 INFO:teuthology.orchestra.run.smithi155.stdout:Vendor : CentOS 2024-04-24T05:34:50.128 INFO:teuthology.orchestra.run.smithi155.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T05:34:50.128 INFO:teuthology.orchestra.run.smithi155.stdout:Summary : Support tools for GRUB. 2024-04-24T05:34:50.128 INFO:teuthology.orchestra.run.smithi155.stdout:Description : 2024-04-24T05:34:50.128 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:34:50.129 INFO:teuthology.orchestra.run.smithi155.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T05:34:50.129 INFO:teuthology.orchestra.run.smithi155.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T05:34:50.129 INFO:teuthology.orchestra.run.smithi155.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T05:34:50.129 INFO:teuthology.orchestra.run.smithi155.stdout:hardware devices. 2024-04-24T05:34:50.129 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:34:50.129 INFO:teuthology.orchestra.run.smithi155.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T05:34:50.130 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T05:34:50.130 INFO:teuthology.task.kernel:Updating grub on smithi155 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:50.130 DEBUG:teuthology.orchestra.run.smithi155:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T05:34:50.649 INFO:teuthology.orchestra.run.smithi155.stderr:Generating grub configuration file ... 2024-04-24T05:34:52.042 INFO:teuthology.orchestra.run.smithi155.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T05:34:52.068 INFO:teuthology.orchestra.run.smithi155.stderr:done 2024-04-24T05:34:52.070 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-04-24T05:34:52.090 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.GWMHZBVLEi 2024-04-24T05:34:52.091 DEBUG:teuthology.orchestra.run.smithi155:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.GWMHZBVLEi 2024-04-24T05:34:52.157 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.GWMHZBVLEi 2024-04-24T05:34:52.297 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.GWMHZBVLEi is 6KB 2024-04-24T05:34:52.320 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.GWMHZBVLEi 2024-04-24T05:34:52.335 DEBUG:teuthology.orchestra.run.smithi155:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T05:34:52.396 INFO:teuthology.orchestra.run.smithi155.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T05:34:52.396 INFO:teuthology.orchestra.run.smithi155.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T05:34:52.396 INFO:teuthology.orchestra.run.smithi155.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T05:34:52.396 INFO:teuthology.orchestra.run.smithi155.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T05:34:52.398 DEBUG:teuthology.orchestra.run.smithi155:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T05:34:52.544 DEBUG:teuthology.orchestra.run.smithi155:> sudo shutdown -r now 2024-04-24T05:35:12.294 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-24T05:35:12.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-24T05:35:12.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T05:35:22.556 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-24T05:35:22.557 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-24T05:35:22.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T05:35:30.850 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-24T05:35:39.860 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-24T05:35:39.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T05:35:41.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-04-24T05:35:42.942 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-24T05:35:50.095 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-24T05:35:50.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T05:35:53.150 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-04-24T05:35:54.944 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-24T05:35:54.945 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T05:36:05.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-24T05:36:05.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T05:36:13.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-24T05:36:28.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-24T05:36:28.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T05:36:28.828 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-04-24T05:36:29.435 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-24T05:36:29.435 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T05:36:29.435 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-04-24T05:36:29.474 INFO:teuthology.orchestra.run.smithi123.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T05:36:29.475 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T05:36:29.475 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T05:36:29.475 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T05:36:30.475 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-24T05:36:30.476 DEBUG:teuthology.orchestra.run.smithi123:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T05:36:30.608 INFO:teuthology.orchestra.run.smithi123.stdout:ttyS1 2024-04-24T05:36:30.656 DEBUG:teuthology.parallel:result is None 2024-04-24T05:37:05.163 DEBUG:teuthology.orchestra.remote:timed out 2024-04-24T05:37:20.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-24T05:37:20.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T05:37:20.555 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-04-24T05:37:21.246 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-24T05:37:21.246 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T05:37:21.247 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-04-24T05:37:21.301 INFO:teuthology.orchestra.run.smithi155.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T05:37:21.301 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T05:37:21.301 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T05:37:21.301 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T05:37:22.302 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-24T05:37:22.302 DEBUG:teuthology.orchestra.run.smithi155:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T05:37:22.459 INFO:teuthology.orchestra.run.smithi155.stdout:ttyS1 2024-04-24T05:37:22.526 DEBUG:teuthology.parallel:result is None 2024-04-24T05:37:22.526 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T05:37:22.538 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T05:37:22.538 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T05:37:22.547 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T05:37:22.604 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T05:37:22.612 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T05:37:22.619 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T05:37:22.619 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T05:37:22.624 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T05:37:22.690 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T05:37:22.697 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T05:37:22.697 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T05:37:22.702 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T05:37:22.788 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T05:37:22.793 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T05:37:22.820 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T05:37:22.824 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T05:37:22.829 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T05:37:22.839 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T05:37:22.839 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T05:37:22.844 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T05:37:22.929 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T05:37:22.939 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T05:37:22.940 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T05:37:22.945 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T05:37:22.999 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T05:37:23.081 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T05:37:23.170 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T05:37:23.250 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T05:37:23.334 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:37:23.335 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T05:37:23.422 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T05:37:23.512 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T05:37:23.603 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T05:37:23.687 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T05:37:23.777 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:37:23.778 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T05:37:23.864 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-04-24T05:37:23.871 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-04-24T05:37:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T05:37:24.012 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T05:37:24.433 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T05:37:24.440 INFO:teuthology.task.internal:Starting timer... 2024-04-24T05:37:24.440 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T05:37:24.450 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T05:37:24.458 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-24T05:37:24.458 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-04-24T05:37:24.586 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-04-24T05:37:24.591 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2024-04-24T05:37:24.738 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2024-04-24T05:37:24.742 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T05:37:24.743 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-04-24T05:37:24.799 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-04-24T05:37:24.800 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2024-04-24T05:37:24.859 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2024-04-24T05:37:24.859 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi123.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive'} 2024-04-24T05:37:24.860 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T05:37:24.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T05:37:24.940 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T05:37:25.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T05:37:25.016 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T05:37:25.017 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-04-24T05:37:25.088 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2024-04-24T05:37:25.150 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T05:37:25.159 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T05:37:25.179 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T05:37:25.180 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi123.front.sepia.ceph.com,smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T05:42:33.371 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi123.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2024-04-24T05:42:33.373 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-24T05:42:33.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T05:42:33.478 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-04-24T05:42:33.622 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-24T05:42:33.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-24T05:42:33.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T05:42:33.730 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-04-24T05:42:33.883 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-04-24T05:42:33.884 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T05:42:33.897 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T05:42:33.897 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T05:42:33.898 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T05:42:33.906 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T05:42:33.906 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T05:42:33.992 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T05:42:33.992 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T05:42:34.031 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T05:42:34.033 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T05:42:34.105 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2024-04-24T05:42:34.106 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-04-24T05:42:34.136 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2024-04-24T05:42:34.140 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-04-24T05:42:34.175 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T05:42:34.181 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T05:42:34.218 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T05:42:34.223 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T05:42:34.343 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-04-24T05:42:34.351 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T05:42:34.351 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-04-24T05:42:34.351 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T05:42:34.351 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T05:42:34.352 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T05:42:34.352 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T05:42:34.355 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-04-24T05:42:34.364 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T05:42:34.364 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-04-24T05:42:34.364 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T05:42:34.364 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T05:42:34.365 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T05:42:34.365 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T05:42:34.366 INFO:teuthology.run_tasks:Running task install... 2024-04-24T05:42:34.375 DEBUG:teuthology.task.install:project ceph 2024-04-24T05:42:34.375 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'}} 2024-04-24T05:42:34.375 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'} 2024-04-24T05:42:34.375 INFO:teuthology.task.install:Using flavor: default 2024-04-24T05:42:34.381 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T05:42:34.381 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T05:42:34.382 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'tag': None, 'wait_for_package': False} 2024-04-24T05:42:34.383 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T05:42:34.385 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'tag': None, 'wait_for_package': False} 2024-04-24T05:42:34.385 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T05:42:34.532 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/ 2024-04-24T05:42:34.532 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3124.ga3c8a22b 2024-04-24T05:42:34.734 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/ 2024-04-24T05:42:34.734 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3124.ga3c8a22b 2024-04-24T05:42:35.044 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T05:42:35.044 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:42:35.044 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T05:42:35.132 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T05:42:35.132 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T05:42:35.158 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T05:42:35.158 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:42:35.158 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T05:42:35.248 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T05:42:35.248 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T05:42:35.264 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T05:42:35.370 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T05:42:35.375 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T05:42:35.447 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2024-04-24T05:42:35.452 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2024-04-24T05:42:35.480 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T05:42:35.554 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-04-24T05:42:35.558 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-04-24T05:42:35.770 INFO:teuthology.orchestra.run.smithi155.stdout:48 files removed 2024-04-24T05:42:35.809 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T05:42:35.877 INFO:teuthology.orchestra.run.smithi123.stdout:48 files removed 2024-04-24T05:42:35.915 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T05:42:37.239 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 94 kB/s | 83 kB 00:00 2024-04-24T05:42:37.266 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 81 kB/s | 83 kB 00:01 2024-04-24T05:42:38.047 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-04-24T05:42:38.091 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-04-24T05:42:38.958 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 2.4 kB/s | 1.9 kB 00:00 2024-04-24T05:42:38.990 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 2.2 kB/s | 1.9 kB 00:00 2024-04-24T05:42:43.588 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.0 MB 00:04 2024-04-24T05:42:45.539 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-24T05:42:50.772 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 703 kB/s | 8.0 MB 00:11 2024-04-24T05:42:50.964 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.2 MB 00:01 2024-04-24T05:42:54.221 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 8.6 kB/s | 16 kB 00:01 2024-04-24T05:42:56.171 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-24T05:43:01.705 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 1.9 MB/s | 19 MB 00:09 2024-04-24T05:43:02.066 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-24T05:43:04.426 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T05:43:04.427 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T05:43:04.446 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T05:43:04.447 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T05:43:04.448 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T05:43:04.450 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T05:43:04.452 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T05:43:04.455 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T05:43:04.456 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T05:43:04.526 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-24T05:43:04.543 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-24T05:43:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-24T05:43:04.543 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-24T05:43:04.543 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-24T05:43:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T05:43:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 6.6 k 2024-04-24T05:43:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 5.4 M 2024-04-24T05:43:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 890 k 2024-04-24T05:43:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 145 k 2024-04-24T05:43:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 1.5 M 2024-04-24T05:43:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 142 k 2024-04-24T05:43:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 3.5 M 2024-04-24T05:43:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 7.4 M 2024-04-24T05:43:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 49 k 2024-04-24T05:43:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 11 M 2024-04-24T05:43:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 49 M 2024-04-24T05:43:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 273 k 2024-04-24T05:43:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 762 k 2024-04-24T05:43:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 224 k 2024-04-24T05:43:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 27 k 2024-04-24T05:43:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 18 k 2024-04-24T05:43:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T05:43:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T05:43:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T05:43:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 32 k 2024-04-24T05:43:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 747 k 2024-04-24T05:43:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 127 k 2024-04-24T05:43:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T05:43:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T05:43:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T05:43:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T05:43:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T05:43:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T05:43:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 162 k 2024-04-24T05:43:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 322 k 2024-04-24T05:43:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 297 k 2024-04-24T05:43:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 99 k 2024-04-24T05:43:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 85 k 2024-04-24T05:43:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.1 M 2024-04-24T05:43:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 170 k 2024-04-24T05:43:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T05:43:04.546 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-04-24T05:43:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.4 M 2024-04-24T05:43:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.1 M 2024-04-24T05:43:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T05:43:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T05:43:04.547 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-24T05:43:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T05:43:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T05:43:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T05:43:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 21 M 2024-04-24T05:43:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 34 k 2024-04-24T05:43:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 2.4 M 2024-04-24T05:43:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 249 k 2024-04-24T05:43:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 4.7 M 2024-04-24T05:43:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 17 M 2024-04-24T05:43:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 16 k 2024-04-24T05:43:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 25 k 2024-04-24T05:43:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T05:43:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T05:43:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T05:43:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T05:43:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T05:43:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T05:43:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T05:43:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T05:43:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 163 k 2024-04-24T05:43:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T05:43:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T05:43:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T05:43:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T05:43:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T05:43:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 501 k 2024-04-24T05:43:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T05:43:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 5.4 M 2024-04-24T05:43:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T05:43:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T05:43:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T05:43:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T05:43:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T05:43:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T05:43:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T05:43:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T05:43:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T05:43:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T05:43:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T05:43:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T05:43:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T05:43:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T05:43:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T05:43:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T05:43:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T05:43:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 45 k 2024-04-24T05:43:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 135 k 2024-04-24T05:43:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T05:43:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T05:43:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T05:43:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T05:43:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T05:43:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T05:43:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T05:43:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T05:43:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T05:43:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T05:43:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T05:43:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T05:43:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T05:43:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T05:43:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T05:43:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T05:43:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T05:43:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T05:43:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T05:43:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T05:43:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T05:43:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T05:43:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T05:43:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T05:43:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T05:43:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T05:43:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T05:43:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T05:43:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T05:43:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T05:43:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T05:43:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T05:43:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T05:43:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T05:43:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T05:43:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T05:43:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T05:43:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T05:43:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T05:43:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T05:43:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T05:43:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T05:43:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T05:43:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T05:43:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T05:43:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T05:43:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T05:43:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T05:43:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T05:43:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T05:43:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T05:43:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T05:43:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T05:43:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T05:43:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T05:43:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T05:43:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T05:43:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T05:43:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T05:43:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T05:43:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T05:43:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T05:43:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T05:43:04.556 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-04-24T05:43:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T05:43:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:43:04.556 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-24T05:43:04.556 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-24T05:43:04.556 INFO:teuthology.orchestra.run.smithi123.stdout:Install 148 Packages 2024-04-24T05:43:04.556 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 4 Packages 2024-04-24T05:43:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:43:04.557 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 222 M 2024-04-24T05:43:04.557 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-24T05:43:05.402 INFO:teuthology.orchestra.run.smithi123.stdout:(1/152): ceph-19.0.0-3124.ga3c8a22b.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-24T05:43:06.126 INFO:teuthology.orchestra.run.smithi123.stdout:(2/152): ceph-fuse-19.0.0-3124.ga3c8a22b.el9.x8 1.2 MB/s | 890 kB 00:00 2024-04-24T05:43:06.232 INFO:teuthology.orchestra.run.smithi123.stdout:(3/152): ceph-immutable-object-cache-19.0.0-312 1.4 MB/s | 145 kB 00:00 2024-04-24T05:43:06.444 INFO:teuthology.orchestra.run.smithi123.stdout:(4/152): ceph-base-19.0.0-3124.ga3c8a22b.el9.x8 3.9 MB/s | 5.4 MB 00:01 2024-04-24T05:43:06.619 INFO:teuthology.orchestra.run.smithi123.stdout:(5/152): ceph-mgr-19.0.0-3124.ga3c8a22b.el9.x86 8.6 MB/s | 1.5 MB 00:00 2024-04-24T05:43:06.856 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.2 MB 00:01 2024-04-24T05:43:07.119 INFO:teuthology.orchestra.run.smithi123.stdout:(6/152): ceph-mon-19.0.0-3124.ga3c8a22b.el9.x86 9.4 MB/s | 4.7 MB 00:00 2024-04-24T05:43:07.269 INFO:teuthology.orchestra.run.smithi123.stdout:(7/152): ceph-mds-19.0.0-3124.ga3c8a22b.el9.x86 2.3 MB/s | 2.4 MB 00:01 2024-04-24T05:43:07.753 INFO:teuthology.orchestra.run.smithi123.stdout:(8/152): ceph-common-19.0.0-3124.ga3c8a22b.el9. 7.9 MB/s | 21 MB 00:02 2024-04-24T05:43:07.811 INFO:teuthology.orchestra.run.smithi123.stdout:(9/152): ceph-selinux-19.0.0-3124.ga3c8a22b.el9 439 kB/s | 25 kB 00:00 2024-04-24T05:43:08.612 INFO:teuthology.orchestra.run.smithi123.stdout:(10/152): ceph-osd-19.0.0-3124.ga3c8a22b.el9.x8 11 MB/s | 17 MB 00:01 2024-04-24T05:43:08.678 INFO:teuthology.orchestra.run.smithi123.stdout:(11/152): cephfs-mirror-19.0.0-3124.ga3c8a22b.e 3.4 MB/s | 224 kB 00:00 2024-04-24T05:43:08.729 INFO:teuthology.orchestra.run.smithi123.stdout:(12/152): cephfs-shell-19.0.0-3124.ga3c8a22b.el 560 kB/s | 27 kB 00:00 2024-04-24T05:43:08.779 INFO:teuthology.orchestra.run.smithi123.stdout:(13/152): libcephfs-devel-19.0.0-3124.ga3c8a22b 644 kB/s | 32 kB 00:00 2024-04-24T05:43:08.904 INFO:teuthology.orchestra.run.smithi123.stdout:(14/152): libcephfs2-19.0.0-3124.ga3c8a22b.el9. 5.9 MB/s | 747 kB 00:00 2024-04-24T05:43:08.979 INFO:teuthology.orchestra.run.smithi123.stdout:(15/152): libcephsqlite-19.0.0-3124.ga3c8a22b.e 2.2 MB/s | 163 kB 00:00 2024-04-24T05:43:09.025 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 16 kB 00:00 2024-04-24T05:43:09.047 INFO:teuthology.orchestra.run.smithi123.stdout:(16/152): librados-devel-19.0.0-3124.ga3c8a22b. 1.9 MB/s | 127 kB 00:00 2024-04-24T05:43:09.139 INFO:teuthology.orchestra.run.smithi123.stdout:(17/152): libradosstriper1-19.0.0-3124.ga3c8a22 5.4 MB/s | 501 kB 00:00 2024-04-24T05:43:09.605 INFO:teuthology.orchestra.run.smithi123.stdout:(18/152): librgw2-19.0.0-3124.ga3c8a22b.el9.x86 12 MB/s | 5.4 MB 00:00 2024-04-24T05:43:09.664 INFO:teuthology.orchestra.run.smithi123.stdout:(19/152): python3-ceph-argparse-19.0.0-3124.ga3 791 kB/s | 45 kB 00:00 2024-04-24T05:43:09.731 INFO:teuthology.orchestra.run.smithi123.stdout:(20/152): python3-ceph-common-19.0.0-3124.ga3c8 2.0 MB/s | 135 kB 00:00 2024-04-24T05:43:09.898 INFO:teuthology.orchestra.run.smithi123.stdout:(21/152): ceph-radosgw-19.0.0-3124.ga3c8a22b.el 4.0 MB/s | 11 MB 00:02 2024-04-24T05:43:09.931 INFO:teuthology.orchestra.run.smithi123.stdout:(22/152): python3-cephfs-19.0.0-3124.ga3c8a22b. 813 kB/s | 162 kB 00:00 2024-04-24T05:43:10.006 INFO:teuthology.orchestra.run.smithi123.stdout:(23/152): python3-rbd-19.0.0-3124.ga3c8a22b.el9 3.9 MB/s | 297 kB 00:00 2024-04-24T05:43:10.065 INFO:teuthology.orchestra.run.smithi123.stdout:(24/152): python3-rados-19.0.0-3124.ga3c8a22b.e 1.9 MB/s | 322 kB 00:00 2024-04-24T05:43:10.091 INFO:teuthology.orchestra.run.smithi123.stdout:(25/152): python3-rgw-19.0.0-3124.ga3c8a22b.el9 1.2 MB/s | 99 kB 00:00 2024-04-24T05:43:10.149 INFO:teuthology.orchestra.run.smithi123.stdout:(26/152): rbd-fuse-19.0.0-3124.ga3c8a22b.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-24T05:43:10.258 INFO:teuthology.orchestra.run.smithi123.stdout:(27/152): rbd-nbd-19.0.0-3124.ga3c8a22b.el9.x86 1.6 MB/s | 170 kB 00:00 2024-04-24T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout:(28/152): rbd-mirror-19.0.0-3124.ga3c8a22b.el9. 11 MB/s | 3.1 MB 00:00 2024-04-24T05:43:10.408 INFO:teuthology.orchestra.run.smithi123.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3124.g 229 kB/s | 34 kB 00:00 2024-04-24T05:43:10.450 INFO:teuthology.orchestra.run.smithi123.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3124.ga3c8a22 2.1 MB/s | 142 kB 00:00 2024-04-24T05:43:10.571 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-24T05:43:11.092 INFO:teuthology.orchestra.run.smithi123.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-04-24T05:43:11.167 INFO:teuthology.orchestra.run.smithi123.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3124.ga3 3.3 MB/s | 249 kB 00:00 2024-04-24T05:43:11.264 INFO:teuthology.orchestra.run.smithi123.stdout:(33/152): ceph-mgr-rook-19.0.0-3124.ga3c8a22b.e 514 kB/s | 49 kB 00:00 2024-04-24T05:43:11.360 INFO:teuthology.orchestra.run.smithi123.stdout:(34/152): ceph-mgr-dashboard-19.0.0-3124.ga3c8a 3.7 MB/s | 3.5 MB 00:00 2024-04-24T05:43:11.385 INFO:teuthology.orchestra.run.smithi123.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3124.ga 138 kB/s | 16 kB 00:00 2024-04-24T05:43:11.452 INFO:teuthology.orchestra.run.smithi123.stdout:(36/152): ceph-volume-19.0.0-3124.ga3c8a22b.el9 2.9 MB/s | 273 kB 00:00 2024-04-24T05:43:11.502 INFO:teuthology.orchestra.run.smithi123.stdout:(37/152): cephadm-19.0.0-3124.ga3c8a22b.el9.noa 6.4 MB/s | 762 kB 00:00 2024-04-24T05:43:11.527 INFO:teuthology.orchestra.run.smithi123.stdout:(38/152): cephfs-top-19.0.0-3124.ga3c8a22b.el9. 245 kB/s | 18 kB 00:00 2024-04-24T05:43:12.020 INFO:teuthology.orchestra.run.smithi123.stdout:(39/152): ceph-test-19.0.0-3124.ga3c8a22b.el9.x 12 MB/s | 49 MB 00:04 2024-04-24T05:43:12.111 INFO:teuthology.orchestra.run.smithi123.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 132 kB/s | 80 kB 00:00 2024-04-24T05:43:12.178 INFO:teuthology.orchestra.run.smithi123.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-24T05:43:12.228 INFO:teuthology.orchestra.run.smithi123.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-24T05:43:12.262 INFO:teuthology.orchestra.run.smithi123.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-04-24T05:43:12.279 INFO:teuthology.orchestra.run.smithi123.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-24T05:43:12.313 INFO:teuthology.orchestra.run.smithi123.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-24T05:43:12.346 INFO:teuthology.orchestra.run.smithi123.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-24T05:43:12.380 INFO:teuthology.orchestra.run.smithi123.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-04-24T05:43:12.430 INFO:teuthology.orchestra.run.smithi123.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.2 MB 00:00 2024-04-24T05:43:12.555 INFO:teuthology.orchestra.run.smithi123.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-04-24T05:43:12.581 INFO:teuthology.orchestra.run.smithi123.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 716 kB/s | 107 kB 00:00 2024-04-24T05:43:12.606 INFO:teuthology.orchestra.run.smithi123.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 436 kB/s | 21 kB 00:00 2024-04-24T05:43:12.656 INFO:teuthology.orchestra.run.smithi123.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-04-24T05:43:12.682 INFO:teuthology.orchestra.run.smithi123.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 439 kB/s | 33 kB 00:00 2024-04-24T05:43:12.715 INFO:teuthology.orchestra.run.smithi123.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-24T05:43:12.749 INFO:teuthology.orchestra.run.smithi123.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 275 kB/s | 18 kB 00:00 2024-04-24T05:43:12.774 INFO:teuthology.orchestra.run.smithi123.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 192 kB/s | 11 kB 00:00 2024-04-24T05:43:12.807 INFO:teuthology.orchestra.run.smithi123.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-24T05:43:12.833 INFO:teuthology.orchestra.run.smithi123.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 786 kB/s | 45 kB 00:00 2024-04-24T05:43:12.891 INFO:teuthology.orchestra.run.smithi123.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-24T05:43:12.933 INFO:teuthology.orchestra.run.smithi123.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.8 MB/s | 582 kB 00:00 2024-04-24T05:43:12.959 INFO:teuthology.orchestra.run.smithi123.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 386 kB/s | 25 kB 00:00 2024-04-24T05:43:12.984 INFO:teuthology.orchestra.run.smithi123.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 962 kB/s | 47 kB 00:00 2024-04-24T05:43:13.017 INFO:teuthology.orchestra.run.smithi123.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-24T05:43:13.068 INFO:teuthology.orchestra.run.smithi123.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-24T05:43:13.093 INFO:teuthology.orchestra.run.smithi123.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 475 kB/s | 35 kB 00:00 2024-04-24T05:43:13.193 INFO:teuthology.orchestra.run.smithi123.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.6 MB/s | 3.0 MB 00:00 2024-04-24T05:43:13.302 INFO:teuthology.orchestra.run.smithi123.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-24T05:43:13.410 INFO:teuthology.orchestra.run.smithi123.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-04-24T05:43:13.461 INFO:teuthology.orchestra.run.smithi123.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-04-24T05:43:13.553 INFO:teuthology.orchestra.run.smithi123.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.7 MB/s | 4.4 MB 00:00 2024-04-24T05:43:13.578 INFO:teuthology.orchestra.run.smithi123.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 800 kB/s | 133 kB 00:00 2024-04-24T05:43:13.628 INFO:teuthology.orchestra.run.smithi123.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-24T05:43:13.670 INFO:teuthology.orchestra.run.smithi123.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-04-24T05:43:13.720 INFO:teuthology.orchestra.run.smithi123.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-04-24T05:43:13.770 INFO:teuthology.orchestra.run.smithi123.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 5.0 MB/s | 248 kB 00:00 2024-04-24T05:43:13.813 INFO:teuthology.orchestra.run.smithi123.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 630 kB/s | 26 kB 00:00 2024-04-24T05:43:13.855 INFO:teuthology.orchestra.run.smithi123.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 470 kB/s | 19 kB 00:00 2024-04-24T05:43:13.904 INFO:teuthology.orchestra.run.smithi123.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-24T05:43:14.022 INFO:teuthology.orchestra.run.smithi123.stdout:(79/152): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-04-24T05:43:14.055 INFO:teuthology.orchestra.run.smithi123.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 232 kB/s | 35 kB 00:00 2024-04-24T05:43:14.105 INFO:teuthology.orchestra.run.smithi123.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-04-24T05:43:14.222 INFO:teuthology.orchestra.run.smithi123.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-24T05:43:14.256 INFO:teuthology.orchestra.run.smithi123.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-04-24T05:43:14.281 INFO:teuthology.orchestra.run.smithi123.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-24T05:43:14.298 INFO:teuthology.orchestra.run.smithi123.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-24T05:43:14.332 INFO:teuthology.orchestra.run.smithi123.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-24T05:43:14.357 INFO:teuthology.orchestra.run.smithi123.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-24T05:43:14.390 INFO:teuthology.orchestra.run.smithi123.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.8 MB/s | 279 kB 00:00 2024-04-24T05:43:14.416 INFO:teuthology.orchestra.run.smithi123.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 884 kB/s | 51 kB 00:00 2024-04-24T05:43:14.442 INFO:teuthology.orchestra.run.smithi123.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-24T05:43:14.483 INFO:teuthology.orchestra.run.smithi123.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-24T05:43:14.526 INFO:teuthology.orchestra.run.smithi123.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-04-24T05:43:14.576 INFO:teuthology.orchestra.run.smithi123.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 221 kB 00:00 2024-04-24T05:43:14.676 INFO:teuthology.orchestra.run.smithi123.stdout:(94/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.8 MB/s | 5.1 MB 00:00 2024-04-24T05:43:14.701 INFO:teuthology.orchestra.run.smithi123.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 289 kB/s | 36 kB 00:00 2024-04-24T05:43:14.726 INFO:teuthology.orchestra.run.smithi123.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 963 kB/s | 47 kB 00:00 2024-04-24T05:43:14.760 INFO:teuthology.orchestra.run.smithi123.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-24T05:43:14.785 INFO:teuthology.orchestra.run.smithi123.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-24T05:43:14.810 INFO:teuthology.orchestra.run.smithi123.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-24T05:43:14.861 INFO:teuthology.orchestra.run.smithi123.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 857 kB/s | 64 kB 00:00 2024-04-24T05:43:14.886 INFO:teuthology.orchestra.run.smithi123.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-04-24T05:43:15.203 INFO:teuthology.orchestra.run.smithi123.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-24T05:43:15.553 INFO:teuthology.orchestra.run.smithi123.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-04-24T05:43:15.620 INFO:teuthology.orchestra.run.smithi123.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 4.6 MB/s | 309 kB 00:00 2024-04-24T05:43:15.679 INFO:teuthology.orchestra.run.smithi123.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 337 kB/s | 19 kB 00:00 2024-04-24T05:43:15.704 INFO:teuthology.orchestra.run.smithi123.stdout:(106/152): lua-devel-5.4.4-4.el9.x86_64.rpm 26 kB/s | 22 kB 00:00 2024-04-24T05:43:15.946 INFO:teuthology.orchestra.run.smithi123.stdout:(107/152): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-24T05:43:15.971 INFO:teuthology.orchestra.run.smithi123.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 98 kB/s | 26 kB 00:00 2024-04-24T05:43:16.038 INFO:teuthology.orchestra.run.smithi123.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-24T05:43:16.230 INFO:teuthology.orchestra.run.smithi123.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 793 kB/s | 151 kB 00:00 2024-04-24T05:43:16.255 INFO:teuthology.orchestra.run.smithi123.stdout:(111/152): liboath-2.6.7-2.el9.x86_64.rpm 157 kB/s | 49 kB 00:00 2024-04-24T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.0 MB/s | 548 kB 00:00 2024-04-24T05:43:16.532 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-24T05:43:16.572 INFO:teuthology.orchestra.run.smithi123.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.4 MB/s | 838 kB 00:00 2024-04-24T05:43:16.656 INFO:teuthology.orchestra.run.smithi123.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 390 kB/s | 32 kB 00:00 2024-04-24T05:43:16.681 INFO:teuthology.orchestra.run.smithi123.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 275 kB/s | 43 kB 00:00 2024-04-24T05:43:16.715 INFO:teuthology.orchestra.run.smithi123.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 258 kB/s | 15 kB 00:00 2024-04-24T05:43:16.757 INFO:teuthology.orchestra.run.smithi123.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-24T05:43:16.799 INFO:teuthology.orchestra.run.smithi123.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-24T05:43:16.849 INFO:teuthology.orchestra.run.smithi123.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 718 kB/s | 35 kB 00:00 2024-04-24T05:43:16.891 INFO:teuthology.orchestra.run.smithi123.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-24T05:43:16.967 INFO:teuthology.orchestra.run.smithi123.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-24T05:43:17.017 INFO:teuthology.orchestra.run.smithi123.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.9 MB/s | 144 kB 00:00 2024-04-24T05:43:17.068 INFO:teuthology.orchestra.run.smithi123.stdout:(123/152): protobuf-compiler-3.14.0-13.el9.x86_ 396 kB/s | 863 kB 00:02 2024-04-24T05:43:17.093 INFO:teuthology.orchestra.run.smithi123.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 144 kB/s | 11 kB 00:00 2024-04-24T05:43:17.235 INFO:teuthology.orchestra.run.smithi123.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.9 MB/s | 2.0 MB 00:00 2024-04-24T05:43:17.251 INFO:teuthology.orchestra.run.smithi123.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 97 kB/s | 18 kB 00:00 2024-04-24T05:43:17.268 INFO:teuthology.orchestra.run.smithi123.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 220 kB/s | 23 kB 00:00 2024-04-24T05:43:17.294 INFO:teuthology.orchestra.run.smithi123.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 336 kB/s | 19 kB 00:00 2024-04-24T05:43:17.335 INFO:teuthology.orchestra.run.smithi123.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 299 kB/s | 20 kB 00:00 2024-04-24T05:43:17.353 INFO:teuthology.orchestra.run.smithi123.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 799 kB/s | 46 kB 00:00 2024-04-24T05:43:17.411 INFO:teuthology.orchestra.run.smithi123.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-24T05:43:17.436 INFO:teuthology.orchestra.run.smithi123.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 696 kB/s | 58 kB 00:00 2024-04-24T05:43:17.495 INFO:teuthology.orchestra.run.smithi123.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-04-24T05:43:17.537 INFO:teuthology.orchestra.run.smithi123.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2024-04-24T05:43:17.570 INFO:teuthology.orchestra.run.smithi123.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-24T05:43:17.596 INFO:teuthology.orchestra.run.smithi123.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 489 kB/s | 28 kB 00:00 2024-04-24T05:43:17.629 INFO:teuthology.orchestra.run.smithi123.stdout:(137/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-04-24T05:43:17.654 INFO:teuthology.orchestra.run.smithi123.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 371 kB/s | 31 kB 00:00 2024-04-24T05:43:17.688 INFO:teuthology.orchestra.run.smithi123.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-24T05:43:17.713 INFO:teuthology.orchestra.run.smithi123.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 712 kB/s | 59 kB 00:00 2024-04-24T05:43:17.739 INFO:teuthology.orchestra.run.smithi123.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 431 kB/s | 36 kB 00:00 2024-04-24T05:43:17.764 INFO:teuthology.orchestra.run.smithi123.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 686 kB/s | 51 kB 00:00 2024-04-24T05:43:17.789 INFO:teuthology.orchestra.run.smithi123.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-24T05:43:17.823 INFO:teuthology.orchestra.run.smithi123.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-24T05:43:17.848 INFO:teuthology.orchestra.run.smithi123.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 346 kB/s | 20 kB 00:00 2024-04-24T05:43:17.881 INFO:teuthology.orchestra.run.smithi123.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.3 MB/s | 192 kB 00:00 2024-04-24T05:43:17.925 INFO:teuthology.orchestra.run.smithi123.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-04-24T05:43:18.018 INFO:teuthology.orchestra.run.smithi123.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.4 MB/s | 1.6 MB 00:00 2024-04-24T05:43:18.234 INFO:teuthology.orchestra.run.smithi123.stdout:(149/152): librbd1-19.0.0-3124.ga3c8a22b.el9.x8 10 MB/s | 3.1 MB 00:00 2024-04-24T05:43:18.301 INFO:teuthology.orchestra.run.smithi123.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 4.8 MB/s | 1.3 MB 00:00 2024-04-24T05:43:18.560 INFO:teuthology.orchestra.run.smithi123.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 2.4 MB 00:00 2024-04-24T05:43:18.635 INFO:teuthology.orchestra.run.smithi123.stdout:(152/152): librados2-19.0.0-3124.ga3c8a22b.el9. 4.5 MB/s | 3.4 MB 00:00 2024-04-24T05:43:18.647 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-24T05:43:18.647 INFO:teuthology.orchestra.run.smithi123.stdout:Total 16 MB/s | 222 MB 00:14 2024-04-24T05:43:18.881 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T05:43:18.883 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T05:43:18.902 INFO:teuthology.orchestra.run.smithi155.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T05:43:18.904 INFO:teuthology.orchestra.run.smithi155.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T05:43:18.905 INFO:teuthology.orchestra.run.smithi155.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T05:43:18.906 INFO:teuthology.orchestra.run.smithi155.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T05:43:18.908 INFO:teuthology.orchestra.run.smithi155.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T05:43:18.911 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T05:43:18.913 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T05:43:18.982 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-04-24T05:43:19.001 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================= 2024-04-24T05:43:19.001 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-04-24T05:43:19.001 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================= 2024-04-24T05:43:19.001 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-04-24T05:43:19.001 INFO:teuthology.orchestra.run.smithi155.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T05:43:19.001 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 6.6 k 2024-04-24T05:43:19.001 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 5.4 M 2024-04-24T05:43:19.001 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 890 k 2024-04-24T05:43:19.002 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 145 k 2024-04-24T05:43:19.002 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 1.5 M 2024-04-24T05:43:19.002 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 142 k 2024-04-24T05:43:19.002 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 3.5 M 2024-04-24T05:43:19.002 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 7.4 M 2024-04-24T05:43:19.002 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 49 k 2024-04-24T05:43:19.002 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 11 M 2024-04-24T05:43:19.002 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 49 M 2024-04-24T05:43:19.002 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 273 k 2024-04-24T05:43:19.003 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 762 k 2024-04-24T05:43:19.003 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 224 k 2024-04-24T05:43:19.003 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-shell x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 27 k 2024-04-24T05:43:19.003 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 18 k 2024-04-24T05:43:19.003 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T05:43:19.003 INFO:teuthology.orchestra.run.smithi155.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T05:43:19.003 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T05:43:19.003 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 32 k 2024-04-24T05:43:19.004 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 747 k 2024-04-24T05:43:19.004 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 127 k 2024-04-24T05:43:19.004 INFO:teuthology.orchestra.run.smithi155.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T05:43:19.004 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T05:43:19.004 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T05:43:19.004 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T05:43:19.004 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T05:43:19.005 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T05:43:19.005 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 162 k 2024-04-24T05:43:19.005 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 322 k 2024-04-24T05:43:19.005 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 297 k 2024-04-24T05:43:19.005 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 99 k 2024-04-24T05:43:19.005 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 85 k 2024-04-24T05:43:19.005 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.1 M 2024-04-24T05:43:19.005 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 170 k 2024-04-24T05:43:19.006 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T05:43:19.006 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2024-04-24T05:43:19.006 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.4 M 2024-04-24T05:43:19.006 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.1 M 2024-04-24T05:43:19.006 INFO:teuthology.orchestra.run.smithi155.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T05:43:19.006 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T05:43:19.006 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-04-24T05:43:19.006 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T05:43:19.006 INFO:teuthology.orchestra.run.smithi155.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T05:43:19.007 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T05:43:19.007 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 21 M 2024-04-24T05:43:19.007 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 34 k 2024-04-24T05:43:19.007 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 2.4 M 2024-04-24T05:43:19.007 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 249 k 2024-04-24T05:43:19.007 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 4.7 M 2024-04-24T05:43:19.007 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 17 M 2024-04-24T05:43:19.007 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 16 k 2024-04-24T05:43:19.007 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 25 k 2024-04-24T05:43:19.007 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T05:43:19.007 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T05:43:19.007 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T05:43:19.007 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T05:43:19.008 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T05:43:19.008 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T05:43:19.008 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T05:43:19.008 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T05:43:19.008 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 163 k 2024-04-24T05:43:19.008 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T05:43:19.008 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T05:43:19.008 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T05:43:19.008 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T05:43:19.008 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T05:43:19.008 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 501 k 2024-04-24T05:43:19.008 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T05:43:19.008 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 5.4 M 2024-04-24T05:43:19.008 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T05:43:19.009 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T05:43:19.010 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T05:43:19.010 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 45 k 2024-04-24T05:43:19.010 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 135 k 2024-04-24T05:43:19.010 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T05:43:19.010 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T05:43:19.010 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T05:43:19.010 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T05:43:19.010 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T05:43:19.010 INFO:teuthology.orchestra.run.smithi155.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T05:43:19.010 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T05:43:19.010 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T05:43:19.010 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T05:43:19.010 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T05:43:19.011 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T05:43:19.011 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T05:43:19.011 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T05:43:19.011 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T05:43:19.011 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T05:43:19.011 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T05:43:19.011 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T05:43:19.011 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T05:43:19.011 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T05:43:19.011 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T05:43:19.011 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T05:43:19.011 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T05:43:19.011 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T05:43:19.011 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T05:43:19.012 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T05:43:19.012 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T05:43:19.012 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T05:43:19.012 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T05:43:19.012 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T05:43:19.012 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T05:43:19.012 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T05:43:19.012 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T05:43:19.012 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T05:43:19.012 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T05:43:19.012 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T05:43:19.012 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T05:43:19.012 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T05:43:19.012 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T05:43:19.013 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T05:43:19.013 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T05:43:19.013 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T05:43:19.013 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T05:43:19.013 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T05:43:19.013 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T05:43:19.013 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T05:43:19.013 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T05:43:19.013 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T05:43:19.013 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T05:43:19.013 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T05:43:19.013 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T05:43:19.013 INFO:teuthology.orchestra.run.smithi155.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T05:43:19.013 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T05:43:19.014 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T05:43:19.014 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T05:43:19.014 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T05:43:19.014 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T05:43:19.014 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T05:43:19.014 INFO:teuthology.orchestra.run.smithi155.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T05:43:19.014 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T05:43:19.014 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T05:43:19.014 INFO:teuthology.orchestra.run.smithi155.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T05:43:19.014 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T05:43:19.014 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T05:43:19.015 INFO:teuthology.orchestra.run.smithi155.stdout:Installing weak dependencies: 2024-04-24T05:43:19.015 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T05:43:19.015 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:43:19.015 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-04-24T05:43:19.015 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================= 2024-04-24T05:43:19.015 INFO:teuthology.orchestra.run.smithi155.stdout:Install 148 Packages 2024-04-24T05:43:19.015 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 4 Packages 2024-04-24T05:43:19.015 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:43:19.016 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 222 M 2024-04-24T05:43:19.016 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-04-24T05:43:20.200 INFO:teuthology.orchestra.run.smithi155.stdout:(1/152): ceph-19.0.0-3124.ga3c8a22b.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-24T05:43:20.450 INFO:teuthology.orchestra.run.smithi155.stdout:(2/152): ceph-fuse-19.0.0-3124.ga3c8a22b.el9.x8 3.5 MB/s | 890 kB 00:00 2024-04-24T05:43:20.509 INFO:teuthology.orchestra.run.smithi155.stdout:(3/152): ceph-immutable-object-cache-19.0.0-312 2.5 MB/s | 145 kB 00:00 2024-04-24T05:43:20.834 INFO:teuthology.orchestra.run.smithi155.stdout:(4/152): ceph-base-19.0.0-3124.ga3c8a22b.el9.x8 6.0 MB/s | 5.4 MB 00:00 2024-04-24T05:43:20.909 INFO:teuthology.orchestra.run.smithi155.stdout:(5/152): ceph-mds-19.0.0-3124.ga3c8a22b.el9.x86 5.9 MB/s | 2.4 MB 00:00 2024-04-24T05:43:21.026 INFO:teuthology.orchestra.run.smithi155.stdout:(6/152): ceph-mgr-19.0.0-3124.ga3c8a22b.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-04-24T05:43:21.495 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-24T05:43:21.501 INFO:teuthology.orchestra.run.smithi155.stdout:(7/152): ceph-mon-19.0.0-3124.ga3c8a22b.el9.x86 8.0 MB/s | 4.7 MB 00:00 2024-04-24T05:43:21.733 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-24T05:43:21.733 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-24T05:43:22.752 INFO:teuthology.orchestra.run.smithi155.stdout:(8/152): ceph-radosgw-19.0.0-3124.ga3c8a22b.el9 8.5 MB/s | 11 MB 00:01 2024-04-24T05:43:22.969 INFO:teuthology.orchestra.run.smithi155.stdout:(9/152): ceph-osd-19.0.0-3124.ga3c8a22b.el9.x86 8.7 MB/s | 17 MB 00:01 2024-04-24T05:43:22.994 INFO:teuthology.orchestra.run.smithi155.stdout:(10/152): ceph-selinux-19.0.0-3124.ga3c8a22b.el 104 kB/s | 25 kB 00:00 2024-04-24T05:43:23.061 INFO:teuthology.orchestra.run.smithi155.stdout:(11/152): cephfs-mirror-19.0.0-3124.ga3c8a22b.e 3.4 MB/s | 224 kB 00:00 2024-04-24T05:43:23.144 INFO:teuthology.orchestra.run.smithi155.stdout:(12/152): cephfs-shell-19.0.0-3124.ga3c8a22b.el 327 kB/s | 27 kB 00:00 2024-04-24T05:43:23.411 INFO:teuthology.orchestra.run.smithi155.stdout:(13/152): ceph-common-19.0.0-3124.ga3c8a22b.el9 6.1 MB/s | 21 MB 00:03 2024-04-24T05:43:23.437 INFO:teuthology.orchestra.run.smithi155.stdout:(14/152): libcephfs-devel-19.0.0-3124.ga3c8a22b 109 kB/s | 32 kB 00:00 2024-04-24T05:43:23.495 INFO:teuthology.orchestra.run.smithi155.stdout:(15/152): libcephsqlite-19.0.0-3124.ga3c8a22b.e 2.8 MB/s | 163 kB 00:00 2024-04-24T05:43:23.579 INFO:teuthology.orchestra.run.smithi155.stdout:(16/152): librados-devel-19.0.0-3124.ga3c8a22b. 1.5 MB/s | 127 kB 00:00 2024-04-24T05:43:23.620 INFO:teuthology.orchestra.run.smithi155.stdout:(17/152): libcephfs2-19.0.0-3124.ga3c8a22b.el9. 3.5 MB/s | 747 kB 00:00 2024-04-24T05:43:23.696 INFO:teuthology.orchestra.run.smithi155.stdout:(18/152): libradosstriper1-19.0.0-3124.ga3c8a22 4.2 MB/s | 501 kB 00:00 2024-04-24T05:43:23.780 INFO:teuthology.orchestra.run.smithi155.stdout:(19/152): python3-ceph-argparse-19.0.0-3124.ga3 547 kB/s | 45 kB 00:00 2024-04-24T05:43:23.855 INFO:teuthology.orchestra.run.smithi155.stdout:(20/152): python3-ceph-common-19.0.0-3124.ga3c8 1.8 MB/s | 135 kB 00:00 2024-04-24T05:43:23.947 INFO:teuthology.orchestra.run.smithi155.stdout:(21/152): python3-cephfs-19.0.0-3124.ga3c8a22b. 1.7 MB/s | 162 kB 00:00 2024-04-24T05:43:24.039 INFO:teuthology.orchestra.run.smithi155.stdout:(22/152): python3-rados-19.0.0-3124.ga3c8a22b.e 3.5 MB/s | 322 kB 00:00 2024-04-24T05:43:24.122 INFO:teuthology.orchestra.run.smithi155.stdout:(23/152): python3-rbd-19.0.0-3124.ga3c8a22b.el9 3.5 MB/s | 297 kB 00:00 2024-04-24T05:43:24.239 INFO:teuthology.orchestra.run.smithi155.stdout:(24/152): librgw2-19.0.0-3124.ga3c8a22b.el9.x86 8.7 MB/s | 5.4 MB 00:00 2024-04-24T05:43:24.265 INFO:teuthology.orchestra.run.smithi155.stdout:(25/152): python3-rgw-19.0.0-3124.ga3c8a22b.el9 704 kB/s | 99 kB 00:00 2024-04-24T05:43:24.315 INFO:teuthology.orchestra.run.smithi155.stdout:(26/152): rbd-fuse-19.0.0-3124.ga3c8a22b.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-24T05:43:24.407 INFO:teuthology.orchestra.run.smithi155.stdout:(27/152): rbd-nbd-19.0.0-3124.ga3c8a22b.el9.x86 1.8 MB/s | 170 kB 00:00 2024-04-24T05:43:24.471 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-24T05:43:24.472 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-24T05:43:24.490 INFO:teuthology.orchestra.run.smithi155.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3124.g 415 kB/s | 34 kB 00:00 2024-04-24T05:43:24.599 INFO:teuthology.orchestra.run.smithi155.stdout:(29/152): rbd-mirror-19.0.0-3124.ga3c8a22b.el9. 9.3 MB/s | 3.1 MB 00:00 2024-04-24T05:43:24.633 INFO:teuthology.orchestra.run.smithi155.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3124.ga3c8a22 1.0 MB/s | 142 kB 00:00 2024-04-24T05:43:25.033 INFO:teuthology.orchestra.run.smithi155.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3124.ga3c8a 8.2 MB/s | 3.5 MB 00:00 2024-04-24T05:43:25.116 INFO:teuthology.orchestra.run.smithi155.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3124.ga3 3.0 MB/s | 249 kB 00:00 2024-04-24T05:43:25.183 INFO:teuthology.orchestra.run.smithi155.stdout:(33/152): ceph-mgr-rook-19.0.0-3124.ga3c8a22b.e 748 kB/s | 49 kB 00:00 2024-04-24T05:43:25.250 INFO:teuthology.orchestra.run.smithi155.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3124.ga 251 kB/s | 16 kB 00:00 2024-04-24T05:43:25.326 INFO:teuthology.orchestra.run.smithi155.stdout:(35/152): ceph-volume-19.0.0-3124.ga3c8a22b.el9 3.6 MB/s | 273 kB 00:00 2024-04-24T05:43:25.468 INFO:teuthology.orchestra.run.smithi155.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 8.9 MB/s | 7.4 MB 00:00 2024-04-24T05:43:25.534 INFO:teuthology.orchestra.run.smithi155.stdout:(37/152): cephadm-19.0.0-3124.ga3c8a22b.el9.noa 3.6 MB/s | 762 kB 00:00 2024-04-24T05:43:25.560 INFO:teuthology.orchestra.run.smithi155.stdout:(38/152): cephfs-top-19.0.0-3124.ga3c8a22b.el9. 201 kB/s | 18 kB 00:00 2024-04-24T05:43:25.823 INFO:teuthology.orchestra.run.smithi155.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 279 kB/s | 80 kB 00:00 2024-04-24T05:43:25.985 INFO:teuthology.orchestra.run.smithi155.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 803 kB 00:00 2024-04-24T05:43:26.110 INFO:teuthology.orchestra.run.smithi155.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 685 kB/s | 195 kB 00:00 2024-04-24T05:43:26.236 INFO:teuthology.orchestra.run.smithi155.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-04-24T05:43:26.378 INFO:teuthology.orchestra.run.smithi155.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 4.6 MB/s | 1.2 MB 00:00 2024-04-24T05:43:26.545 INFO:teuthology.orchestra.run.smithi155.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 346 kB/s | 106 kB 00:00 2024-04-24T05:43:26.645 INFO:teuthology.orchestra.run.smithi155.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 508 kB/s | 135 kB 00:00 2024-04-24T05:43:26.787 INFO:teuthology.orchestra.run.smithi155.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 520 kB/s | 125 kB 00:00 2024-04-24T05:43:27.437 INFO:teuthology.orchestra.run.smithi155.stdout:(47/152): ceph-test-19.0.0-3124.ga3c8a22b.el9.x 11 MB/s | 49 MB 00:04 2024-04-24T05:43:27.505 INFO:teuthology.orchestra.run.smithi155.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 251 kB/s | 215 kB 00:00 2024-04-24T05:43:27.763 INFO:teuthology.orchestra.run.smithi155.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 416 kB/s | 107 kB 00:00 2024-04-24T05:43:27.873 INFO:teuthology.orchestra.run.smithi155.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 197 kB/s | 21 kB 00:00 2024-04-24T05:43:27.922 INFO:teuthology.orchestra.run.smithi155.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 608 kB/s | 689 kB 00:01 2024-04-24T05:43:27.980 INFO:teuthology.orchestra.run.smithi155.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 1.7 MB/s | 943 kB 00:00 2024-04-24T05:43:28.014 INFO:teuthology.orchestra.run.smithi155.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 358 kB/s | 33 kB 00:00 2024-04-24T05:43:28.097 INFO:teuthology.orchestra.run.smithi155.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 220 kB/s | 18 kB 00:00 2024-04-24T05:43:28.131 INFO:teuthology.orchestra.run.smithi155.stdout:(55/152): flex-2.6.4-9.el9.x86_64.rpm 1.2 MB/s | 313 kB 00:00 2024-04-24T05:43:28.181 INFO:teuthology.orchestra.run.smithi155.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 134 kB/s | 11 kB 00:00 2024-04-24T05:43:28.248 INFO:teuthology.orchestra.run.smithi155.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 684 kB/s | 45 kB 00:00 2024-04-24T05:43:28.282 INFO:teuthology.orchestra.run.smithi155.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-04-24T05:43:28.690 INFO:teuthology.orchestra.run.smithi155.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.5 MB/s | 662 kB 00:00 2024-04-24T05:43:28.723 INFO:teuthology.orchestra.run.smithi155.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 1.3 MB/s | 582 kB 00:00 2024-04-24T05:43:28.816 INFO:teuthology.orchestra.run.smithi155.stdout:(61/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.6 MB/s | 3.0 MB 00:00 2024-04-24T05:43:28.841 INFO:teuthology.orchestra.run.smithi155.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 170 kB/s | 25 kB 00:00 2024-04-24T05:43:28.866 INFO:teuthology.orchestra.run.smithi155.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 334 kB/s | 47 kB 00:00 2024-04-24T05:43:28.941 INFO:teuthology.orchestra.run.smithi155.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 474 kB/s | 35 kB 00:00 2024-04-24T05:43:28.975 INFO:teuthology.orchestra.run.smithi155.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-04-24T05:43:29.075 INFO:teuthology.orchestra.run.smithi155.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.1 MB/s | 292 kB 00:00 2024-04-24T05:43:29.775 INFO:teuthology.orchestra.run.smithi155.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 2024-04-24T05:43:30.151 INFO:teuthology.orchestra.run.smithi155.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 2.4 MB/s | 911 kB 00:00 2024-04-24T05:43:30.242 INFO:teuthology.orchestra.run.smithi155.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-04-24T05:43:30.334 INFO:teuthology.orchestra.run.smithi155.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 3.4 MB/s | 4.7 MB 00:01 2024-04-24T05:43:30.360 INFO:teuthology.orchestra.run.smithi155.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 641 kB/s | 133 kB 00:00 2024-04-24T05:43:30.427 INFO:teuthology.orchestra.run.smithi155.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-24T05:43:30.652 INFO:teuthology.orchestra.run.smithi155.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.2 MB/s | 1.0 MB 00:00 2024-04-24T05:43:30.752 INFO:teuthology.orchestra.run.smithi155.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-04-24T05:43:30.828 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-24T05:43:30.860 INFO:teuthology.orchestra.run.smithi155.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.3 MB/s | 248 kB 00:00 2024-04-24T05:43:30.919 INFO:teuthology.orchestra.run.smithi155.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 443 kB/s | 26 kB 00:00 2024-04-24T05:43:30.932 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T05:43:30.978 INFO:teuthology.orchestra.run.smithi155.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 333 kB/s | 19 kB 00:00 2024-04-24T05:43:31.035 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T05:43:31.078 INFO:teuthology.orchestra.run.smithi155.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-04-24T05:43:31.137 INFO:teuthology.orchestra.run.smithi155.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 600 kB/s | 35 kB 00:00 2024-04-24T05:43:31.143 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T05:43:31.203 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T05:43:31.243 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T05:43:31.389 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T05:43:31.604 INFO:teuthology.orchestra.run.smithi155.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 4.2 MB/s | 5.8 MB 00:01 2024-04-24T05:43:31.729 INFO:teuthology.orchestra.run.smithi155.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-04-24T05:43:31.821 INFO:teuthology.orchestra.run.smithi155.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-24T05:43:31.828 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T05:43:31.889 INFO:teuthology.orchestra.run.smithi155.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-24T05:43:31.949 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T05:43:31.955 INFO:teuthology.orchestra.run.smithi155.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 643 kB/s | 42 kB 00:00 2024-04-24T05:43:32.063 INFO:teuthology.orchestra.run.smithi155.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-04-24T05:43:32.129 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T05:43:32.130 INFO:teuthology.orchestra.run.smithi155.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 159 kB 00:00 2024-04-24T05:43:32.158 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T05:43:32.231 INFO:teuthology.orchestra.run.smithi155.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-04-24T05:43:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T05:43:32.289 INFO:teuthology.orchestra.run.smithi155.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 885 kB/s | 51 kB 00:00 2024-04-24T05:43:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 10/156 2024-04-24T05:43:32.440 INFO:teuthology.orchestra.run.smithi155.stdout:(89/152): python3-babel-2.9.1-2.el9.noarch.rpm 3.0 MB/s | 6.0 MB 00:02 2024-04-24T05:43:32.465 INFO:teuthology.orchestra.run.smithi155.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 307 kB/s | 54 kB 00:00 2024-04-24T05:43:32.565 INFO:teuthology.orchestra.run.smithi155.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.6 MB/s | 5.1 MB 00:01 2024-04-24T05:43:32.590 INFO:teuthology.orchestra.run.smithi155.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 334 kB/s | 42 kB 00:00 2024-04-24T05:43:32.632 INFO:teuthology.orchestra.run.smithi155.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 668 kB/s | 44 kB 00:00 2024-04-24T05:43:32.675 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T05:43:32.707 INFO:teuthology.orchestra.run.smithi155.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.9 MB/s | 221 kB 00:00 2024-04-24T05:43:32.732 INFO:teuthology.orchestra.run.smithi155.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 362 kB/s | 36 kB 00:00 2024-04-24T05:43:32.737 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T05:43:32.774 INFO:teuthology.orchestra.run.smithi155.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 718 kB/s | 47 kB 00:00 2024-04-24T05:43:32.787 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T05:43:32.850 INFO:teuthology.orchestra.run.smithi155.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-04-24T05:43:32.875 INFO:teuthology.orchestra.run.smithi155.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 583 kB/s | 58 kB 00:00 2024-04-24T05:43:32.942 INFO:teuthology.orchestra.run.smithi155.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-24T05:43:32.942 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T05:43:32.967 INFO:teuthology.orchestra.run.smithi155.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 703 kB/s | 64 kB 00:00 2024-04-24T05:43:32.971 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 14/156 2024-04-24T05:43:33.001 INFO:teuthology.orchestra.run.smithi155.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 778 kB/s | 45 kB 00:00 2024-04-24T05:43:33.058 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 14/156 2024-04-24T05:43:33.089 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T05:43:33.161 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T05:43:33.471 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T05:43:33.523 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T05:43:33.651 INFO:teuthology.orchestra.run.smithi155.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 33 kB/s | 22 kB 00:00 2024-04-24T05:43:33.706 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T05:43:33.826 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T05:43:33.865 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T05:43:33.912 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T05:43:33.926 INFO:teuthology.orchestra.run.smithi155.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-04-24T05:43:33.986 INFO:teuthology.orchestra.run.smithi155.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-04-24T05:43:33.997 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T05:43:34.027 INFO:teuthology.orchestra.run.smithi155.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 487 kB/s | 19 kB 00:00 2024-04-24T05:43:34.052 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T05:43:34.106 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T05:43:34.144 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T05:43:34.178 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T05:43:34.202 INFO:teuthology.orchestra.run.smithi155.stdout:(106/152): libarrow-9.0.0-10.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-04-24T05:43:34.244 INFO:teuthology.orchestra.run.smithi155.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 632 kB/s | 26 kB 00:00 2024-04-24T05:43:34.284 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T05:43:34.286 INFO:teuthology.orchestra.run.smithi155.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 1.2 MB/s | 49 kB 00:00 2024-04-24T05:43:34.336 INFO:teuthology.orchestra.run.smithi155.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-24T05:43:34.349 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T05:43:34.381 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T05:43:34.386 INFO:teuthology.orchestra.run.smithi155.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-24T05:43:34.435 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T05:43:34.462 INFO:teuthology.orchestra.run.smithi155.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-04-24T05:43:34.529 INFO:teuthology.orchestra.run.smithi155.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-04-24T05:43:34.543 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T05:43:34.576 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9. 31/156 2024-04-24T05:43:34.579 INFO:teuthology.orchestra.run.smithi155.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 878 kB/s | 43 kB 00:00 2024-04-24T05:43:34.630 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el 32/156 2024-04-24T05:43:34.630 INFO:teuthology.orchestra.run.smithi155.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 652 kB/s | 32 kB 00:00 2024-04-24T05:43:34.666 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_6 33/156 2024-04-24T05:43:34.680 INFO:teuthology.orchestra.run.smithi155.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 304 kB/s | 15 kB 00:00 2024-04-24T05:43:34.730 INFO:teuthology.orchestra.run.smithi155.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-24T05:43:34.779 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T05:43:34.788 INFO:teuthology.orchestra.run.smithi155.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 6.1 MB/s | 358 kB 00:00 2024-04-24T05:43:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T05:43:34.847 INFO:teuthology.orchestra.run.smithi155.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-04-24T05:43:34.906 INFO:teuthology.orchestra.run.smithi155.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 611 kB/s | 35 kB 00:00 2024-04-24T05:43:34.935 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T05:43:34.956 INFO:teuthology.orchestra.run.smithi155.stdout:(120/152): python3-google-auth-2.29.0-1.el9.noa 4.1 MB/s | 206 kB 00:00 2024-04-24T05:43:34.999 INFO:teuthology.orchestra.run.smithi155.stdout:(121/152): protobuf-compiler-3.14.0-13.el9.x86_ 432 kB/s | 863 kB 00:01 2024-04-24T05:43:35.091 INFO:teuthology.orchestra.run.smithi155.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-04-24T05:43:35.110 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T05:43:35.115 INFO:teuthology.orchestra.run.smithi155.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-24T05:43:35.141 INFO:teuthology.orchestra.run.smithi155.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 223 kB/s | 11 kB 00:00 2024-04-24T05:43:35.166 INFO:teuthology.orchestra.run.smithi155.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 356 kB/s | 18 kB 00:00 2024-04-24T05:43:35.181 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T05:43:35.192 INFO:teuthology.orchestra.run.smithi155.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 471 kB/s | 23 kB 00:00 2024-04-24T05:43:35.217 INFO:teuthology.orchestra.run.smithi155.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 395 kB/s | 19 kB 00:00 2024-04-24T05:43:35.242 INFO:teuthology.orchestra.run.smithi155.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 400 kB/s | 20 kB 00:00 2024-04-24T05:43:35.292 INFO:teuthology.orchestra.run.smithi155.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-04-24T05:43:35.318 INFO:teuthology.orchestra.run.smithi155.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 714 kB/s | 46 kB 00:00 2024-04-24T05:43:35.343 INFO:teuthology.orchestra.run.smithi155.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2024-04-24T05:43:35.368 INFO:teuthology.orchestra.run.smithi155.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.2 MB/s | 58 kB 00:00 2024-04-24T05:43:35.402 INFO:teuthology.orchestra.run.smithi155.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.7 MB/s | 272 kB 00:00 2024-04-24T05:43:35.435 INFO:teuthology.orchestra.run.smithi155.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 258 kB/s | 16 kB 00:00 2024-04-24T05:43:35.461 INFO:teuthology.orchestra.run.smithi155.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-24T05:43:35.486 INFO:teuthology.orchestra.run.smithi155.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 582 kB/s | 28 kB 00:00 2024-04-24T05:43:35.528 INFO:teuthology.orchestra.run.smithi155.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 468 kB/s | 31 kB 00:00 2024-04-24T05:43:35.562 INFO:teuthology.orchestra.run.smithi155.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-24T05:43:35.587 INFO:teuthology.orchestra.run.smithi155.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-24T05:43:35.612 INFO:teuthology.orchestra.run.smithi155.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 728 kB/s | 36 kB 00:00 2024-04-24T05:43:35.638 INFO:teuthology.orchestra.run.smithi155.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-24T05:43:35.671 INFO:teuthology.orchestra.run.smithi155.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-04-24T05:43:35.696 INFO:teuthology.orchestra.run.smithi155.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.6 MB/s | 90 kB 00:00 2024-04-24T05:43:35.730 INFO:teuthology.orchestra.run.smithi155.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 7.3 MB/s | 427 kB 00:00 2024-04-24T05:43:35.755 INFO:teuthology.orchestra.run.smithi155.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 368 kB/s | 20 kB 00:00 2024-04-24T05:43:35.789 INFO:teuthology.orchestra.run.smithi155.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-24T05:43:35.856 INFO:teuthology.orchestra.run.smithi155.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-24T05:43:35.857 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T05:43:35.952 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T05:43:35.989 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T05:43:36.034 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T05:43:36.074 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_6 43/156 2024-04-24T05:43:36.142 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T05:43:36.156 INFO:teuthology.orchestra.run.smithi155.stdout:(148/152): librados2-19.0.0-3124.ga3c8a22b.el9. 9.7 MB/s | 3.4 MB 00:00 2024-04-24T05:43:36.281 INFO:teuthology.orchestra.run.smithi155.stdout:(149/152): librbd1-19.0.0-3124.ga3c8a22b.el9.x8 7.4 MB/s | 3.1 MB 00:00 2024-04-24T05:43:36.507 INFO:teuthology.orchestra.run.smithi155.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.9 MB/s | 1.3 MB 00:00 2024-04-24T05:43:36.606 INFO:teuthology.orchestra.run.smithi155.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 2.4 MB 00:00 2024-04-24T05:43:36.633 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T05:43:36.694 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T05:43:36.774 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T05:43:36.873 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T05:43:36.917 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T05:43:36.957 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T05:43:37.007 INFO:teuthology.orchestra.run.smithi155.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.6 MB/s | 16 MB 00:04 2024-04-24T05:43:37.016 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-04-24T05:43:37.017 INFO:teuthology.orchestra.run.smithi155.stdout:Total 12 MB/s | 222 MB 00:18 2024-04-24T05:43:37.026 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T05:43:37.073 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T05:43:37.136 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T05:43:37.200 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T05:43:37.422 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T05:43:37.478 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T05:43:37.517 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T05:43:37.575 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T05:43:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T05:43:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T05:43:37.798 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T05:43:37.830 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T05:43:37.863 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T05:43:38.029 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T05:43:38.206 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T05:43:38.253 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T05:43:38.298 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T05:43:38.347 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T05:43:38.416 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T05:43:38.475 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T05:43:38.523 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T05:43:38.571 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T05:43:38.613 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T05:43:38.684 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T05:43:38.741 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T05:43:38.867 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T05:43:38.921 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T05:43:38.965 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T05:43:39.024 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T05:43:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T05:43:39.238 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T05:43:39.870 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-04-24T05:43:40.113 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-04-24T05:43:40.114 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-04-24T05:43:40.186 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T05:43:40.237 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T05:43:40.270 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T05:43:40.456 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T05:43:40.552 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T05:43:40.596 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T05:43:40.822 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T05:43:40.860 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T05:43:40.964 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T05:43:41.774 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T05:43:41.914 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T05:43:42.878 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-04-24T05:43:42.882 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-04-24T05:43:43.490 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T05:43:43.633 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T05:43:43.676 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T05:43:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 99/156 2024-04-24T05:43:44.224 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 99/156 2024-04-24T05:43:44.269 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 100/156 2024-04-24T05:43:44.317 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T05:43:44.863 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T05:43:44.898 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 103/156 2024-04-24T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 103/156 2024-04-24T05:43:45.025 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 104/156 2024-04-24T05:43:47.706 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T05:43:47.746 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T05:43:48.404 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T05:43:48.449 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 106/156 2024-04-24T05:43:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 106/156 2024-04-24T05:43:48.606 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T05:43:48.606 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T05:43:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:43:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 107/156 2024-04-24T05:43:49.264 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-04-24T05:43:49.398 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T05:43:49.462 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T05:43:49.766 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T05:43:49.826 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T05:43:49.867 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T05:43:50.005 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T05:43:50.446 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T05:43:50.547 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T05:43:50.728 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T05:43:50.757 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T05:43:50.880 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T05:43:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 10/156 2024-04-24T05:43:51.231 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T05:43:51.294 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T05:43:51.319 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T05:43:51.467 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T05:43:51.520 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 14/156 2024-04-24T05:43:51.616 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 14/156 2024-04-24T05:43:51.646 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T05:43:51.726 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T05:43:52.052 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T05:43:52.113 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T05:43:52.312 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T05:43:52.441 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T05:43:52.488 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T05:43:52.536 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T05:43:52.640 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T05:43:52.691 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T05:43:52.753 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T05:43:52.804 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T05:43:52.876 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T05:43:53.000 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T05:43:53.075 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T05:43:53.113 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T05:43:53.171 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T05:43:53.280 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T05:43:53.341 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9. 31/156 2024-04-24T05:43:53.397 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el 32/156 2024-04-24T05:43:53.440 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_6 33/156 2024-04-24T05:43:53.563 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T05:43:53.617 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T05:43:53.727 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T05:43:53.944 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T05:43:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T05:43:54.728 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T05:43:54.826 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T05:43:54.870 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T05:43:54.915 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T05:43:54.955 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_6 43/156 2024-04-24T05:43:55.029 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T05:43:55.283 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T05:43:55.347 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T05:43:55.419 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T05:43:55.519 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T05:43:55.556 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T05:43:55.619 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T05:43:55.688 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T05:43:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T05:43:55.791 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T05:43:55.856 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T05:43:56.055 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T05:43:56.109 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T05:43:56.146 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T05:43:56.203 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T05:43:56.335 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T05:43:56.388 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T05:43:56.437 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T05:43:56.495 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T05:43:56.528 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T05:43:56.729 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T05:43:56.909 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T05:43:56.975 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T05:43:57.020 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T05:43:57.072 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T05:43:57.140 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T05:43:57.197 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T05:43:57.255 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T05:43:57.309 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T05:43:57.359 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T05:43:57.453 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T05:43:57.515 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T05:43:57.660 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T05:43:57.738 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T05:43:57.781 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T05:43:57.851 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T05:43:58.007 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T05:43:58.069 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T05:43:59.047 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T05:43:59.103 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T05:43:59.134 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T05:43:59.258 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T05:43:59.320 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T05:43:59.425 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T05:43:59.476 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T05:43:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T05:43:59.748 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T05:43:59.866 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T05:44:00.718 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T05:44:00.855 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T05:44:01.753 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 107/156 2024-04-24T05:44:01.753 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-04-24T05:44:01.753 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-04-24T05:44:01.753 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-04-24T05:44:01.753 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-04-24T05:44:01.753 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-04-24T05:44:01.753 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-04-24T05:44:01.753 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-04-24T05:44:01.753 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:01.996 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 108/156 2024-04-24T05:44:02.045 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 108/156 2024-04-24T05:44:02.045 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:02.045 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T05:44:02.046 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:02.491 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T05:44:02.604 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T05:44:02.654 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T05:44:02.709 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T05:44:03.104 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T05:44:03.149 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 99/156 2024-04-24T05:44:03.302 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 99/156 2024-04-24T05:44:03.340 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 100/156 2024-04-24T05:44:03.410 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T05:44:03.683 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 109/156 2024-04-24T05:44:03.783 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 109/156 2024-04-24T05:44:03.783 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:03.783 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T05:44:03.783 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T05:44:03.783 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T05:44:03.783 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:03.960 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T05:44:03.993 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 103/156 2024-04-24T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 110/156 2024-04-24T05:44:04.084 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 103/156 2024-04-24T05:44:04.112 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 104/156 2024-04-24T05:44:04.127 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 110/156 2024-04-24T05:44:04.128 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:04.128 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T05:44:04.128 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T05:44:04.128 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T05:44:04.128 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:04.616 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 111/156 2024-04-24T05:44:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 111/156 2024-04-24T05:44:04.692 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:04.692 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T05:44:04.692 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T05:44:04.692 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T05:44:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:04.824 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T05:44:04.885 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T05:44:04.960 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T05:44:05.192 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T05:44:05.254 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T05:44:05.344 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T05:44:05.583 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T05:44:05.749 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T05:44:05.794 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T05:44:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T05:44:05.877 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T05:44:06.451 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 123/156 2024-04-24T05:44:06.496 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 123/156 2024-04-24T05:44:06.549 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.e 124/156 2024-04-24T05:44:06.660 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b. 125/156 2024-04-24T05:44:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 126/156 2024-04-24T05:44:06.952 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T05:44:06.993 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T05:44:07.671 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T05:44:07.712 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 106/156 2024-04-24T05:44:07.939 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 106/156 2024-04-24T05:44:07.939 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T05:44:07.940 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T05:44:07.940 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:07.997 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 107/156 2024-04-24T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 126/156 2024-04-24T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 127/156 2024-04-24T05:44:09.136 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 127/156 2024-04-24T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 128/156 2024-04-24T05:44:09.312 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 128/156 2024-04-24T05:44:09.482 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el 129/156 2024-04-24T05:44:09.510 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 130/156 2024-04-24T05:44:09.584 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 130/156 2024-04-24T05:44:09.584 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:09.584 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T05:44:09.584 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T05:44:09.585 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T05:44:09.585 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:09.635 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 131/156 2024-04-24T05:44:09.673 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 131/156 2024-04-24T05:44:09.784 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 132/156 2024-04-24T05:44:09.870 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 133/156 2024-04-24T05:44:10.900 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T05:44:10.938 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 135/156 2024-04-24T05:44:11.007 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 135/156 2024-04-24T05:44:11.007 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:11.007 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T05:44:11.008 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T05:44:11.008 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T05:44:11.008 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:11.056 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 136/156 2024-04-24T05:44:11.135 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 136/156 2024-04-24T05:44:11.135 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:11.135 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T05:44:11.135 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:11.215 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 137/156 2024-04-24T05:44:11.289 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 137/156 2024-04-24T05:44:11.289 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:11.289 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T05:44:11.289 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T05:44:11.289 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T05:44:11.289 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:11.632 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 138/156 2024-04-24T05:44:11.700 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 138/156 2024-04-24T05:44:11.700 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:11.700 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T05:44:11.700 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T05:44:11.700 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T05:44:11.700 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 139/156 2024-04-24T05:44:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 140/156 2024-04-24T05:44:16.811 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 141/156 2024-04-24T05:44:16.855 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T05:44:16.903 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T05:44:17.220 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 144/156 2024-04-24T05:44:17.275 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T05:44:17.303 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_ 146/156 2024-04-24T05:44:18.158 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T05:44:18.200 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T05:44:18.256 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T05:44:19.587 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 148/156 2024-04-24T05:44:19.690 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T05:44:19.768 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T05:44:19.933 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T05:44:19.979 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T05:44:19.979 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T05:44:20.089 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T05:44:20.128 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T05:44:20.128 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T05:44:20.260 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T05:44:20.391 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T05:44:20.681 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T05:44:20.681 INFO:teuthology.orchestra.run.smithi123.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T05:44:20.682 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:21.331 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 107/156 2024-04-24T05:44:21.331 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2024-04-24T05:44:21.331 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2024-04-24T05:44:21.331 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2024-04-24T05:44:21.331 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2024-04-24T05:44:21.331 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2024-04-24T05:44:21.331 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /root 2024-04-24T05:44:21.331 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2024-04-24T05:44:21.331 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:21.590 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 108/156 2024-04-24T05:44:21.637 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 108/156 2024-04-24T05:44:21.637 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:21.637 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T05:44:21.637 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:23.317 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 109/156 2024-04-24T05:44:23.422 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 109/156 2024-04-24T05:44:23.423 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:23.423 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T05:44:23.423 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T05:44:23.423 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T05:44:23.423 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:23.676 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 110/156 2024-04-24T05:44:23.762 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 110/156 2024-04-24T05:44:23.762 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:23.762 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T05:44:23.762 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T05:44:23.762 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T05:44:23.762 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:24.247 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 111/156 2024-04-24T05:44:24.338 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 111/156 2024-04-24T05:44:24.338 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:24.338 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T05:44:24.338 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T05:44:24.338 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T05:44:24.339 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:24.487 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T05:44:24.549 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T05:44:24.623 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T05:44:24.856 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T05:44:24.939 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T05:44:25.033 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T05:44:25.142 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T05:44:25.328 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T05:44:25.388 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T05:44:25.438 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T05:44:25.471 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T05:44:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 1/156 2024-04-24T05:44:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2/156 2024-04-24T05:44:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 3/156 2024-04-24T05:44:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 4/156 2024-04-24T05:44:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 5/156 2024-04-24T05:44:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 6/156 2024-04-24T05:44:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 7/156 2024-04-24T05:44:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T05:44:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T05:44:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 10/156 2024-04-24T05:44:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 11/156 2024-04-24T05:44:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 12/156 2024-04-24T05:44:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T05:44:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 14/156 2024-04-24T05:44:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_ 15/156 2024-04-24T05:44:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 16/156 2024-04-24T05:44:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 17/156 2024-04-24T05:44:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_6 18/156 2024-04-24T05:44:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 19/156 2024-04-24T05:44:26.043 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 20/156 2024-04-24T05:44:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el 21/156 2024-04-24T05:44:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9. 22/156 2024-04-24T05:44:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_6 23/156 2024-04-24T05:44:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 24/156 2024-04-24T05:44:26.044 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 25/156 2024-04-24T05:44:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 26/156 2024-04-24T05:44:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 27/156 2024-04-24T05:44:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 28/156 2024-04-24T05:44:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 29/156 2024-04-24T05:44:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b. 30/156 2024-04-24T05:44:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 31/156 2024-04-24T05:44:26.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 32/156 2024-04-24T05:44:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 33/156 2024-04-24T05:44:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el 34/156 2024-04-24T05:44:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 35/156 2024-04-24T05:44:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.e 36/156 2024-04-24T05:44:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 37/156 2024-04-24T05:44:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 38/156 2024-04-24T05:44:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 39/156 2024-04-24T05:44:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T05:44:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T05:44:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T05:44:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T05:44:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T05:44:26.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T05:44:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T05:44:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T05:44:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T05:44:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T05:44:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T05:44:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T05:44:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T05:44:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T05:44:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T05:44:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T05:44:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T05:44:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T05:44:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T05:44:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T05:44:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T05:44:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T05:44:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T05:44:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T05:44:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T05:44:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T05:44:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T05:44:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T05:44:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T05:44:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T05:44:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T05:44:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T05:44:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T05:44:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T05:44:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T05:44:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T05:44:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T05:44:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T05:44:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T05:44:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T05:44:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T05:44:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T05:44:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T05:44:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T05:44:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T05:44:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T05:44:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T05:44:26.060 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T05:44:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T05:44:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T05:44:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T05:44:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T05:44:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T05:44:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T05:44:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T05:44:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T05:44:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T05:44:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T05:44:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T05:44:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T05:44:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T05:44:26.061 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T05:44:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T05:44:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T05:44:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T05:44:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T05:44:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T05:44:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T05:44:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T05:44:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T05:44:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T05:44:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T05:44:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T05:44:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T05:44:26.062 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T05:44:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T05:44:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T05:44:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T05:44:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T05:44:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T05:44:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T05:44:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T05:44:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T05:44:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T05:44:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T05:44:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T05:44:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T05:44:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T05:44:26.063 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T05:44:26.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T05:44:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T05:44:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T05:44:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T05:44:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T05:44:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T05:44:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 149/156 2024-04-24T05:44:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T05:44:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 151/156 2024-04-24T05:44:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T05:44:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T05:44:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T05:44:26.065 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T05:44:26.066 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 123/156 2024-04-24T05:44:26.098 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 123/156 2024-04-24T05:44:26.154 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.e 124/156 2024-04-24T05:44:26.251 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b. 125/156 2024-04-24T05:44:26.292 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 126/156 2024-04-24T05:44:27.616 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 126/156 2024-04-24T05:44:27.660 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 127/156 2024-04-24T05:44:28.485 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T05:44:28.485 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:28.486 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-04-24T05:44:28.486 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.486 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.486 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T05:44:28.486 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T05:44:28.486 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-24T05:44:28.486 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T05:44:28.486 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T05:44:28.486 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T05:44:28.486 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T05:44:28.486 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.486 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:28.488 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.488 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.488 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.488 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:28.488 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:28.488 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.488 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.488 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:28.488 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T05:44:28.488 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T05:44:28.488 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T05:44:28.488 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T05:44:28.489 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T05:44:28.489 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T05:44:28.489 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T05:44:28.489 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T05:44:28.489 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T05:44:28.489 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T05:44:28.489 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T05:44:28.489 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.489 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.489 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.489 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T05:44:28.489 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T05:44:28.489 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T05:44:28.490 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T05:44:28.490 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T05:44:28.490 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.490 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.490 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T05:44:28.490 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.490 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T05:44:28.490 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T05:44:28.490 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T05:44:28.490 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T05:44:28.490 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T05:44:28.490 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T05:44:28.491 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T05:44:28.491 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T05:44:28.491 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T05:44:28.491 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T05:44:28.491 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T05:44:28.491 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T05:44:28.491 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T05:44:28.491 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T05:44:28.491 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T05:44:28.491 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T05:44:28.491 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T05:44:28.491 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T05:44:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T05:44:28.493 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T05:44:28.493 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T05:44:28.493 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T05:44:28.493 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T05:44:28.493 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T05:44:28.493 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T05:44:28.493 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T05:44:28.493 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T05:44:28.493 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T05:44:28.493 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T05:44:28.493 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T05:44:28.493 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T05:44:28.493 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T05:44:28.494 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T05:44:28.494 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T05:44:28.494 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T05:44:28.494 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T05:44:28.494 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T05:44:28.494 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T05:44:28.494 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T05:44:28.494 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T05:44:28.494 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T05:44:28.494 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T05:44:28.494 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T05:44:28.494 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T05:44:28.494 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T05:44:28.494 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T05:44:28.495 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T05:44:28.495 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T05:44:28.495 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T05:44:28.495 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T05:44:28.495 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T05:44:28.495 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T05:44:28.495 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.495 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.495 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T05:44:28.495 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T05:44:28.495 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T05:44:28.495 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.495 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T05:44:28.496 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T05:44:28.496 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T05:44:28.496 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T05:44:28.496 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T05:44:28.496 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T05:44:28.496 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T05:44:28.496 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T05:44:28.496 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T05:44:28.496 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T05:44:28.496 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T05:44:28.496 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T05:44:28.496 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T05:44:28.496 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T05:44:28.496 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T05:44:28.497 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.497 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.497 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:28.497 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T05:44:28.497 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T05:44:28.497 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T05:44:28.497 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T05:44:28.497 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T05:44:28.497 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T05:44:28.497 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T05:44:28.498 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:28.498 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-24T05:44:28.761 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 127/156 2024-04-24T05:44:28.794 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 128/156 2024-04-24T05:44:28.939 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 128/156 2024-04-24T05:44:29.117 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el 129/156 2024-04-24T05:44:29.129 DEBUG:teuthology.parallel:result is None 2024-04-24T05:44:29.154 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 130/156 2024-04-24T05:44:29.223 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 130/156 2024-04-24T05:44:29.223 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:29.223 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T05:44:29.223 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T05:44:29.223 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T05:44:29.223 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:29.280 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 131/156 2024-04-24T05:44:29.317 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 131/156 2024-04-24T05:44:29.458 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 132/156 2024-04-24T05:44:29.531 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 133/156 2024-04-24T05:44:30.538 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T05:44:30.574 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 135/156 2024-04-24T05:44:30.641 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 135/156 2024-04-24T05:44:30.641 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:30.641 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T05:44:30.641 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T05:44:30.641 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T05:44:30.642 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:30.691 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 136/156 2024-04-24T05:44:30.758 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 136/156 2024-04-24T05:44:30.758 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:30.758 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T05:44:30.758 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:30.833 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 137/156 2024-04-24T05:44:30.907 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 137/156 2024-04-24T05:44:30.907 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:30.907 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T05:44:30.907 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T05:44:30.907 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T05:44:30.907 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:31.259 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 138/156 2024-04-24T05:44:31.338 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 138/156 2024-04-24T05:44:31.338 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T05:44:31.338 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T05:44:31.338 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T05:44:31.339 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T05:44:31.339 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:36.343 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 139/156 2024-04-24T05:44:36.397 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 140/156 2024-04-24T05:44:36.468 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 141/156 2024-04-24T05:44:36.512 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T05:44:36.560 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T05:44:36.906 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 144/156 2024-04-24T05:44:36.964 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T05:44:36.983 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_ 146/156 2024-04-24T05:44:37.919 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T05:44:37.974 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T05:44:38.039 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T05:44:39.349 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 148/156 2024-04-24T05:44:39.454 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T05:44:39.539 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T05:44:39.755 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T05:44:39.811 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T05:44:39.811 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T05:44:39.937 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T05:44:39.992 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T05:44:40.040 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T05:44:40.133 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T05:44:40.288 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T05:44:40.563 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T05:44:40.563 INFO:teuthology.orchestra.run.smithi155.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T05:44:40.563 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:46.412 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 1/156 2024-04-24T05:44:46.412 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2/156 2024-04-24T05:44:46.412 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 3/156 2024-04-24T05:44:46.412 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 4/156 2024-04-24T05:44:46.412 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 5/156 2024-04-24T05:44:46.412 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 6/156 2024-04-24T05:44:46.413 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 7/156 2024-04-24T05:44:46.413 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T05:44:46.413 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T05:44:46.413 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 10/156 2024-04-24T05:44:46.413 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 11/156 2024-04-24T05:44:46.413 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 12/156 2024-04-24T05:44:46.413 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T05:44:46.413 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 14/156 2024-04-24T05:44:46.413 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_ 15/156 2024-04-24T05:44:46.413 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 16/156 2024-04-24T05:44:46.413 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 17/156 2024-04-24T05:44:46.413 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_6 18/156 2024-04-24T05:44:46.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 19/156 2024-04-24T05:44:46.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 20/156 2024-04-24T05:44:46.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el 21/156 2024-04-24T05:44:46.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9. 22/156 2024-04-24T05:44:46.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_6 23/156 2024-04-24T05:44:46.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 24/156 2024-04-24T05:44:46.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 25/156 2024-04-24T05:44:46.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 26/156 2024-04-24T05:44:46.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 27/156 2024-04-24T05:44:46.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 28/156 2024-04-24T05:44:46.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 29/156 2024-04-24T05:44:46.414 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b. 30/156 2024-04-24T05:44:46.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 31/156 2024-04-24T05:44:46.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 32/156 2024-04-24T05:44:46.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 33/156 2024-04-24T05:44:46.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el 34/156 2024-04-24T05:44:46.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 35/156 2024-04-24T05:44:46.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.e 36/156 2024-04-24T05:44:46.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 37/156 2024-04-24T05:44:46.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 38/156 2024-04-24T05:44:46.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 39/156 2024-04-24T05:44:46.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T05:44:46.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T05:44:46.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T05:44:46.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T05:44:46.416 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T05:44:46.416 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T05:44:46.416 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T05:44:46.416 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T05:44:46.416 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T05:44:46.416 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T05:44:46.416 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T05:44:46.416 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T05:44:46.416 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T05:44:46.416 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T05:44:46.416 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T05:44:46.416 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T05:44:46.417 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T05:44:46.417 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T05:44:46.417 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T05:44:46.417 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T05:44:46.417 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T05:44:46.417 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T05:44:46.417 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T05:44:46.417 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T05:44:46.417 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T05:44:46.417 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T05:44:46.417 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T05:44:46.417 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T05:44:46.417 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T05:44:46.418 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T05:44:46.418 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T05:44:46.418 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T05:44:46.418 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T05:44:46.418 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T05:44:46.418 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T05:44:46.418 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T05:44:46.418 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T05:44:46.418 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T05:44:46.418 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T05:44:46.418 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T05:44:46.418 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T05:44:46.418 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T05:44:46.419 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T05:44:46.419 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T05:44:46.419 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T05:44:46.419 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T05:44:46.419 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T05:44:46.419 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T05:44:46.419 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T05:44:46.419 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T05:44:46.419 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T05:44:46.419 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T05:44:46.419 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T05:44:46.419 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T05:44:46.419 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T05:44:46.420 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T05:44:46.420 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T05:44:46.420 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T05:44:46.420 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T05:44:46.420 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T05:44:46.420 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T05:44:46.420 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T05:44:46.420 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T05:44:46.420 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T05:44:46.420 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T05:44:46.420 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T05:44:46.420 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T05:44:46.420 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T05:44:46.421 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T05:44:46.421 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T05:44:46.421 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T05:44:46.421 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T05:44:46.421 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T05:44:46.421 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T05:44:46.421 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T05:44:46.421 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T05:44:46.421 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T05:44:46.421 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T05:44:46.421 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T05:44:46.421 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T05:44:46.421 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T05:44:46.422 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T05:44:46.422 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T05:44:46.422 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T05:44:46.422 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T05:44:46.422 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T05:44:46.422 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T05:44:46.422 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T05:44:46.422 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T05:44:46.422 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T05:44:46.422 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T05:44:46.422 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T05:44:46.422 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T05:44:46.422 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T05:44:46.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T05:44:46.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T05:44:46.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T05:44:46.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T05:44:46.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T05:44:46.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T05:44:46.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T05:44:46.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T05:44:46.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T05:44:46.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T05:44:46.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T05:44:46.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T05:44:46.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T05:44:46.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T05:44:46.424 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T05:44:46.424 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 149/156 2024-04-24T05:44:46.424 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T05:44:46.424 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 151/156 2024-04-24T05:44:46.424 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T05:44:46.424 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T05:44:46.424 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T05:44:46.424 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T05:44:48.834 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T05:44:48.835 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:48.835 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2024-04-24T05:44:48.835 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.835 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.835 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T05:44:48.835 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T05:44:48.835 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-04-24T05:44:48.835 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T05:44:48.835 INFO:teuthology.orchestra.run.smithi155.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T05:44:48.835 INFO:teuthology.orchestra.run.smithi155.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T05:44:48.836 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T05:44:48.836 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.836 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.836 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.836 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.836 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:48.836 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.836 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.836 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.836 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:48.836 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:48.836 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:48.836 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:48.836 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:48.837 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.837 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.837 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:48.837 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.837 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.837 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.837 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:48.837 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:48.837 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.837 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.837 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T05:44:48.837 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T05:44:48.837 INFO:teuthology.orchestra.run.smithi155.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T05:44:48.837 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T05:44:48.838 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T05:44:48.838 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T05:44:48.838 INFO:teuthology.orchestra.run.smithi155.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T05:44:48.838 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T05:44:48.838 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T05:44:48.838 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T05:44:48.838 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T05:44:48.838 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T05:44:48.838 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.838 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.838 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.838 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T05:44:48.838 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T05:44:48.838 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T05:44:48.839 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T05:44:48.839 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T05:44:48.839 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.839 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.839 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T05:44:48.839 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.839 INFO:teuthology.orchestra.run.smithi155.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T05:44:48.839 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T05:44:48.839 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T05:44:48.839 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T05:44:48.839 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T05:44:48.839 INFO:teuthology.orchestra.run.smithi155.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T05:44:48.839 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T05:44:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T05:44:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T05:44:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T05:44:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T05:44:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T05:44:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T05:44:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T05:44:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T05:44:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T05:44:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T05:44:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T05:44:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T05:44:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T05:44:48.840 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T05:44:48.841 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T05:44:48.841 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T05:44:48.841 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.841 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.841 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.841 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T05:44:48.841 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T05:44:48.841 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T05:44:48.841 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T05:44:48.841 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T05:44:48.841 INFO:teuthology.orchestra.run.smithi155.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T05:44:48.841 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T05:44:48.841 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T05:44:48.841 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T05:44:48.842 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T05:44:48.843 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T05:44:48.843 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T05:44:48.843 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T05:44:48.843 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T05:44:48.843 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T05:44:48.843 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T05:44:48.843 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T05:44:48.843 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T05:44:48.843 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T05:44:48.843 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T05:44:48.843 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T05:44:48.843 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T05:44:48.843 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T05:44:48.843 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T05:44:48.844 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T05:44:48.845 INFO:teuthology.orchestra.run.smithi155.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T05:44:48.845 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T05:44:48.845 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T05:44:48.845 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T05:44:48.845 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T05:44:48.845 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T05:44:48.845 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T05:44:48.845 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T05:44:48.845 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.845 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.845 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T05:44:48.845 INFO:teuthology.orchestra.run.smithi155.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T05:44:48.845 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T05:44:48.845 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T05:44:48.846 INFO:teuthology.orchestra.run.smithi155.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T05:44:48.846 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T05:44:48.846 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T05:44:48.846 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T05:44:48.846 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:48.846 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-04-24T05:44:49.442 DEBUG:teuthology.parallel:result is None 2024-04-24T05:44:49.442 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T05:44:49.601 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T05:44:49.668 INFO:teuthology.orchestra.run.smithi123.stdout:19.0.0-3124.ga3c8a22b.el9 2024-04-24T05:44:49.669 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3124.ga3c8a22b.el9 2024-04-24T05:44:49.669 INFO:teuthology.task.install:The correct ceph version 19.0.0-3124.ga3c8a22b is installed. 2024-04-24T05:44:49.671 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T05:44:49.888 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T05:44:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:19.0.0-3124.ga3c8a22b.el9 2024-04-24T05:44:49.948 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3124.ga3c8a22b.el9 2024-04-24T05:44:49.948 INFO:teuthology.task.install:The correct ceph version 19.0.0-3124.ga3c8a22b is installed. 2024-04-24T05:44:49.950 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T05:44:49.950 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:44:49.950 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T05:44:50.047 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:44:50.047 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T05:44:50.118 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T05:44:50.118 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:44:50.118 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T05:44:50.207 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T05:44:50.287 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:44:50.287 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T05:44:50.356 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T05:44:50.462 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T05:44:50.463 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:44:50.463 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T05:44:50.554 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T05:44:50.689 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:44:50.689 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T05:44:50.756 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T05:44:50.865 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T05:44:50.866 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:44:50.866 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T05:44:50.957 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T05:44:51.047 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:44:51.047 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T05:44:51.116 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T05:44:51.219 INFO:teuthology.run_tasks:Running task ceph... 2024-04-24T05:44:51.385 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-24T05:44:51.385 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /var/log/ceph 2024-04-24T05:44:51.393 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /var/log/ceph 2024-04-24T05:44:51.476 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-24T05:44:51.477 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T05:44:51.481 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T05:44:51.557 INFO:tasks.ceph:Creating extra log directories... 2024-04-24T05:44:51.557 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T05:44:51.562 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T05:44:51.646 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-24T05:44:51.646 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-24T05:44:51.647 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-24T05:44:51.647 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:44:51.647 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-24T05:44:51.699 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-24T05:44:51.912 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-24T05:44:51.998 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:44:51.998 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-24T05:44:52.040 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-24T05:44:52.217 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-24T05:44:52.297 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-24T05:44:52.297 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-24T05:44:52.297 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671152', 'branch': 'wip-rishabh-testing-20240416.193735-5', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev}', 'email': None, 'first_in_suite': False, 'job_id': '7671152', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240416.193735-5'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240416.193735-5', 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7954, 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'sleep_before_teardown': 0, 'subset': '364/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240416.193735-5', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi123.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJZ8f+QA2XGb+7sd1j7CUoN/FV5hrqTKOcg3SNpKfFMdBNWX7SiedWdG2mbVrPra1dFWU7VmpzqSwfB/VO3IRHE=', 'smithi155.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKhrioUtGsH6Ui2kfSAZASIczHQFe7It+EGUuixVBaf0uPjpauvKx1TVJ9KRiMtkxG7AOBmqFbBRz3DRQRhqszQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_05:22:11', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-24T05:44:52.298 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T05:44:52.305 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T05:44:52.357 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T05:44:52.362 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T05:44:52.439 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:44:52.439 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T05:44:52.495 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T05:44:52.495 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-04-24T05:44:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T05:44:52.545 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T05:44:52.545 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-24T05:44:52.545 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T05:44:52.545 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T05:44:52.545 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-24 05:44:21.178745034 +0000 2024-04-24T05:44:52.545 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-24 05:42:06.258873693 +0000 2024-04-24T05:44:52.546 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-24 05:42:06.258873693 +0000 2024-04-24T05:44:52.546 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-24 05:42:06.258873693 +0000 2024-04-24T05:44:52.546 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T05:44:52.622 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-24T05:44:52.622 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-24T05:44:52.622 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000358427 s, 1.4 MB/s 2024-04-24T05:44:52.626 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T05:44:52.678 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-04-24T05:44:52.721 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T05:44:52.721 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T05:44:52.721 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-24T05:44:52.721 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T05:44:52.721 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T05:44:52.721 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-24 05:44:21.179745010 +0000 2024-04-24T05:44:52.721 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-24 05:42:06.852859787 +0000 2024-04-24T05:44:52.721 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-24 05:42:06.852859787 +0000 2024-04-24T05:44:52.721 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-24 05:42:06.852859787 +0000 2024-04-24T05:44:52.722 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T05:44:52.846 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-24T05:44:52.846 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-24T05:44:52.846 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000318018 s, 1.6 MB/s 2024-04-24T05:44:52.850 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T05:44:52.907 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-04-24T05:44:52.958 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T05:44:52.958 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T05:44:52.958 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-24T05:44:52.958 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T05:44:52.958 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T05:44:52.958 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-24 05:44:21.180744986 +0000 2024-04-24T05:44:52.958 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-24 05:42:07.457845623 +0000 2024-04-24T05:44:52.958 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-24 05:42:07.457845623 +0000 2024-04-24T05:44:52.958 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-24 05:42:07.457845623 +0000 2024-04-24T05:44:52.959 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T05:44:53.028 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-24T05:44:53.028 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-24T05:44:53.029 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000343102 s, 1.5 MB/s 2024-04-24T05:44:53.032 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T05:44:53.084 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-04-24T05:44:53.136 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T05:44:53.137 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T05:44:53.137 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-24T05:44:53.137 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T05:44:53.137 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T05:44:53.137 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-24 05:44:21.182744939 +0000 2024-04-24T05:44:53.137 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-24 05:42:08.061831483 +0000 2024-04-24T05:44:53.137 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-24 05:42:08.061831483 +0000 2024-04-24T05:44:53.137 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-24 05:42:08.061831483 +0000 2024-04-24T05:44:53.137 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T05:44:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-24T05:44:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-24T05:44:53.216 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000343468 s, 1.5 MB/s 2024-04-24T05:44:53.219 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T05:44:53.275 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T05:44:53.275 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:44:53.275 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T05:44:53.321 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T05:44:53.321 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-04-24T05:44:53.398 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T05:44:53.398 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T05:44:53.398 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-24T05:44:53.399 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T05:44:53.399 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T05:44:53.399 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-24 05:44:41.028490608 +0000 2024-04-24T05:44:53.399 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-24 05:42:04.271477804 +0000 2024-04-24T05:44:53.399 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-24 05:42:04.271477804 +0000 2024-04-24T05:44:53.399 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-04-24 05:42:04.271477804 +0000 2024-04-24T05:44:53.399 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T05:44:53.500 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-24T05:44:53.500 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-24T05:44:53.501 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000373305 s, 1.4 MB/s 2024-04-24T05:44:53.503 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T05:44:53.543 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-04-24T05:44:53.625 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T05:44:53.625 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T05:44:53.625 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-24T05:44:53.625 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T05:44:53.625 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T05:44:53.625 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-24 05:44:41.028490608 +0000 2024-04-24T05:44:53.625 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-24 05:42:04.864465710 +0000 2024-04-24T05:44:53.625 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-24 05:42:04.864465710 +0000 2024-04-24T05:44:53.625 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-04-24 05:42:04.864465710 +0000 2024-04-24T05:44:53.625 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T05:44:53.724 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-24T05:44:53.724 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-24T05:44:53.724 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000317655 s, 1.6 MB/s 2024-04-24T05:44:53.727 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T05:44:53.767 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-04-24T05:44:53.852 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T05:44:53.852 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T05:44:53.852 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-24T05:44:53.852 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T05:44:53.852 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T05:44:53.852 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-24 05:44:41.030490568 +0000 2024-04-24T05:44:53.852 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-24 05:42:05.430454159 +0000 2024-04-24T05:44:53.852 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-24 05:42:05.430454159 +0000 2024-04-24T05:44:53.852 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-04-24 05:42:05.430454159 +0000 2024-04-24T05:44:53.852 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T05:44:53.959 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-24T05:44:53.959 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-24T05:44:53.959 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000273984 s, 1.9 MB/s 2024-04-24T05:44:53.962 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T05:44:54.002 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-04-24T05:44:54.084 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T05:44:54.084 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T05:44:54.084 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-24T05:44:54.084 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T05:44:54.084 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T05:44:54.084 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-04-24 05:44:41.030490568 +0000 2024-04-24T05:44:54.084 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-04-24 05:42:06.027441974 +0000 2024-04-24T05:44:54.085 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-04-24 05:42:06.027441974 +0000 2024-04-24T05:44:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-04-24 05:42:06.027441974 +0000 2024-04-24T05:44:54.085 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T05:44:54.184 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-04-24T05:44:54.185 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-04-24T05:44:54.185 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.00031037 s, 1.6 MB/s 2024-04-24T05:44:54.188 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T05:44:54.228 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T05:44:54.228 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi155.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T05:44:54.228 INFO:tasks.ceph:Generating config... 2024-04-24T05:44:54.230 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-24T05:44:54.230 INFO:tasks.ceph:[client] debug client = 20 2024-04-24T05:44:54.230 INFO:tasks.ceph:[client] debug ms = 1 2024-04-24T05:44:54.230 INFO:tasks.ceph:[client] pid file = 2024-04-24T05:44:54.230 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-24T05:44:54.231 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-24T05:44:54.231 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-24T05:44:54.231 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-24T05:44:54.231 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-24T05:44:54.231 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-24T05:44:54.231 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-24T05:44:54.231 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-24T05:44:54.231 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-24T05:44:54.231 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-24T05:44:54.231 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-24T05:44:54.231 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-24T05:44:54.231 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-24T05:44:54.231 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-24T05:44:54.231 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-24T05:44:54.231 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-24T05:44:54.232 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-24T05:44:54.232 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-24T05:44:54.232 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-24T05:44:54.232 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-24T05:44:54.232 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-24T05:44:54.232 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-24T05:44:54.232 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-24T05:44:54.232 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-24T05:44:54.232 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-24T05:44:54.232 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-24T05:44:54.232 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-24T05:44:54.232 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-24T05:44:54.233 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-24T05:44:54.233 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-24T05:44:54.233 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-24T05:44:54.233 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-24T05:44:54.233 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-24T05:44:54.233 INFO:tasks.ceph:Setting up mon.a... 2024-04-24T05:44:54.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-24T05:44:54.363 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.keyring 2024-04-24T05:44:54.387 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-24T05:44:54.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T05:44:54.595 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.123'), ('mon.b', '172.21.15.155'), ('mon.c', '[v2:172.21.15.155:3301,v1:172.21.15.155:6790]')] 2024-04-24T05:44:54.596 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.123,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-24T05:44:54.597 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:44:54.597 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T05:44:54.649 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.123 --add b 172.21.15.155 --addv c '[v2:172.21.15.155:3301,v1:172.21.15.155:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-24T05:44:54.803 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-24T05:44:54.803 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: generated fsid 6efb8524-c0d1-4076-b7c9-7485084c7ada 2024-04-24T05:44:54.803 INFO:teuthology.orchestra.run.smithi123.stdout:setting min_mon_release = quincy 2024-04-24T05:44:54.803 INFO:teuthology.orchestra.run.smithi123.stdout:epoch 0 2024-04-24T05:44:54.803 INFO:teuthology.orchestra.run.smithi123.stdout:fsid 6efb8524-c0d1-4076-b7c9-7485084c7ada 2024-04-24T05:44:54.803 INFO:teuthology.orchestra.run.smithi123.stdout:last_changed 2024-04-24T05:44:54.802840+0000 2024-04-24T05:44:54.803 INFO:teuthology.orchestra.run.smithi123.stdout:created 2024-04-24T05:44:54.802840+0000 2024-04-24T05:44:54.803 INFO:teuthology.orchestra.run.smithi123.stdout:min_mon_release 17 (quincy) 2024-04-24T05:44:54.804 INFO:teuthology.orchestra.run.smithi123.stdout:election_strategy: 1 2024-04-24T05:44:54.804 INFO:teuthology.orchestra.run.smithi123.stdout:0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-04-24T05:44:54.804 INFO:teuthology.orchestra.run.smithi123.stdout:1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.b 2024-04-24T05:44:54.804 INFO:teuthology.orchestra.run.smithi123.stdout:2: [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] mon.c 2024-04-24T05:44:54.804 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-24T05:44:54.806 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T05:44:54.855 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6efb8524-c0d1-4076-b7c9-7485084c7ada... 2024-04-24T05:44:54.857 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T05:44:54.862 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T05:44:54.975 INFO:teuthology.orchestra.run.smithi155.stdout:[global] 2024-04-24T05:44:54.975 INFO:teuthology.orchestra.run.smithi155.stdout: chdir = "" 2024-04-24T05:44:54.975 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T05:44:54.975 INFO:teuthology.orchestra.run.smithi155.stdout: auth supported = cephx 2024-04-24T05:44:54.975 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.975 INFO:teuthology.orchestra.run.smithi155.stdout: filestore xattr use omap = true 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: mon clock drift allowed = 1.000 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: osd crush chooseleaf type = 0 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: auth debug = true 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on old message = true 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on bug = true 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: mon pg warn max object skew = 0 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T05:44:54.976 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.977 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default size = 2 2024-04-24T05:44:54.977 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.977 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow primary affinity = true 2024-04-24T05:44:54.977 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow pg remap = true 2024-04-24T05:44:54.977 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on legacy crush tunables = false 2024-04-24T05:44:54.977 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on crush straw calc version zero = false 2024-04-24T05:44:54.977 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on no sortbitwise = false 2024-04-24T05:44:54.977 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on osd down out interval zero = false 2024-04-24T05:44:54.977 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on too few osds = false 2024-04-24T05:44:54.977 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T05:44:54.977 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T05:44:54.977 INFO:teuthology.orchestra.run.smithi155.stdout: mon_allow_pool_size_one = true 2024-04-24T05:44:54.977 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.977 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T05:44:54.978 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.978 INFO:teuthology.orchestra.run.smithi155.stdout: osd default data pool replay window = 5 2024-04-24T05:44:54.978 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.978 INFO:teuthology.orchestra.run.smithi155.stdout: mon allow pool delete = true 2024-04-24T05:44:54.978 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.978 INFO:teuthology.orchestra.run.smithi155.stdout: mon cluster log file level = debug 2024-04-24T05:44:54.978 INFO:teuthology.orchestra.run.smithi155.stdout: debug asserts on shutdown = true 2024-04-24T05:44:54.978 INFO:teuthology.orchestra.run.smithi155.stdout: mon health detail to clog = false 2024-04-24T05:44:54.978 INFO:teuthology.orchestra.run.smithi155.stdout: mon host = "172.21.15.123,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2024-04-24T05:44:54.978 INFO:teuthology.orchestra.run.smithi155.stdout: fsid = 6efb8524-c0d1-4076-b7c9-7485084c7ada 2024-04-24T05:44:54.978 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.978 INFO:teuthology.orchestra.run.smithi155.stdout:[osd] 2024-04-24T05:44:54.978 INFO:teuthology.orchestra.run.smithi155.stdout: osd journal size = 100 2024-04-24T05:44:54.978 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.979 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub load threshold = 5.0 2024-04-24T05:44:54.979 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub max interval = 600 2024-04-24T05:44:54.979 INFO:teuthology.orchestra.run.smithi155.stdout: osd mclock profile = high_recovery_ops 2024-04-24T05:44:54.979 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.979 INFO:teuthology.orchestra.run.smithi155.stdout: osd recover clone overlap = true 2024-04-24T05:44:54.979 INFO:teuthology.orchestra.run.smithi155.stdout: osd recovery max chunk = 1048576 2024-04-24T05:44:54.979 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.979 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug shutdown = true 2024-04-24T05:44:54.979 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug op order = true 2024-04-24T05:44:54.979 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug verify stray on activate = true 2024-04-24T05:44:54.979 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug trim objects = true 2024-04-24T05:44:54.979 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.979 INFO:teuthology.orchestra.run.smithi155.stdout: osd open classes on start = true 2024-04-24T05:44:54.980 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug pg log writeout = true 2024-04-24T05:44:54.980 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.980 INFO:teuthology.orchestra.run.smithi155.stdout: osd deep scrub update digest min age = 30 2024-04-24T05:44:54.980 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.980 INFO:teuthology.orchestra.run.smithi155.stdout: osd map max advance = 10 2024-04-24T05:44:54.980 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.980 INFO:teuthology.orchestra.run.smithi155.stdout: journal zero on create = true 2024-04-24T05:44:54.980 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.980 INFO:teuthology.orchestra.run.smithi155.stdout: filestore ondisk finisher threads = 3 2024-04-24T05:44:54.980 INFO:teuthology.orchestra.run.smithi155.stdout: filestore apply finisher threads = 3 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: bdev debug aio = true 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug misdirected ops = true 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore block size = 96636764160 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore fsck on mount = True 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluefs = 20 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluestore = 20 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: debug osd = 20 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: debug rocksdb = 10 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd full ratio = 0.9 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd nearfull ratio = 0.8 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: osd failsafe full ratio = 0.95 2024-04-24T05:44:54.981 INFO:teuthology.orchestra.run.smithi155.stdout: osd objectstore = bluestore 2024-04-24T05:44:54.982 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-04-24T05:44:54.982 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.982 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr] 2024-04-24T05:44:54.982 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-24T05:44:54.982 INFO:teuthology.orchestra.run.smithi155.stdout: debug mgr = 20 2024-04-24T05:44:54.982 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-04-24T05:44:54.982 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-04-24T05:44:54.982 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-04-24T05:44:54.982 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-04-24T05:44:54.982 INFO:teuthology.orchestra.run.smithi155.stdout: mgr/telemetry/nag = false 2024-04-24T05:44:54.982 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-04-24T05:44:54.982 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.982 INFO:teuthology.orchestra.run.smithi155.stdout:[mon] 2024-04-24T05:44:54.982 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: debug paxos = 20 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: mon data avail warn = 5 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: mon mgr mkfs grace = 240 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd reporter subtree level = osd 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd prime pg temp = true 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.983 INFO:teuthology.orchestra.run.smithi155.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout: # 1m isn't quite enough 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout: mon_down_mkfs_grace = 2m 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_filestore_osds = false 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout: mon down mkfs grace = 300 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout: mon op complaint time = 120 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout:[client] 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout: rgw cache enabled = true 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable ops log = true 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable usage log = true 2024-04-24T05:44:54.984 INFO:teuthology.orchestra.run.smithi155.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout: client mount timeout = 600 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = "" 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.a] 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.b] 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.c] 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.a] 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.c] 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.b] 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.d] 2024-04-24T05:44:54.985 INFO:teuthology.orchestra.run.smithi155.stdout:[mds] 2024-04-24T05:44:54.986 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds = 20 2024-04-24T05:44:54.986 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds balancer = 20 2024-04-24T05:44:54.986 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-04-24T05:44:54.986 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug frag = True 2024-04-24T05:44:54.986 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug scatterstat = True 2024-04-24T05:44:54.986 INFO:teuthology.orchestra.run.smithi155.stdout: mds op complaint time = 180 2024-04-24T05:44:54.986 INFO:teuthology.orchestra.run.smithi155.stdout: mds verify scatter = True 2024-04-24T05:44:54.986 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-04-24T05:44:54.986 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-04-24T05:44:54.986 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-04-24T05:44:54.990 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2024-04-24T05:44:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2024-04-24T05:44:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T05:44:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2024-04-24T05:44:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T05:44:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T05:44:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2024-04-24T05:44:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2024-04-24T05:44:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2024-04-24T05:44:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2024-04-24T05:44:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2024-04-24T05:44:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2024-04-24T05:44:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2024-04-24T05:44:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2024-04-24T05:44:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T05:44:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T05:44:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2024-04-24T05:44:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = "172.21.15.123,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 6efb8524-c0d1-4076-b7c9-7485084c7ada 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2024-04-24T05:44:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2024-04-24T05:44:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2024-04-24T05:44:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2024-04-24T05:44:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2024-04-24T05:44:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2024-04-24T05:44:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2024-04-24T05:44:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2024-04-24T05:44:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2024-04-24T05:44:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug trim objects = true 2024-04-24T05:44:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2024-04-24T05:44:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2024-04-24T05:44:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2024-04-24T05:44:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2024-04-24T05:44:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2024-04-24T05:44:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2024-04-24T05:44:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2024-04-24T05:44:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2024-04-24T05:44:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2024-04-24T05:44:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore block size = 96636764160 2024-04-24T05:44:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore fsck on mount = True 2024-04-24T05:44:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluefs = 20 2024-04-24T05:44:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluestore = 20 2024-04-24T05:44:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-24T05:44:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2024-04-24T05:44:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: debug rocksdb = 10 2024-04-24T05:44:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T05:44:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd full ratio = 0.9 2024-04-24T05:44:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd nearfull ratio = 0.8 2024-04-24T05:44:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: osd failsafe full ratio = 0.95 2024-04-24T05:44:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = bluestore 2024-04-24T05:44:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2024-04-24T05:44:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.996 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2024-04-24T05:44:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2024-04-24T05:44:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2024-04-24T05:44:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-04-24T05:44:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2024-04-24T05:44:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2024-04-24T05:44:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-04-24T05:44:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T05:44:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T05:44:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T05:44:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T05:44:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T05:44:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T05:44:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2024-04-24T05:44:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2024-04-24T05:44:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2024-04-24T05:44:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: mon down mkfs grace = 300 2024-04-24T05:44:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: mon op complaint time = 120 2024-04-24T05:44:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:44:54.999 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2024-04-24T05:44:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2024-04-24T05:44:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2024-04-24T05:44:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2024-04-24T05:44:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T05:44:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T05:44:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: client mount timeout = 600 2024-04-24T05:44:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = "" 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.b] 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.c] 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout:[mds] 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds = 20 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds balancer = 20 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-24T05:44:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug frag = True 2024-04-24T05:44:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug scatterstat = True 2024-04-24T05:44:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: mds op complaint time = 180 2024-04-24T05:44:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: mds verify scatter = True 2024-04-24T05:44:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2024-04-24T05:44:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2024-04-24T05:44:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2024-04-24T05:44:55.025 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-24T05:44:55.025 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-24T05:44:55.156 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-24T05:44:55.157 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:44:55.157 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-24T05:44:55.203 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:44:55.203 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-24T05:44:55.299 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi123.front.sepia.ceph.com 2024-04-24T05:44:55.300 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:44:55.300 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T05:44:55.300 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T05:44:55.414 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:44:55.414 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T05:44:55.511 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi155.front.sepia.ceph.com 2024-04-24T05:44:55.511 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:44:55.511 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T05:44:55.511 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T05:44:55.606 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:44:55.607 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T05:44:55.688 INFO:tasks.ceph:Setting up mon nodes... 2024-04-24T05:44:55.688 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-24T05:44:55.688 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T05:44:55.857 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T05:44:55.868 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T05:44:56.038 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T05:44:56.046 INFO:tasks.ceph:Setting up mds nodes... 2024-04-24T05:44:56.046 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-24T05:44:56.199 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-24T05:44:56.207 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-24T05:44:56.291 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-24T05:44:56.435 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-24T05:44:56.444 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-24T05:44:56.521 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-24T05:44:56.688 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-24T05:44:56.697 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-24T05:44:56.773 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-24T05:44:56.901 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-24T05:44:56.908 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-24T05:44:56.983 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-24T05:44:56.984 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T05:44:57.104 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-24T05:44:57.145 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-24T05:44:57.262 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-24T05:44:57.298 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-24T05:44:57.415 INFO:teuthology.orchestra.run.smithi155.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-24T05:44:57.449 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-24T05:44:57.559 INFO:teuthology.orchestra.run.smithi155.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-24T05:44:57.609 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-24T05:44:57.609 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi155.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T05:44:57.609 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-24T05:44:57.696 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T05:44:57.696 INFO:tasks.ceph:role: osd.0 2024-04-24T05:44:57.697 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-24T05:44:57.697 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T05:44:57.783 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T05:44:57.784 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T05:44:57.784 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T05:44:57.784 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T05:44:57.784 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T05:44:57.784 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-24T05:44:57.784 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T05:44:57.784 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T05:44:57.784 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T05:44:57.784 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T05:44:58.197 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-24T05:44:58.203 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-24T05:44:58.203 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-24T05:44:58.339 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-24T05:44:58.421 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-24T05:44:58.494 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T05:44:58.494 INFO:tasks.ceph:role: osd.1 2024-04-24T05:44:58.494 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-24T05:44:58.494 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T05:44:58.570 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T05:44:58.570 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T05:44:58.570 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T05:44:58.571 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T05:44:58.571 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T05:44:58.571 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-24T05:44:58.571 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T05:44:58.571 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T05:44:58.571 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T05:44:58.571 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T05:44:58.969 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-24T05:44:58.975 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-24T05:44:58.975 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-24T05:44:59.078 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-24T05:44:59.150 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-24T05:44:59.223 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T05:44:59.223 INFO:tasks.ceph:role: osd.2 2024-04-24T05:44:59.223 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-24T05:44:59.223 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T05:44:59.299 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T05:44:59.300 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T05:44:59.300 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T05:44:59.300 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T05:44:59.300 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T05:44:59.300 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-24T05:44:59.300 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T05:44:59.300 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T05:44:59.300 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T05:44:59.300 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T05:44:59.716 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-24T05:44:59.722 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-24T05:44:59.722 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-24T05:44:59.856 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-24T05:44:59.941 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-24T05:45:00.010 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T05:45:00.010 INFO:tasks.ceph:role: osd.3 2024-04-24T05:45:00.011 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-24T05:45:00.011 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T05:45:00.090 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T05:45:00.090 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T05:45:00.090 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T05:45:00.090 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T05:45:00.090 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T05:45:00.090 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-24T05:45:00.090 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T05:45:00.090 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T05:45:00.091 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T05:45:00.091 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T05:45:00.505 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-24T05:45:00.511 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-24T05:45:00.512 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-24T05:45:00.638 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-24T05:45:00.718 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T05:45:00.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:00.833+0000 7fe69dfcb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-24T05:45:00.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:00.834+0000 7fe69dfcb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-24T05:45:00.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:00.834+0000 7fe69dfcb640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T05:45:00.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:00.834+0000 7fe69dfcb640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T05:45:00.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:00.834+0000 7fe69dfcb640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T05:45:00.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:00.838+0000 7fe69dfcb640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-24T05:45:03.805 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T05:45:03.889 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T05:45:04.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:04.019+0000 7fc3a1012640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-24T05:45:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:04.019+0000 7fc3a1012640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-24T05:45:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:04.019+0000 7fc3a1012640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T05:45:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:04.019+0000 7fc3a1012640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T05:45:04.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:04.019+0000 7fc3a1012640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T05:45:04.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:04.022+0000 7fc3a1012640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-24T05:45:07.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T05:45:07.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T05:45:07.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:07.216+0000 7f5a2b222640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-24T05:45:07.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:07.216+0000 7f5a2b222640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-24T05:45:07.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:07.216+0000 7f5a2b222640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T05:45:07.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:07.216+0000 7f5a2b222640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T05:45:07.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:07.216+0000 7f5a2b222640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T05:45:07.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:07.219+0000 7f5a2b222640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-24T05:45:10.442 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T05:45:10.525 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T05:45:10.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:10.648+0000 7f65d1fa1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-24T05:45:10.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:10.648+0000 7f65d1fa1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-24T05:45:10.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:10.648+0000 7f65d1fa1640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T05:45:10.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:10.649+0000 7f65d1fa1640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T05:45:10.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:10.649+0000 7f65d1fa1640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T05:45:10.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:10.651+0000 7f65d1fa1640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-24T05:45:13.606 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T05:45:13.688 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-24T05:45:13.778 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T05:45:13.778 INFO:tasks.ceph:role: osd.4 2024-04-24T05:45:13.778 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-24T05:45:13.778 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T05:45:13.869 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T05:45:13.869 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T05:45:13.869 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T05:45:13.869 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T05:45:13.869 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T05:45:13.869 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-04-24T05:45:13.870 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T05:45:13.870 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T05:45:13.870 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T05:45:13.870 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T05:45:14.280 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-04-24T05:45:14.288 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-04-24T05:45:14.288 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-24T05:45:14.428 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-24T05:45:14.502 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-24T05:45:14.568 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T05:45:14.568 INFO:tasks.ceph:role: osd.5 2024-04-24T05:45:14.568 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-24T05:45:14.568 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T05:45:14.637 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T05:45:14.637 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T05:45:14.637 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T05:45:14.637 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T05:45:14.637 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T05:45:14.637 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-04-24T05:45:14.637 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T05:45:14.638 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T05:45:14.638 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T05:45:14.638 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T05:45:14.993 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-04-24T05:45:14.998 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-04-24T05:45:14.998 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-24T05:45:15.096 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-24T05:45:15.164 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-24T05:45:15.227 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T05:45:15.227 INFO:tasks.ceph:role: osd.6 2024-04-24T05:45:15.227 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-24T05:45:15.227 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T05:45:15.308 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T05:45:15.308 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T05:45:15.308 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T05:45:15.308 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T05:45:15.308 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T05:45:15.308 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-04-24T05:45:15.308 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T05:45:15.308 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T05:45:15.308 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T05:45:15.308 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T05:45:15.687 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-04-24T05:45:15.691 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-04-24T05:45:15.691 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-24T05:45:15.794 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-24T05:45:15.870 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-24T05:45:15.938 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T05:45:15.938 INFO:tasks.ceph:role: osd.7 2024-04-24T05:45:15.938 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi155.front.sepia.ceph.com 2024-04-24T05:45:15.938 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T05:45:16.013 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T05:45:16.013 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T05:45:16.013 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T05:45:16.013 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-24T05:45:16.013 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T05:45:16.013 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-04-24T05:45:16.014 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T05:45:16.014 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-24T05:45:16.014 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T05:45:16.014 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T05:45:16.445 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-04-24T05:45:16.452 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-04-24T05:45:16.452 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-24T05:45:16.614 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-24T05:45:16.705 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T05:45:16.836 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:16.834+0000 7f583240f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-24T05:45:16.837 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:16.835+0000 7f583240f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-24T05:45:16.837 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:16.835+0000 7f583240f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T05:45:16.837 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:16.835+0000 7f583240f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T05:45:16.837 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:16.835+0000 7f583240f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T05:45:16.840 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:16.839+0000 7f583240f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-24T05:45:20.071 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T05:45:20.161 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T05:45:20.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:20.285+0000 7f8cc6b44640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-24T05:45:20.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:20.286+0000 7f8cc6b44640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-24T05:45:20.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:20.286+0000 7f8cc6b44640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T05:45:20.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:20.286+0000 7f8cc6b44640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T05:45:20.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:20.286+0000 7f8cc6b44640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T05:45:20.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:20.289+0000 7f8cc6b44640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-24T05:45:22.299 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:45:22.307 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:45:23.245 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T05:45:23.329 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T05:45:23.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:23.455+0000 7f5e1340e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-24T05:45:23.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:23.456+0000 7f5e1340e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-24T05:45:23.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:23.456+0000 7f5e1340e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T05:45:23.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:23.456+0000 7f5e1340e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T05:45:23.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:23.456+0000 7f5e1340e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T05:45:23.460 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:23.458+0000 7f5e1340e640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-24T05:45:26.416 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-24T05:45:26.502 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T05:45:26.624 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:26.623+0000 7f6084bd8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-24T05:45:26.625 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:26.623+0000 7f6084bd8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-24T05:45:26.625 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:26.623+0000 7f6084bd8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T05:45:26.625 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:26.623+0000 7f6084bd8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T05:45:26.625 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:26.623+0000 7f6084bd8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T05:45:26.627 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:26.626+0000 7f6084bd8640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-24T05:45:29.848 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-24T05:45:29.937 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-24T05:45:29.938 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:45:29.938 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-24T05:45:30.028 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:45:30.029 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-24T05:45:30.113 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:45:30.114 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-24T05:45:30.197 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:45:30.197 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-24T05:45:30.276 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:45:30.276 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-24T05:45:30.402 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:45:30.403 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-24T05:45:30.484 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:45:30.484 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-24T05:45:30.565 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:45:30.565 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-24T05:45:30.655 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:45:30.655 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-24T05:45:30.740 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:45:30.740 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-24T05:45:30.823 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:45:30.823 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-24T05:45:30.903 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:45:30.903 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-24T05:45:30.982 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:45:30.982 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-24T05:45:31.063 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:45:31.063 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-24T05:45:31.146 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:45:31.146 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-24T05:45:31.205 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:45:31.205 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-24T05:45:31.256 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:45:31.256 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-24T05:45:31.314 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:45:31.314 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-24T05:45:31.366 INFO:tasks.ceph:Adding keys to all mons... 2024-04-24T05:45:31.367 DEBUG:teuthology.orchestra.run.smithi123:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T05:45:31.374 DEBUG:teuthology.orchestra.run.smithi155:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T05:45:31.450 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.y] 2024-04-24T05:45:31.451 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBXnChme+AYMxAAbFi71Rqyc9RBMexN54Eumg== 2024-04-24T05:45:31.451 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2024-04-24T05:45:31.451 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBYnChmnq3ZCxAA/jPeNwIiPPR/4z1oxDbmrg== 2024-04-24T05:45:31.451 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2024-04-24T05:45:31.451 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBYnChm1Lz0GRAA5I5XFsVJP/B2b5sJ8Xzfaw== 2024-04-24T05:45:31.451 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.0] 2024-04-24T05:45:31.451 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBcnChmstDGMRAAcjdLoMmhVSQxYoQ2DFmwgw== 2024-04-24T05:45:31.451 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.1] 2024-04-24T05:45:31.451 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBgnChmm/o3ARAAyWAABEnw+Bj0K2Y8C9V4TQ== 2024-04-24T05:45:31.451 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.2] 2024-04-24T05:45:31.451 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBjnChmxUXyDBAA+Te+IW8KZK9VlA6thlpGZQ== 2024-04-24T05:45:31.451 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.3] 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBmnChm6amzJhAAAr0M8SMHGIwpyWvaV2zkag== 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.x] 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBYnChmBMpBAhAAMObhb6ZHg0Zck7O/3zNiew== 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBYnChmJV4HKRAA4sb5ium46BxVm8vPrshrHg== 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBYnChm6IjANRAAOzvW4AE9/wb+nE7UERpUzg== 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.4] 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBsnChm6D7VMRAAp8Oejg6qgdkGE9scSWWHGw== 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.5] 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBwnChmJWQbERAA2/c/kpOLl5re5h2XnQehBw== 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.6] 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBznChmwGE9GxAAm5XkXoA0a7YHyKL4Dvojbg== 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.7] 2024-04-24T05:45:31.452 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB2nChmziE2JRAAeU2jmUbvOctMr4oc9FGuEQ== 2024-04-24T05:45:31.453 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2024-04-24T05:45:31.453 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBZnChmQ4k9BhAAqlK7y/LktbsFTIUhRQCMYg== 2024-04-24T05:45:31.453 INFO:teuthology.orchestra.run.smithi123.stdout:[client.2] 2024-04-24T05:45:31.453 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBZnChmH8+MDxAA6XyqkcDkR8RW5WwxIeDp1A== 2024-04-24T05:45:31.453 INFO:teuthology.orchestra.run.smithi123.stdout:[client.1] 2024-04-24T05:45:31.453 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBZnChmVke+GBAAUWUCOE5KNa5ZiOO2QJWQNw== 2024-04-24T05:45:31.453 INFO:teuthology.orchestra.run.smithi123.stdout:[client.3] 2024-04-24T05:45:31.453 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBZnChmqPdQIRAAzr7jrMslzCOsbGAHQKEH9w== 2024-04-24T05:45:31.454 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr.y] 2024-04-24T05:45:31.454 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBXnChme+AYMxAAbFi71Rqyc9RBMexN54Eumg== 2024-04-24T05:45:31.454 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.a] 2024-04-24T05:45:31.454 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBYnChmnq3ZCxAA/jPeNwIiPPR/4z1oxDbmrg== 2024-04-24T05:45:31.454 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.c] 2024-04-24T05:45:31.454 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBYnChm1Lz0GRAA5I5XFsVJP/B2b5sJ8Xzfaw== 2024-04-24T05:45:31.454 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.0] 2024-04-24T05:45:31.454 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBcnChmstDGMRAAcjdLoMmhVSQxYoQ2DFmwgw== 2024-04-24T05:45:31.454 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.1] 2024-04-24T05:45:31.454 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBgnChmm/o3ARAAyWAABEnw+Bj0K2Y8C9V4TQ== 2024-04-24T05:45:31.454 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.2] 2024-04-24T05:45:31.454 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBjnChmxUXyDBAA+Te+IW8KZK9VlA6thlpGZQ== 2024-04-24T05:45:31.455 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.3] 2024-04-24T05:45:31.455 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBmnChm6amzJhAAAr0M8SMHGIwpyWvaV2zkag== 2024-04-24T05:45:31.455 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr.x] 2024-04-24T05:45:31.455 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBYnChmBMpBAhAAMObhb6ZHg0Zck7O/3zNiew== 2024-04-24T05:45:31.455 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.b] 2024-04-24T05:45:31.455 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBYnChmJV4HKRAA4sb5ium46BxVm8vPrshrHg== 2024-04-24T05:45:31.455 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.d] 2024-04-24T05:45:31.455 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBYnChm6IjANRAAOzvW4AE9/wb+nE7UERpUzg== 2024-04-24T05:45:31.455 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.4] 2024-04-24T05:45:31.455 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBsnChm6D7VMRAAp8Oejg6qgdkGE9scSWWHGw== 2024-04-24T05:45:31.455 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.5] 2024-04-24T05:45:31.455 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBwnChmJWQbERAA2/c/kpOLl5re5h2XnQehBw== 2024-04-24T05:45:31.455 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.6] 2024-04-24T05:45:31.455 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBznChmwGE9GxAAm5XkXoA0a7YHyKL4Dvojbg== 2024-04-24T05:45:31.456 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.7] 2024-04-24T05:45:31.456 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQB2nChmziE2JRAAeU2jmUbvOctMr4oc9FGuEQ== 2024-04-24T05:45:31.456 INFO:teuthology.orchestra.run.smithi155.stdout:[client.0] 2024-04-24T05:45:31.456 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBZnChmQ4k9BhAAqlK7y/LktbsFTIUhRQCMYg== 2024-04-24T05:45:31.456 INFO:teuthology.orchestra.run.smithi155.stdout:[client.2] 2024-04-24T05:45:31.456 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBZnChmH8+MDxAA6XyqkcDkR8RW5WwxIeDp1A== 2024-04-24T05:45:31.456 INFO:teuthology.orchestra.run.smithi155.stdout:[client.1] 2024-04-24T05:45:31.456 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBZnChmVke+GBAAUWUCOE5KNa5ZiOO2QJWQNw== 2024-04-24T05:45:31.456 INFO:teuthology.orchestra.run.smithi155.stdout:[client.3] 2024-04-24T05:45:31.456 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQBZnChmqPdQIRAAzr7jrMslzCOsbGAHQKEH9w== 2024-04-24T05:45:31.458 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T05:45:31.505 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T05:45:31.647 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T05:45:31.655 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T05:45:31.789 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T05:45:31.797 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T05:45:31.929 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:31.936 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:32.066 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:32.071 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:32.196 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:32.203 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:32.338 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:32.343 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:32.475 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T05:45:32.480 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T05:45:32.607 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T05:45:32.611 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T05:45:32.740 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T05:45:32.745 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-24T05:45:32.880 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:32.885 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:33.010 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:33.015 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:33.147 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:33.154 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:33.282 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:33.287 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T05:45:33.422 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T05:45:33.429 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T05:45:33.570 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T05:45:33.577 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T05:45:33.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T05:45:33.715 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T05:45:33.852 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T05:45:33.860 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T05:45:33.995 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-24T05:45:33.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-24T05:45:34.077 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T05:45:34.226 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-24T05:45:34.296 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-24T05:45:34.388 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T05:45:34.536 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-24T05:45:34.608 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-24T05:45:34.685 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T05:45:34.828 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-24T05:45:34.903 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T05:45:34.910 DEBUG:teuthology.orchestra.run.smithi155:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T05:45:34.960 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-24T05:45:34.961 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-24T05:45:34.961 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-24T05:45:34.966 INFO:tasks.ceph.mon.a:Started 2024-04-24T05:45:34.966 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-24T05:45:34.966 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-24T05:45:34.970 INFO:tasks.ceph.mon.b:Started 2024-04-24T05:45:34.971 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-24T05:45:34.971 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-24T05:45:34.976 INFO:tasks.ceph.mon.c:Started 2024-04-24T05:45:34.976 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-24T05:45:34.977 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-24T05:45:34.977 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-24T05:45:34.982 INFO:tasks.ceph.mgr.y:Started 2024-04-24T05:45:34.982 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-24T05:45:34.982 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-24T05:45:34.986 INFO:tasks.ceph.mgr.x:Started 2024-04-24T05:45:34.987 INFO:tasks.ceph:Setting crush tunables to default 2024-04-24T05:45:34.987 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd crush tunables default 2024-04-24T05:45:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.161+0000 7f0cda000640 1 Processor -- start 2024-04-24T05:45:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.162+0000 7f0cda000640 1 -- start start 2024-04-24T05:45:35.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.162+0000 7f0cda000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 0x7f0cd4071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:35.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cda000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0cd40cd2c0 0x7f0cd406cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:35.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cda000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cd40cf170 con 0x7f0cd406c8f0 2024-04-24T05:45:35.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cda000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0cd40cf450 con 0x7f0cd40cd2c0 2024-04-24T05:45:35.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cda000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd40cf730 con 0x7f0cd406c510 2024-04-24T05:45:35.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cd3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0cd40cd2c0 0x7f0cd406cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:35.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cd3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0cd40cd2c0 0x7f0cd406cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37078/0 (socket says 172.21.15.123:37078) 2024-04-24T05:45:35.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cd3e00640 1 -- 172.21.15.123:0/2901186313 learned_addr learned my addr 172.21.15.123:0/2901186313 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:35.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cd3e00640 1 -- 172.21.15.123:0/2901186313 >> v2:172.21.15.123:3300/0 conn(0x7f0cd40cd2c0 msgr2=0x7f0cd406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-24T05:45:35.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cd3e00640 1 -- 172.21.15.123:0/2901186313 >> v2:172.21.15.123:3300/0 conn(0x7f0cd40cd2c0 msgr2=0x7f0cd406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T05:45:35.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cd3e00640 1 --2- 172.21.15.123:0/2901186313 >> v2:172.21.15.123:3300/0 conn(0x7f0cd40cd2c0 0x7f0cd406cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T05:45:35.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cd3e00640 1 --2- 172.21.15.123:0/2901186313 >> v2:172.21.15.123:3300/0 conn(0x7f0cd40cd2c0 0x7f0cd406cd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T05:45:35.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cd3400640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 0x7f0cd4071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:35.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cd3400640 1 -- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 msgr2=0x7f0cd4071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T05:45:35.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cd3400640 1 -- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 msgr2=0x7f0cd4071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T05:45:35.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cd3400640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 0x7f0cd4071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T05:45:35.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.163+0000 7f0cd3400640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 0x7f0cd4071c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T05:45:35.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.364+0000 7f0cd3400640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 0x7f0cd4071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:35.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.364+0000 7f0cd3e00640 1 --2- 172.21.15.123:0/2901186313 >> v2:172.21.15.123:3300/0 conn(0x7f0cd40cd2c0 0x7f0cd406cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:35.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.365+0000 7f0cd3400640 1 -- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 msgr2=0x7f0cd4071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-24T05:45:35.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.365+0000 7f0cd3400640 1 -- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 msgr2=0x7f0cd4071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T05:45:35.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.365+0000 7f0cd3400640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 0x7f0cd4071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T05:45:35.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.365+0000 7f0cd3400640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 0x7f0cd4071c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T05:45:35.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.365+0000 7f0cd3e00640 1 -- 172.21.15.123:0/2901186313 >> v2:172.21.15.123:3300/0 conn(0x7f0cd40cd2c0 msgr2=0x7f0cd406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-24T05:45:35.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.365+0000 7f0cd3e00640 1 -- 172.21.15.123:0/2901186313 >> v2:172.21.15.123:3300/0 conn(0x7f0cd40cd2c0 msgr2=0x7f0cd406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-24T05:45:35.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.365+0000 7f0cd3e00640 1 --2- 172.21.15.123:0/2901186313 >> v2:172.21.15.123:3300/0 conn(0x7f0cd40cd2c0 0x7f0cd406cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T05:45:35.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.365+0000 7f0cd3e00640 1 --2- 172.21.15.123:0/2901186313 >> v2:172.21.15.123:3300/0 conn(0x7f0cd40cd2c0 0x7f0cd406cd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T05:45:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.414+0000 7f0cd2a00640 1 -- 172.21.15.123:0/2901186313 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 724813195 0 0) 0x7f0cd40cf170 con 0x7f0cd406c8f0 2024-04-24T05:45:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.414+0000 7f0cd2a00640 1 -- 172.21.15.123:0/2901186313 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cb8003520 con 0x7f0cd406c8f0 2024-04-24T05:45:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.420+0000 7f0cd2a00640 1 -- 172.21.15.123:0/2901186313 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3486064022 0 0) 0x7f0cb8003520 con 0x7f0cd406c8f0 2024-04-24T05:45:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.420+0000 7f0cd2a00640 1 -- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 msgr2=0x7f0cd4071c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T05:45:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.420+0000 7f0cd2a00640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 0x7f0cd4071c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.420+0000 7f0cd2a00640 1 -- 172.21.15.123:0/2901186313 >> v2:172.21.15.123:3300/0 conn(0x7f0cd40cd2c0 msgr2=0x7f0cd406cd50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T05:45:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.420+0000 7f0cd2a00640 1 --2- 172.21.15.123:0/2901186313 >> v2:172.21.15.123:3300/0 conn(0x7f0cd40cd2c0 0x7f0cd406cd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.420+0000 7f0cd2a00640 1 -- 172.21.15.123:0/2901186313 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd40cfa10 con 0x7f0cd406c8f0 2024-04-24T05:45:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.420+0000 7f0cd2a00640 1 -- 172.21.15.123:0/2901186313 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f0cbc001670 con 0x7f0cd406c8f0 2024-04-24T05:45:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.420+0000 7f0cd2a00640 1 -- 172.21.15.123:0/2901186313 >> v1:172.21.15.155:6789/0 conn(0x7f0cd406c8f0 legacy=0x7f0cd40ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:35.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.420+0000 7f0cd2a00640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb80039b0 0x7f0cb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:35.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.420+0000 7f0cd2a00640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cb8004800 0x7f0cb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:35.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.421+0000 7f0cd2a00640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 0x7f0cb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:35.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.421+0000 7f0cd2a00640 1 -- 172.21.15.123:0/2901186313 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd40cf450 con 0x7f0cb8004800 2024-04-24T05:45:35.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.421+0000 7f0cd2a00640 1 -- 172.21.15.123:0/2901186313 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd40cf730 con 0x7f0cb80039b0 2024-04-24T05:45:35.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.421+0000 7f0cd2a00640 1 -- 172.21.15.123:0/2901186313 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb800bac0 con 0x7f0cd406c510 2024-04-24T05:45:35.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.421+0000 7f0ccaa00640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cb8004800 0x7f0cb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:35.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.421+0000 7f0cd3e00640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 0x7f0cb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:35.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.421+0000 7f0cd3400640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb80039b0 0x7f0cb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:35.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.422+0000 7f0ccaa00640 1 -- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 msgr2=0x7f0cb80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:35.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.422+0000 7f0ccaa00640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 0x7f0cb80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:35.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.422+0000 7f0ccaa00640 1 -- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb80039b0 msgr2=0x7f0cb8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:35.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.422+0000 7f0ccaa00640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb80039b0 0x7f0cb8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:35.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.422+0000 7f0ccaa00640 1 -- 172.21.15.123:0/2901186313 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb8003520 con 0x7f0cb8004800 2024-04-24T05:45:35.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.422+0000 7f0cd3e00640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cd406c510 0x7f0cb80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:45:35.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.424+0000 7f0ccaa00640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cb8004800 0x7f0cb8004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc007f50 tx=0x7f0cb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46240c60b8c243f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:35.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.425+0000 7f0cd2a00640 1 -- 172.21.15.123:0/2901186313 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc036070 con 0x7f0cb8004800 2024-04-24T05:45:35.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.425+0000 7f0cd2a00640 1 -- 172.21.15.123:0/2901186313 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cbc031620 con 0x7f0cb8004800 2024-04-24T05:45:35.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.425+0000 7f0cd2a00640 1 -- 172.21.15.123:0/2901186313 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc007bb0 con 0x7f0cb8004800 2024-04-24T05:45:35.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.426+0000 7f0cda000640 1 -- 172.21.15.123:0/2901186313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cb8004800 msgr2=0x7f0cb8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:35.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.426+0000 7f0cda000640 1 --2- 172.21.15.123:0/2901186313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cb8004800 0x7f0cb8004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc007f50 tx=0x7f0cb8005830 comp rx=0 tx=0).stop 2024-04-24T05:45:35.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.426+0000 7f0cd3e00640 1 -- 172.21.15.123:0/2901186313 reap_dead start 2024-04-24T05:45:35.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.427+0000 7f0cda000640 1 -- 172.21.15.123:0/2901186313 shutdown_connections 2024-04-24T05:45:35.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.427+0000 7f0cda000640 1 -- 172.21.15.123:0/2901186313 >> 172.21.15.123:0/2901186313 conn(0x7f0cd40b51f0 msgr2=0x7f0cd40b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:35.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.427+0000 7f0cda000640 1 -- 172.21.15.123:0/2901186313 shutdown_connections 2024-04-24T05:45:35.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.427+0000 7f0cda000640 1 -- 172.21.15.123:0/2901186313 wait complete. 2024-04-24T05:45:35.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.429+0000 7f0cda000640 1 Processor -- start 2024-04-24T05:45:35.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.430+0000 7f0cda000640 1 -- start start 2024-04-24T05:45:35.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.431+0000 7f0cda000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb8004800 0x7f0cd40aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:35.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.431+0000 7f0cda000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cd40ad480 0x7f0cd415f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:35.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.431+0000 7f0cda000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cd415c700 0x7f0cd415cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:35.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.431+0000 7f0cda000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd40cee80 con 0x7f0cd40ad480 2024-04-24T05:45:35.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.431+0000 7f0cda000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd40ced00 con 0x7f0cd415c700 2024-04-24T05:45:35.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.431+0000 7f0cda000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd4071c50 con 0x7f0cb8004800 2024-04-24T05:45:35.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.431+0000 7f0cd3400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb8004800 0x7f0cd40aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:35.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.431+0000 7f0ccaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cd40ad480 0x7f0cd415f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:35.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.432+0000 7f0cd3400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb8004800 0x7f0cd40aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:59674/0 (socket says 172.21.15.123:59674) 2024-04-24T05:45:35.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.432+0000 7f0cd3400640 1 -- 172.21.15.123:0/2681248414 learned_addr learned my addr 172.21.15.123:0/2681248414 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:35.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.432+0000 7f0cd3e00640 1 --2- 172.21.15.123:0/2681248414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cd415c700 0x7f0cd415cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:35.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.432+0000 7f0ccaa00640 1 -- 172.21.15.123:0/2681248414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb8004800 msgr2=0x7f0cd40aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:35.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.432+0000 7f0ccaa00640 1 --2- 172.21.15.123:0/2681248414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb8004800 0x7f0cd40aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:35.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.432+0000 7f0ccaa00640 1 -- 172.21.15.123:0/2681248414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cd415c700 msgr2=0x7f0cd415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:35.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.432+0000 7f0ccaa00640 1 --2- 172.21.15.123:0/2681248414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cd415c700 0x7f0cd415cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:35.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.432+0000 7f0ccaa00640 1 -- 172.21.15.123:0/2681248414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd415f850 con 0x7f0cd40ad480 2024-04-24T05:45:35.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.433+0000 7f0ccaa00640 1 --2- 172.21.15.123:0/2681248414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cd40ad480 0x7f0cd415f310 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc031000 tx=0x7f0cbc000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:35.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.433+0000 7f0ccbe00640 1 -- 172.21.15.123:0/2681248414 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc036030 con 0x7f0cd40ad480 2024-04-24T05:45:35.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.433+0000 7f0ccbe00640 1 -- 172.21.15.123:0/2681248414 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cbc037040 con 0x7f0cd40ad480 2024-04-24T05:45:35.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.433+0000 7f0cda000640 1 -- 172.21.15.123:0/2681248414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd40ada20 con 0x7f0cd40ad480 2024-04-24T05:45:35.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.433+0000 7f0ccbe00640 1 -- 172.21.15.123:0/2681248414 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc001c10 con 0x7f0cd40ad480 2024-04-24T05:45:35.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.434+0000 7f0cda000640 1 -- 172.21.15.123:0/2681248414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd40adfe0 con 0x7f0cd40ad480 2024-04-24T05:45:35.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.434+0000 7f0ccbe00640 1 -- 172.21.15.123:0/2681248414 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0cbc003980 con 0x7f0cd40ad480 2024-04-24T05:45:35.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.434+0000 7f0ccbe00640 1 -- 172.21.15.123:0/2681248414 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0cbc03cc10 con 0x7f0cd40ad480 2024-04-24T05:45:35.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.435+0000 7f0cda000640 1 -- 172.21.15.123:0/2681248414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd4068a20 con 0x7f0cd40ad480 2024-04-24T05:45:35.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.441+0000 7f0ccbe00640 1 -- 172.21.15.123:0/2681248414 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0cbc03c420 con 0x7f0cd40ad480 2024-04-24T05:45:35.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.515+0000 7f0cda000640 1 -- 172.21.15.123:0/2681248414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f0cd40732b0 con 0x7f0cd40ad480 2024-04-24T05:45:35.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.519+0000 7f0ccbe00640 1 -- 172.21.15.123:0/2681248414 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f0cbc04f480 con 0x7f0cd40ad480 2024-04-24T05:45:35.521 INFO:teuthology.orchestra.run.smithi123.stderr:adjusted tunables profile to default 2024-04-24T05:45:35.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.521+0000 7f0cda000640 1 -- 172.21.15.123:0/2681248414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cd40ad480 msgr2=0x7f0cd415f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:35.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.521+0000 7f0cda000640 1 --2- 172.21.15.123:0/2681248414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cd40ad480 0x7f0cd415f310 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc031000 tx=0x7f0cbc000fe0 comp rx=0 tx=0).stop 2024-04-24T05:45:35.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.521+0000 7f0cda000640 1 -- 172.21.15.123:0/2681248414 shutdown_connections 2024-04-24T05:45:35.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.521+0000 7f0cda000640 1 --2- 172.21.15.123:0/2681248414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cd415c700 0x7f0cd415cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:35.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.521+0000 7f0cda000640 1 --2- 172.21.15.123:0/2681248414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cd40ad480 0x7f0cd415f310 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:35.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.521+0000 7f0cda000640 1 --2- 172.21.15.123:0/2681248414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cb8004800 0x7f0cd40aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:35.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.521+0000 7f0cda000640 1 -- 172.21.15.123:0/2681248414 >> 172.21.15.123:0/2681248414 conn(0x7f0cd40b51f0 msgr2=0x7f0cd40b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:35.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.521+0000 7f0cda000640 1 -- 172.21.15.123:0/2681248414 shutdown_connections 2024-04-24T05:45:35.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:35.521+0000 7f0cda000640 1 -- 172.21.15.123:0/2681248414 wait complete. 2024-04-24T05:45:35.539 INFO:tasks.ceph:check_enable_crimson: False 2024-04-24T05:45:35.539 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-24T05:45:35.540 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:45:35.540 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-24T05:45:35.596 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:35.594+0000 7f3669693040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T05:45:35.603 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:35.601+0000 7f203f7b8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T05:45:35.610 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:45:35.610 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-24T05:45:35.730 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:45:35.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-24T05:45:35.801 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:35.800+0000 7f3669693040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T05:45:35.803 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:45:35.803 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-24T05:45:35.810 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:35.809+0000 7f203f7b8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T05:45:35.867 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:35.866+0000 7f3669693040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T05:45:35.878 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:35.877+0000 7f203f7b8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T05:45:35.880 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:45:35.880 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-24T05:45:35.958 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:45:35.958 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-24T05:45:36.027 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:45:36.027 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-24T05:45:36.141 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:45:36.142 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-24T05:45:36.187 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:36.185+0000 7f203f7b8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T05:45:36.215 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:36.213+0000 7f3669693040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T05:45:36.245 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:36.243+0000 7f203f7b8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T05:45:36.258 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 52b185c5-0e58-4115-b76d-15cf4b46e446 0 2024-04-24T05:45:36.272 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:36.270+0000 7f3669693040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T05:45:36.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.451+0000 7fcd92000640 1 Processor -- start 2024-04-24T05:45:36.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.451+0000 7fcd92000640 1 -- start start 2024-04-24T05:45:36.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.453+0000 7fcd92000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcd8c0ce7c0 0x7fcd8c0d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:36.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.453+0000 7fcd92000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd8c0d1e70 0x7fcd8c0717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:36.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.453+0000 7fcd92000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd8c0d11a0 con 0x7fcd8c0b2e30 2024-04-24T05:45:36.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.453+0000 7fcd92000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcd8c15d250 con 0x7fcd8c0ce7c0 2024-04-24T05:45:36.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.453+0000 7fcd92000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8c15d530 con 0x7fcd8c0d1e70 2024-04-24T05:45:36.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.453+0000 7fcd8be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcd8c0ce7c0 0x7fcd8c0d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:36.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.453+0000 7fcd91600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd8c0d1e70 0x7fcd8c0717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:36.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.453+0000 7fcd8be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcd8c0ce7c0 0x7fcd8c0d1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:36372/0 (socket says 172.21.15.155:36372) 2024-04-24T05:45:36.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.453+0000 7fcd8be00640 1 -- 172.21.15.155:0/1627750997 learned_addr learned my addr 172.21.15.155:0/1627750997 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-24T05:45:36.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.454+0000 7fcd91600640 1 -- 172.21.15.155:0/1627750997 >> v2:172.21.15.155:3300/0 conn(0x7fcd8c0ce7c0 msgr2=0x7fcd8c0d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:36.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.454+0000 7fcd91600640 1 --2- 172.21.15.155:0/1627750997 >> v2:172.21.15.155:3300/0 conn(0x7fcd8c0ce7c0 0x7fcd8c0d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.454+0000 7fcd91600640 1 -- 172.21.15.155:0/1627750997 >> v1:172.21.15.155:6789/0 conn(0x7fcd8c0b2e30 legacy=0x7fcd8c15c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:36.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.454+0000 7fcd91600640 1 -- 172.21.15.155:0/1627750997 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8c0d1410 con 0x7fcd8c0d1e70 2024-04-24T05:45:36.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.455+0000 7fcd91600640 1 --2- 172.21.15.155:0/1627750997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd8c0d1e70 0x7fcd8c0717e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009d90 tx=0x7fcd8402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7038fc151979586d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:36.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.455+0000 7fcd8b400640 1 -- 172.21.15.155:0/1627750997 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84031600 con 0x7fcd8c0d1e70 2024-04-24T05:45:36.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.455+0000 7fcd8b400640 1 -- 172.21.15.155:0/1627750997 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd840317a0 con 0x7fcd8c0d1e70 2024-04-24T05:45:36.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.455+0000 7fcd8b400640 1 -- 172.21.15.155:0/1627750997 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84031b70 con 0x7fcd8c0d1e70 2024-04-24T05:45:36.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.456+0000 7fcd92000640 1 -- 172.21.15.155:0/1627750997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd8c0d1e70 msgr2=0x7fcd8c0717e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:36.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.456+0000 7fcd92000640 1 --2- 172.21.15.155:0/1627750997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd8c0d1e70 0x7fcd8c0717e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009d90 tx=0x7fcd8402a5d0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.456+0000 7fcd92000640 1 -- 172.21.15.155:0/1627750997 shutdown_connections 2024-04-24T05:45:36.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.456+0000 7fcd92000640 1 --2- 172.21.15.155:0/1627750997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd8c0d1e70 0x7fcd8c0717e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.456+0000 7fcd92000640 1 --2- 172.21.15.155:0/1627750997 >> v2:172.21.15.155:3300/0 conn(0x7fcd8c0ce7c0 0x7fcd8c0d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.456+0000 7fcd92000640 1 -- 172.21.15.155:0/1627750997 >> 172.21.15.155:0/1627750997 conn(0x7fcd8c0b4180 msgr2=0x7fcd8c0b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:36.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.456+0000 7fcd92000640 1 -- 172.21.15.155:0/1627750997 shutdown_connections 2024-04-24T05:45:36.458 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.456+0000 7fcd92000640 1 -- 172.21.15.155:0/1627750997 wait complete. 2024-04-24T05:45:36.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.458+0000 7fcd92000640 1 Processor -- start 2024-04-24T05:45:36.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.458+0000 7fcd92000640 1 -- start start 2024-04-24T05:45:36.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd92000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd8c0b2e30 0x7fcd8c0cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:36.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd92000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8c0ce7c0 0x7fcd8c14e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:36.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd92000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd8c0d1e70 0x7fcd8c14ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:36.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd92000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c071d20 con 0x7fcd8c0d1e70 2024-04-24T05:45:36.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd92000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c15d810 con 0x7fcd8c0ce7c0 2024-04-24T05:45:36.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd92000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8c071ea0 con 0x7fcd8c0b2e30 2024-04-24T05:45:36.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd8be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8c0ce7c0 0x7fcd8c14e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:36.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd90c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd8c0b2e30 0x7fcd8c0cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:36.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd8be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8c0ce7c0 0x7fcd8c14e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:36374/0 (socket says 172.21.15.155:36374) 2024-04-24T05:45:36.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd8be00640 1 -- 172.21.15.155:0/3023340926 learned_addr learned my addr 172.21.15.155:0/3023340926 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-24T05:45:36.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd90c00640 1 --2- 172.21.15.155:0/3023340926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd8c0b2e30 0x7fcd8c0cc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:47960/0 (socket says 172.21.15.155:47960) 2024-04-24T05:45:36.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd91600640 1 --2- 172.21.15.155:0/3023340926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd8c0d1e70 0x7fcd8c14ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:36.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd8be00640 1 -- 172.21.15.155:0/3023340926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd8c0b2e30 msgr2=0x7fcd8c0cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:36.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd8be00640 1 --2- 172.21.15.155:0/3023340926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd8c0b2e30 0x7fcd8c0cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd8be00640 1 -- 172.21.15.155:0/3023340926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd8c0d1e70 msgr2=0x7fcd8c14ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:36.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd8be00640 1 --2- 172.21.15.155:0/3023340926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd8c0d1e70 0x7fcd8c14ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd8be00640 1 -- 172.21.15.155:0/3023340926 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8c06df50 con 0x7fcd8c0ce7c0 2024-04-24T05:45:36.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.459+0000 7fcd90c00640 1 --2- 172.21.15.155:0/3023340926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd8c0b2e30 0x7fcd8c0cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:45:36.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.460+0000 7fcd8be00640 1 --2- 172.21.15.155:0/3023340926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8c0ce7c0 0x7fcd8c14e810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800e870 tx=0x7fcd7800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:36.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.460+0000 7fcd88c00640 1 -- 172.21.15.155:0/3023340926 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78017070 con 0x7fcd8c0ce7c0 2024-04-24T05:45:36.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.460+0000 7fcd88c00640 1 -- 172.21.15.155:0/3023340926 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd7800c8e0 con 0x7fcd8c0ce7c0 2024-04-24T05:45:36.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.460+0000 7fcd92000640 1 -- 172.21.15.155:0/3023340926 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8c06e230 con 0x7fcd8c0ce7c0 2024-04-24T05:45:36.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.460+0000 7fcd88c00640 1 -- 172.21.15.155:0/3023340926 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7800cc90 con 0x7fcd8c0ce7c0 2024-04-24T05:45:36.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.460+0000 7fcd92000640 1 -- 172.21.15.155:0/3023340926 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd8c06e7f0 con 0x7fcd8c0ce7c0 2024-04-24T05:45:36.465 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.460+0000 7fcd88c00640 1 -- 172.21.15.155:0/3023340926 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcd7801a070 con 0x7fcd8c0ce7c0 2024-04-24T05:45:36.465 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.460+0000 7fcd88c00640 1 -- 172.21.15.155:0/3023340926 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fcd7801c820 con 0x7fcd8c0ce7c0 2024-04-24T05:45:36.466 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.461+0000 7fcd92000640 1 -- 172.21.15.155:0/3023340926 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd8c069130 con 0x7fcd8c0ce7c0 2024-04-24T05:45:36.466 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.464+0000 7fcd88c00640 1 -- 172.21.15.155:0/3023340926 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcd78013470 con 0x7fcd8c0ce7c0 2024-04-24T05:45:36.545 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.543+0000 7fcd92000640 1 -- 172.21.15.155:0/3023340926 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "52b185c5-0e58-4115-b76d-15cf4b46e446", "id": 0} v 0) -- 0x7fcd8c0b24d0 con 0x7fcd8c0ce7c0 2024-04-24T05:45:36.552 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.551+0000 7fcd88c00640 1 -- 172.21.15.155:0/3023340926 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "52b185c5-0e58-4115-b76d-15cf4b46e446", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fcd7801caf0 con 0x7fcd8c0ce7c0 2024-04-24T05:45:36.553 INFO:teuthology.orchestra.run.smithi155.stdout:0 2024-04-24T05:45:36.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.553+0000 7fcd92000640 1 -- 172.21.15.155:0/3023340926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8c0ce7c0 msgr2=0x7fcd8c14e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:36.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.553+0000 7fcd92000640 1 --2- 172.21.15.155:0/3023340926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8c0ce7c0 0x7fcd8c14e810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800e870 tx=0x7fcd7800ed30 comp rx=0 tx=0).stop 2024-04-24T05:45:36.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.553+0000 7fcd92000640 1 -- 172.21.15.155:0/3023340926 shutdown_connections 2024-04-24T05:45:36.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.553+0000 7fcd92000640 1 --2- 172.21.15.155:0/3023340926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd8c0d1e70 0x7fcd8c14ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.553+0000 7fcd92000640 1 --2- 172.21.15.155:0/3023340926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8c0ce7c0 0x7fcd8c14e810 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.553+0000 7fcd92000640 1 --2- 172.21.15.155:0/3023340926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd8c0b2e30 0x7fcd8c0cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.553+0000 7fcd92000640 1 -- 172.21.15.155:0/3023340926 >> 172.21.15.155:0/3023340926 conn(0x7fcd8c0b4180 msgr2=0x7fcd8c15eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:36.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.553+0000 7fcd92000640 1 -- 172.21.15.155:0/3023340926 shutdown_connections 2024-04-24T05:45:36.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.553+0000 7fcd92000640 1 -- 172.21.15.155:0/3023340926 wait complete. 2024-04-24T05:45:36.572 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new d11108b7-2b27-456b-8fb0-ef134af5b33a 1 2024-04-24T05:45:36.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.745+0000 7f15f1e00640 1 Processor -- start 2024-04-24T05:45:36.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.745+0000 7f15f1e00640 1 -- start start 2024-04-24T05:45:36.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.746+0000 7f15f1e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f15ec0b6420 0x7f15ec0d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:36.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.747+0000 7f15f1e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15ec0b6910 con 0x7f15ec0cfdb0 2024-04-24T05:45:36.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.747+0000 7f15f1e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15ec0b6bb0 con 0x7f15ec0d3bf0 2024-04-24T05:45:36.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.747+0000 7f15f1e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f15ec0d3760 con 0x7f15ec0b6420 2024-04-24T05:45:36.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.747+0000 7f15ebe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f15ec0b6420 0x7f15ec0d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:36.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.747+0000 7f15ebe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f15ec0b6420 0x7f15ec0d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:36390/0 (socket says 172.21.15.155:36390) 2024-04-24T05:45:36.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.747+0000 7f15ebe00640 1 -- 172.21.15.155:0/1092009703 learned_addr learned my addr 172.21.15.155:0/1092009703 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-24T05:45:36.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.747+0000 7f15f1400640 1 --1- 172.21.15.155:0/1092009703 >> v1:172.21.15.155:6789/0 conn(0x7f15ec0d3bf0 0x7f15ec0d4080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:45262/0 (socket says 172.21.15.155:45262) 2024-04-24T05:45:36.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.747+0000 7f15ebe00640 1 -- 172.21.15.155:0/1092009703 >> v1:172.21.15.155:6789/0 conn(0x7f15ec0d3bf0 legacy=0x7f15ec0d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:36.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.747+0000 7f15ebe00640 1 -- 172.21.15.155:0/1092009703 >> v1:172.21.15.123:6789/0 conn(0x7f15ec0cfdb0 legacy=0x7f15ec0d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:36.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.747+0000 7f15ebe00640 1 -- 172.21.15.155:0/1092009703 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ec0d33f0 con 0x7f15ec0b6420 2024-04-24T05:45:36.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.748+0000 7f15ebe00640 1 --2- 172.21.15.155:0/1092009703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec0b6420 0x7f15ec0d16e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009870 tx=0x7f15d802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f64661013406489 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:36.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.749+0000 7f15eb400640 1 -- 172.21.15.155:0/1092009703 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8037070 con 0x7f15ec0b6420 2024-04-24T05:45:36.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.749+0000 7f15eb400640 1 -- 172.21.15.155:0/1092009703 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d8032440 con 0x7f15ec0b6420 2024-04-24T05:45:36.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.749+0000 7f15eb400640 1 -- 172.21.15.155:0/1092009703 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8032810 con 0x7f15ec0b6420 2024-04-24T05:45:36.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.749+0000 7f15f1e00640 1 -- 172.21.15.155:0/1092009703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec0b6420 msgr2=0x7f15ec0d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:36.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.749+0000 7f15f1e00640 1 --2- 172.21.15.155:0/1092009703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec0b6420 0x7f15ec0d16e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009870 tx=0x7f15d802a270 comp rx=0 tx=0).stop 2024-04-24T05:45:36.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.749+0000 7f15f1e00640 1 -- 172.21.15.155:0/1092009703 shutdown_connections 2024-04-24T05:45:36.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.749+0000 7f15f1e00640 1 --2- 172.21.15.155:0/1092009703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec0b6420 0x7f15ec0d16e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.749+0000 7f15f1e00640 1 -- 172.21.15.155:0/1092009703 >> 172.21.15.155:0/1092009703 conn(0x7f15ec08bf70 msgr2=0x7f15ec09d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:36.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.750+0000 7f15f1e00640 1 -- 172.21.15.155:0/1092009703 shutdown_connections 2024-04-24T05:45:36.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.750+0000 7f15f1e00640 1 -- 172.21.15.155:0/1092009703 wait complete. 2024-04-24T05:45:36.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.751+0000 7f15f1e00640 1 Processor -- start 2024-04-24T05:45:36.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.751+0000 7f15f1e00640 1 -- start start 2024-04-24T05:45:36.755 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.752+0000 7f15f1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ec0b6420 0x7f15ec0b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:36.755 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.752+0000 7f15f1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec0cfdb0 0x7f15ec0c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:36.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.752+0000 7f15f1e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15ec0d3bf0 0x7f15ec0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:36.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.752+0000 7f15f1e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec0d1da0 con 0x7f15ec0b6420 2024-04-24T05:45:36.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.752+0000 7f15f1e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec0d1c20 con 0x7f15ec0cfdb0 2024-04-24T05:45:36.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.752+0000 7f15f1e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f15ec15e900 con 0x7f15ec0d3bf0 2024-04-24T05:45:36.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.753+0000 7f15ebe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec0cfdb0 0x7f15ec0c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:36.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.753+0000 7f15f1400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15ec0d3bf0 0x7f15ec0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:36.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.753+0000 7f15ebe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec0cfdb0 0x7f15ec0c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:36404/0 (socket says 172.21.15.155:36404) 2024-04-24T05:45:36.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.753+0000 7f15ebe00640 1 -- 172.21.15.155:0/3693268622 learned_addr learned my addr 172.21.15.155:0/3693268622 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-24T05:45:36.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.753+0000 7f15f0a00640 1 --2- 172.21.15.155:0/3693268622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ec0b6420 0x7f15ec0b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:36.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.753+0000 7f15ebe00640 1 -- 172.21.15.155:0/3693268622 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15ec0d3bf0 msgr2=0x7f15ec0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:36.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.753+0000 7f15ebe00640 1 --2- 172.21.15.155:0/3693268622 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15ec0d3bf0 0x7f15ec0cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.753+0000 7f15ebe00640 1 -- 172.21.15.155:0/3693268622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ec0b6420 msgr2=0x7f15ec0b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:36.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.753+0000 7f15ebe00640 1 --2- 172.21.15.155:0/3693268622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ec0b6420 0x7f15ec0b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.753+0000 7f15ebe00640 1 -- 172.21.15.155:0/3693268622 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ec15edc0 con 0x7f15ec0cfdb0 2024-04-24T05:45:36.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.753+0000 7f15ebe00640 1 --2- 172.21.15.155:0/3693268622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec0cfdb0 0x7f15ec0c8140 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f15d8000c00 tx=0x7f15d80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:36.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.754+0000 7f15e8c00640 1 -- 172.21.15.155:0/3693268622 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8037040 con 0x7f15ec0cfdb0 2024-04-24T05:45:36.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.754+0000 7f15e8c00640 1 -- 172.21.15.155:0/3693268622 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d8042070 con 0x7f15ec0cfdb0 2024-04-24T05:45:36.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.754+0000 7f15e8c00640 1 -- 172.21.15.155:0/3693268622 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8030bd0 con 0x7f15ec0cfdb0 2024-04-24T05:45:36.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.754+0000 7f15f1e00640 1 -- 172.21.15.155:0/3693268622 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ec15f0a0 con 0x7f15ec0cfdb0 2024-04-24T05:45:36.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.754+0000 7f15f1e00640 1 -- 172.21.15.155:0/3693268622 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ec15f660 con 0x7f15ec0cfdb0 2024-04-24T05:45:36.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.754+0000 7f15e8c00640 1 -- 172.21.15.155:0/3693268622 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f15d8047460 con 0x7f15ec0cfdb0 2024-04-24T05:45:36.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.754+0000 7f15e8c00640 1 -- 172.21.15.155:0/3693268622 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f15d803d7f0 con 0x7f15ec0cfdb0 2024-04-24T05:45:36.758 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.755+0000 7f15f1e00640 1 -- 172.21.15.155:0/3693268622 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ec068a20 con 0x7f15ec0cfdb0 2024-04-24T05:45:36.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.758+0000 7f15e8c00640 1 -- 172.21.15.155:0/3693268622 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f15d803d9f0 con 0x7f15ec0cfdb0 2024-04-24T05:45:36.840 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.838+0000 7f15f1e00640 1 -- 172.21.15.155:0/3693268622 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d11108b7-2b27-456b-8fb0-ef134af5b33a", "id": 1} v 0) -- 0x7f15ec074e70 con 0x7f15ec0cfdb0 2024-04-24T05:45:36.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.845+0000 7f15e8c00640 1 -- 172.21.15.155:0/3693268622 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d11108b7-2b27-456b-8fb0-ef134af5b33a", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f15d803dbd0 con 0x7f15ec0cfdb0 2024-04-24T05:45:36.847 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-24T05:45:36.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.847+0000 7f15f1e00640 1 -- 172.21.15.155:0/3693268622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec0cfdb0 msgr2=0x7f15ec0c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:36.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.847+0000 7f15f1e00640 1 --2- 172.21.15.155:0/3693268622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec0cfdb0 0x7f15ec0c8140 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f15d8000c00 tx=0x7f15d80046c0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.849 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.847+0000 7f15f1e00640 1 -- 172.21.15.155:0/3693268622 shutdown_connections 2024-04-24T05:45:36.849 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.847+0000 7f15f1e00640 1 --2- 172.21.15.155:0/3693268622 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15ec0d3bf0 0x7f15ec0cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.849 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.847+0000 7f15f1e00640 1 --2- 172.21.15.155:0/3693268622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15ec0cfdb0 0x7f15ec0c8140 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.849 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.847+0000 7f15f1e00640 1 --2- 172.21.15.155:0/3693268622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ec0b6420 0x7f15ec0b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:36.849 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.847+0000 7f15f1e00640 1 -- 172.21.15.155:0/3693268622 >> 172.21.15.155:0/3693268622 conn(0x7f15ec08bf70 msgr2=0x7f15ec0b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:36.849 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.847+0000 7f15f1e00640 1 -- 172.21.15.155:0/3693268622 shutdown_connections 2024-04-24T05:45:36.849 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:36.847+0000 7f15f1e00640 1 -- 172.21.15.155:0/3693268622 wait complete. 2024-04-24T05:45:36.865 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc 2 2024-04-24T05:45:37.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.033+0000 7f644fa00640 1 Processor -- start 2024-04-24T05:45:37.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.033+0000 7f644fa00640 1 -- start start 2024-04-24T05:45:37.036 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.035+0000 7f644fa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f64480d3e10 0x7f64480d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.035+0000 7f644fa00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64480cfdf0 con 0x7f644815e970 2024-04-24T05:45:37.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.035+0000 7f644fa00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64480d0090 con 0x7f64480d4c00 2024-04-24T05:45:37.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.035+0000 7f644fa00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f64480d09c0 con 0x7f64480d3e10 2024-04-24T05:45:37.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.035+0000 7f6447e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f64480d3e10 0x7f64480d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.035+0000 7f644da00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f64480d4c00 0x7f64480d1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:51010/0 (socket says 172.21.15.155:51010) 2024-04-24T05:45:37.037 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.035+0000 7f644da00640 1 -- 172.21.15.155:0/3468110781 learned_addr learned my addr 172.21.15.155:0/3468110781 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-24T05:45:37.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.036+0000 7f6447e00640 1 -- 172.21.15.155:0/3468110781 >> v1:172.21.15.155:6789/0 conn(0x7f64480d4c00 legacy=0x7f64480d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.036+0000 7f6447e00640 1 -- 172.21.15.155:0/3468110781 >> v1:172.21.15.123:6789/0 conn(0x7f644815e970 legacy=0x7f64480d44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.036+0000 7f6447400640 1 -- 172.21.15.155:0/3468110781 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3121168479 0 0) 0x7f64480d0090 con 0x7f64480d4c00 2024-04-24T05:45:37.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.036+0000 7f6447e00640 1 -- 172.21.15.155:0/3468110781 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64480d0370 con 0x7f64480d3e10 2024-04-24T05:45:37.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.037+0000 7f6447e00640 1 --2- 172.21.15.155:0/3468110781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64480d3e10 0x7f64480d0e50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f64340097a0 tx=0x7f643402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7859ef0ecf9d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:37.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.037+0000 7f6447400640 1 -- 172.21.15.155:0/3468110781 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434037070 con 0x7f64480d3e10 2024-04-24T05:45:37.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.037+0000 7f6447400640 1 -- 172.21.15.155:0/3468110781 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6434032440 con 0x7f64480d3e10 2024-04-24T05:45:37.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.037+0000 7f6447400640 1 -- 172.21.15.155:0/3468110781 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434032810 con 0x7f64480d3e10 2024-04-24T05:45:37.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.037+0000 7f644fa00640 1 -- 172.21.15.155:0/3468110781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64480d3e10 msgr2=0x7f64480d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.038+0000 7f644fa00640 1 --2- 172.21.15.155:0/3468110781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64480d3e10 0x7f64480d0e50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f64340097a0 tx=0x7f643402a1a0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.038+0000 7f644fa00640 1 -- 172.21.15.155:0/3468110781 shutdown_connections 2024-04-24T05:45:37.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.038+0000 7f644fa00640 1 --2- 172.21.15.155:0/3468110781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64480d3e10 0x7f64480d0e50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.038+0000 7f644fa00640 1 -- 172.21.15.155:0/3468110781 >> 172.21.15.155:0/3468110781 conn(0x7f64480b9a40 msgr2=0x7f64480b75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:37.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.038+0000 7f644fa00640 1 -- 172.21.15.155:0/3468110781 shutdown_connections 2024-04-24T05:45:37.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.038+0000 7f644fa00640 1 -- 172.21.15.155:0/3468110781 wait complete. 2024-04-24T05:45:37.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.039+0000 7f644fa00640 1 Processor -- start 2024-04-24T05:45:37.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.039+0000 7f644fa00640 1 -- start start 2024-04-24T05:45:37.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.040+0000 7f644fa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64480d3e10 0x7f644806e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.040+0000 7f644fa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64480d4c00 0x7f64480b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.040+0000 7f644fa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f644815e970 0x7f64481f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.040+0000 7f644fa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64480b12b0 con 0x7f64480d4c00 2024-04-24T05:45:37.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.040+0000 7f644fa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f644815ed50 con 0x7f644815e970 2024-04-24T05:45:37.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.040+0000 7f644fa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f64480d53a0 con 0x7f64480d3e10 2024-04-24T05:45:37.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.041+0000 7f644d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64480d3e10 0x7f644806e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.041+0000 7f644da00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f644815e970 0x7f64481f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.041+0000 7f644d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64480d3e10 0x7f644806e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:42876/0 (socket says 172.21.15.155:42876) 2024-04-24T05:45:37.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.041+0000 7f644d000640 1 -- 172.21.15.155:0/1613819981 learned_addr learned my addr 172.21.15.155:0/1613819981 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-24T05:45:37.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.041+0000 7f6447e00640 1 --2- 172.21.15.155:0/1613819981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64480d4c00 0x7f64480b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.044 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.041+0000 7f644d000640 1 -- 172.21.15.155:0/1613819981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f644815e970 msgr2=0x7f64481f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.041+0000 7f644d000640 1 --2- 172.21.15.155:0/1613819981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f644815e970 0x7f64481f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.041+0000 7f644d000640 1 -- 172.21.15.155:0/1613819981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64480d4c00 msgr2=0x7f64480b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.041+0000 7f644d000640 1 --2- 172.21.15.155:0/1613819981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64480d4c00 0x7f64480b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.041+0000 7f644d000640 1 -- 172.21.15.155:0/1613819981 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644806ba00 con 0x7f64480d3e10 2024-04-24T05:45:37.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.041+0000 7f644d000640 1 --2- 172.21.15.155:0/1613819981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64480d3e10 0x7f644806e690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f643c00c400 tx=0x7f643c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:37.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.042+0000 7f6444c00640 1 -- 172.21.15.155:0/1613819981 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c017070 con 0x7f64480d3e10 2024-04-24T05:45:37.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.042+0000 7f6444c00640 1 -- 172.21.15.155:0/1613819981 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f643c0052d0 con 0x7f64480d3e10 2024-04-24T05:45:37.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.042+0000 7f6444c00640 1 -- 172.21.15.155:0/1613819981 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c003830 con 0x7f64480d3e10 2024-04-24T05:45:37.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.042+0000 7f644fa00640 1 -- 172.21.15.155:0/1613819981 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64480b4b10 con 0x7f64480d3e10 2024-04-24T05:45:37.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.042+0000 7f644fa00640 1 -- 172.21.15.155:0/1613819981 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f644806bf80 con 0x7f64480d3e10 2024-04-24T05:45:37.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.042+0000 7f6444c00640 1 -- 172.21.15.155:0/1613819981 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f643c01a070 con 0x7f64480d3e10 2024-04-24T05:45:37.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.042+0000 7f6444c00640 1 -- 172.21.15.155:0/1613819981 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f643c012420 con 0x7f64480d3e10 2024-04-24T05:45:37.046 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.043+0000 7f644fa00640 1 -- 172.21.15.155:0/1613819981 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6448068a20 con 0x7f64480d3e10 2024-04-24T05:45:37.047 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.046+0000 7f6444c00640 1 -- 172.21.15.155:0/1613819981 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f643c021070 con 0x7f64480d3e10 2024-04-24T05:45:37.126 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.125+0000 7f644fa00640 1 -- 172.21.15.155:0/1613819981 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc", "id": 2} v 0) -- 0x7f64480d3350 con 0x7f64480d3e10 2024-04-24T05:45:37.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.130+0000 7f6444c00640 1 -- 172.21.15.155:0/1613819981 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f643c0115d0 con 0x7f64480d3e10 2024-04-24T05:45:37.132 INFO:teuthology.orchestra.run.smithi155.stdout:2 2024-04-24T05:45:37.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.132+0000 7f644fa00640 1 -- 172.21.15.155:0/1613819981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64480d3e10 msgr2=0x7f644806e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.132+0000 7f644fa00640 1 --2- 172.21.15.155:0/1613819981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64480d3e10 0x7f644806e690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f643c00c400 tx=0x7f643c00c8c0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.132+0000 7f644fa00640 1 -- 172.21.15.155:0/1613819981 shutdown_connections 2024-04-24T05:45:37.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.132+0000 7f644fa00640 1 --2- 172.21.15.155:0/1613819981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f644815e970 0x7f64481f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.132+0000 7f644fa00640 1 --2- 172.21.15.155:0/1613819981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64480d4c00 0x7f64480b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.132+0000 7f644fa00640 1 --2- 172.21.15.155:0/1613819981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64480d3e10 0x7f644806e690 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.132+0000 7f644fa00640 1 -- 172.21.15.155:0/1613819981 >> 172.21.15.155:0/1613819981 conn(0x7f64480b9a40 msgr2=0x7f644815da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:37.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.132+0000 7f644fa00640 1 -- 172.21.15.155:0/1613819981 shutdown_connections 2024-04-24T05:45:37.134 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.132+0000 7f644fa00640 1 -- 172.21.15.155:0/1613819981 wait complete. 2024-04-24T05:45:37.150 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 7ecbd207-67c6-418f-b481-96074db16fd8 3 2024-04-24T05:45:37.211 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:37.209+0000 7f203f7b8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T05:45:37.226 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:37.225+0000 7f3669693040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T05:45:37.323 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:37.321+0000 7f203f7b8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T05:45:37.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.322+0000 7f43cbe00640 1 Processor -- start 2024-04-24T05:45:37.324 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.323+0000 7f43cbe00640 1 -- start start 2024-04-24T05:45:37.326 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.324+0000 7f43cbe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f43cc0d4c00 0x7f43cc169200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.326 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.324+0000 7f43cbe00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43cc0b6910 con 0x7f43cc0cfdb0 2024-04-24T05:45:37.326 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.324+0000 7f43cbe00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43cc0b6bb0 con 0x7f43cc0b6420 2024-04-24T05:45:37.326 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.324+0000 7f43cbe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f43cc0d3760 con 0x7f43cc0d4c00 2024-04-24T05:45:37.326 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.324+0000 7f43cb400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f43cc0d4c00 0x7f43cc169200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.326 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.324+0000 7f43cb400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f43cc0d4c00 0x7f43cc169200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:60568/0 (socket says 172.21.15.155:60568) 2024-04-24T05:45:37.327 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.324+0000 7f43cb400640 1 -- 172.21.15.155:0/3148639523 learned_addr learned my addr 172.21.15.155:0/3148639523 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-24T05:45:37.327 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.324+0000 7f43c2000640 1 --1- 172.21.15.155:0/3148639523 >> v1:172.21.15.155:6789/0 conn(0x7f43cc0b6420 0x7f43cc0d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:51012/0 (socket says 172.21.15.155:51012) 2024-04-24T05:45:37.327 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.325+0000 7f43cb400640 1 -- 172.21.15.155:0/3148639523 >> v1:172.21.15.155:6789/0 conn(0x7f43cc0b6420 legacy=0x7f43cc0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.327 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.325+0000 7f43cb400640 1 -- 172.21.15.155:0/3148639523 >> v1:172.21.15.123:6789/0 conn(0x7f43cc0cfdb0 legacy=0x7f43cc0d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.327 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.325+0000 7f43cb400640 1 -- 172.21.15.155:0/3148639523 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc0d33f0 con 0x7f43cc0d4c00 2024-04-24T05:45:37.327 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.326+0000 7f43cb400640 1 --2- 172.21.15.155:0/3148639523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc0d4c00 0x7f43cc169200 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0d38e0 tx=0x7f43c402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ccafc6bb1d89a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:37.328 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.326+0000 7f43ca000640 1 -- 172.21.15.155:0/3148639523 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4037070 con 0x7f43cc0d4c00 2024-04-24T05:45:37.328 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.326+0000 7f43ca000640 1 -- 172.21.15.155:0/3148639523 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c4032440 con 0x7f43cc0d4c00 2024-04-24T05:45:37.328 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.326+0000 7f43ca000640 1 -- 172.21.15.155:0/3148639523 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4032810 con 0x7f43cc0d4c00 2024-04-24T05:45:37.329 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.327+0000 7f43cbe00640 1 -- 172.21.15.155:0/3148639523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc0d4c00 msgr2=0x7f43cc169200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.329 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.327+0000 7f43cbe00640 1 --2- 172.21.15.155:0/3148639523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc0d4c00 0x7f43cc169200 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0d38e0 tx=0x7f43c402a250 comp rx=0 tx=0).stop 2024-04-24T05:45:37.329 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.327+0000 7f43cbe00640 1 -- 172.21.15.155:0/3148639523 shutdown_connections 2024-04-24T05:45:37.329 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.327+0000 7f43cbe00640 1 --2- 172.21.15.155:0/3148639523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc0d4c00 0x7f43cc169200 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.329 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.327+0000 7f43cbe00640 1 -- 172.21.15.155:0/3148639523 >> 172.21.15.155:0/3148639523 conn(0x7f43cc08bf70 msgr2=0x7f43cc09d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:37.329 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.327+0000 7f43cbe00640 1 -- 172.21.15.155:0/3148639523 shutdown_connections 2024-04-24T05:45:37.329 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.327+0000 7f43cbe00640 1 -- 172.21.15.155:0/3148639523 wait complete. 2024-04-24T05:45:37.330 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.329+0000 7f43cbe00640 1 Processor -- start 2024-04-24T05:45:37.330 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.329+0000 7f43cbe00640 1 -- start start 2024-04-24T05:45:37.332 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.329+0000 7f43cbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc0b6420 0x7f43cc0cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.329+0000 7f43cbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43cc0cfdb0 0x7f43cc0ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.330+0000 7f43cbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc0d4c00 0x7f43cc0b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.330+0000 7f43cbe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc15ea80 con 0x7f43cc0d4c00 2024-04-24T05:45:37.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.330+0000 7f43cbe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc15e900 con 0x7f43cc0b6420 2024-04-24T05:45:37.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.330+0000 7f43cbe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f43cc169740 con 0x7f43cc0cfdb0 2024-04-24T05:45:37.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.330+0000 7f43caa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc0b6420 0x7f43cc0cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.330+0000 7f43c2000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43cc0cfdb0 0x7f43cc0ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.330+0000 7f43caa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc0b6420 0x7f43cc0cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:60584/0 (socket says 172.21.15.155:60584) 2024-04-24T05:45:37.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.330+0000 7f43caa00640 1 -- 172.21.15.155:0/843450555 learned_addr learned my addr 172.21.15.155:0/843450555 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-24T05:45:37.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.330+0000 7f43cb400640 1 --2- 172.21.15.155:0/843450555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc0d4c00 0x7f43cc0b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.330+0000 7f43caa00640 1 -- 172.21.15.155:0/843450555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43cc0cfdb0 msgr2=0x7f43cc0ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.330+0000 7f43caa00640 1 --2- 172.21.15.155:0/843450555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43cc0cfdb0 0x7f43cc0ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.333 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.330+0000 7f43caa00640 1 -- 172.21.15.155:0/843450555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc0d4c00 msgr2=0x7f43cc0b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.330+0000 7f43caa00640 1 --2- 172.21.15.155:0/843450555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc0d4c00 0x7f43cc0b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.330+0000 7f43caa00640 1 -- 172.21.15.155:0/843450555 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc0ccbf0 con 0x7f43cc0b6420 2024-04-24T05:45:37.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.331+0000 7f43caa00640 1 --2- 172.21.15.155:0/843450555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc0b6420 0x7f43cc0cc610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00c770 tx=0x7f43bc00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:37.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.331+0000 7f43c3400640 1 -- 172.21.15.155:0/843450555 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc016020 con 0x7f43cc0b6420 2024-04-24T05:45:37.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.331+0000 7f43c3400640 1 -- 172.21.15.155:0/843450555 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43bc006de0 con 0x7f43cc0b6420 2024-04-24T05:45:37.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.331+0000 7f43c3400640 1 -- 172.21.15.155:0/843450555 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc003c00 con 0x7f43cc0b6420 2024-04-24T05:45:37.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.331+0000 7f43cbe00640 1 -- 172.21.15.155:0/843450555 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43cc0cced0 con 0x7f43cc0b6420 2024-04-24T05:45:37.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.331+0000 7f43cbe00640 1 -- 172.21.15.155:0/843450555 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43cc0cd770 con 0x7f43cc0b6420 2024-04-24T05:45:37.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.331+0000 7f43c3400640 1 -- 172.21.15.155:0/843450555 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f43bc00abe0 con 0x7f43cc0b6420 2024-04-24T05:45:37.334 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.331+0000 7f43c3400640 1 -- 172.21.15.155:0/843450555 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f43bc005560 con 0x7f43cc0b6420 2024-04-24T05:45:37.335 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.332+0000 7f43cbe00640 1 -- 172.21.15.155:0/843450555 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43cc0cd0a0 con 0x7f43cc0b6420 2024-04-24T05:45:37.336 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.335+0000 7f43c3400640 1 -- 172.21.15.155:0/843450555 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f43bc020020 con 0x7f43cc0b6420 2024-04-24T05:45:37.342 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:37.340+0000 7f3669693040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T05:45:37.379 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:37.378+0000 7f203f7b8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T05:45:37.405 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:37.404+0000 7f3669693040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T05:45:37.431 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.429+0000 7f43cbe00640 1 -- 172.21.15.155:0/843450555 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7ecbd207-67c6-418f-b481-96074db16fd8", "id": 3} v 0) -- 0x7f43cc0cd970 con 0x7f43cc0b6420 2024-04-24T05:45:37.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.438+0000 7f43c3400640 1 -- 172.21.15.155:0/843450555 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7ecbd207-67c6-418f-b481-96074db16fd8", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f43cc0cd970 con 0x7f43cc0b6420 2024-04-24T05:45:37.440 INFO:teuthology.orchestra.run.smithi155.stdout:3 2024-04-24T05:45:37.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.439+0000 7f43cbe00640 1 -- 172.21.15.155:0/843450555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc0b6420 msgr2=0x7f43cc0cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.439+0000 7f43cbe00640 1 --2- 172.21.15.155:0/843450555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc0b6420 0x7f43cc0cc610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00c770 tx=0x7f43bc00cc30 comp rx=0 tx=0).stop 2024-04-24T05:45:37.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.439+0000 7f43cbe00640 1 -- 172.21.15.155:0/843450555 shutdown_connections 2024-04-24T05:45:37.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.439+0000 7f43cbe00640 1 --2- 172.21.15.155:0/843450555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43cc0d4c00 0x7f43cc0b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.439+0000 7f43cbe00640 1 --2- 172.21.15.155:0/843450555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43cc0cfdb0 0x7f43cc0ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.439+0000 7f43cbe00640 1 --2- 172.21.15.155:0/843450555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43cc0b6420 0x7f43cc0cc610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.439+0000 7f43cbe00640 1 -- 172.21.15.155:0/843450555 >> 172.21.15.155:0/843450555 conn(0x7f43cc08bf70 msgr2=0x7f43cc0b5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:37.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.440+0000 7f43cbe00640 1 -- 172.21.15.155:0/843450555 shutdown_connections 2024-04-24T05:45:37.441 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.440+0000 7f43cbe00640 1 -- 172.21.15.155:0/843450555 wait complete. 2024-04-24T05:45:37.458 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 1fe968d4-69f8-4f2c-a1ad-455c263fd3c3 4 2024-04-24T05:45:37.589 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:37.587+0000 7f203f7b8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T05:45:37.610 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:37.609+0000 7f3669693040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T05:45:37.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.629+0000 7f8f4fe00640 1 Processor -- start 2024-04-24T05:45:37.631 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.629+0000 7f8f4fe00640 1 -- start start 2024-04-24T05:45:37.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.631+0000 7f8f4fe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8f500cfdb0 0x7f8f500d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.631+0000 7f8f4fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8f500b6420 0x7f8f500d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.631+0000 7f8f4fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500ceb40 0x7f8f500cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.631+0000 7f8f4fe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8f500d3760 con 0x7f8f500b6420 2024-04-24T05:45:37.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.631+0000 7f8f4fe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8f500b6910 con 0x7f8f500cfdb0 2024-04-24T05:45:37.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.631+0000 7f8f4fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8f500b6bb0 con 0x7f8f500ceb40 2024-04-24T05:45:37.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.631+0000 7f8f4ea00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8f500cfdb0 0x7f8f500d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.632+0000 7f8f4f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500ceb40 0x7f8f500cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.632+0000 7f8f4ea00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8f500cfdb0 0x7f8f500d0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:60596/0 (socket says 172.21.15.155:60596) 2024-04-24T05:45:37.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.632+0000 7f8f4ea00640 1 -- 172.21.15.155:0/1005914999 learned_addr learned my addr 172.21.15.155:0/1005914999 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-24T05:45:37.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.632+0000 7f8f46000640 1 --2- 172.21.15.155:0/1005914999 >> v2:172.21.15.123:3300/0 conn(0x7f8f500b6420 0x7f8f500d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.632+0000 7f8f4ea00640 1 -- 172.21.15.155:0/1005914999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500ceb40 msgr2=0x7f8f500cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:37.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.632+0000 7f8f4ea00640 1 --2- 172.21.15.155:0/1005914999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500ceb40 0x7f8f500cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.632+0000 7f8f4ea00640 1 -- 172.21.15.155:0/1005914999 >> v2:172.21.15.123:3300/0 conn(0x7f8f500b6420 msgr2=0x7f8f500d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:37.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.632+0000 7f8f4ea00640 1 --2- 172.21.15.155:0/1005914999 >> v2:172.21.15.123:3300/0 conn(0x7f8f500b6420 0x7f8f500d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.632+0000 7f8f4ea00640 1 -- 172.21.15.155:0/1005914999 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f500d06d0 con 0x7f8f500cfdb0 2024-04-24T05:45:37.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.632+0000 7f8f4f400640 1 --2- 172.21.15.155:0/1005914999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500ceb40 0x7f8f500cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:45:37.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.633+0000 7f8f4ea00640 1 --2- 172.21.15.155:0/1005914999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500cfdb0 0x7f8f500d0190 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8f500cfd80 tx=0x7f8f4002a120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f19c9ab07a70d27d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:37.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.633+0000 7f8f4e000640 1 -- 172.21.15.155:0/1005914999 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40004030 con 0x7f8f500cfdb0 2024-04-24T05:45:37.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.633+0000 7f8f4e000640 1 -- 172.21.15.155:0/1005914999 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f400041d0 con 0x7f8f500cfdb0 2024-04-24T05:45:37.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.633+0000 7f8f4e000640 1 -- 172.21.15.155:0/1005914999 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40003610 con 0x7f8f500cfdb0 2024-04-24T05:45:37.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.634+0000 7f8f4fe00640 1 -- 172.21.15.155:0/1005914999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500cfdb0 msgr2=0x7f8f500d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.634+0000 7f8f4fe00640 1 --2- 172.21.15.155:0/1005914999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500cfdb0 0x7f8f500d0190 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8f500cfd80 tx=0x7f8f4002a120 comp rx=0 tx=0).stop 2024-04-24T05:45:37.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.634+0000 7f8f4fe00640 1 -- 172.21.15.155:0/1005914999 shutdown_connections 2024-04-24T05:45:37.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.634+0000 7f8f4fe00640 1 --2- 172.21.15.155:0/1005914999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500ceb40 0x7f8f500cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.634+0000 7f8f4fe00640 1 --2- 172.21.15.155:0/1005914999 >> v2:172.21.15.123:3300/0 conn(0x7f8f500b6420 0x7f8f500d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.634+0000 7f8f4fe00640 1 --2- 172.21.15.155:0/1005914999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500cfdb0 0x7f8f500d0190 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.634+0000 7f8f4fe00640 1 -- 172.21.15.155:0/1005914999 >> 172.21.15.155:0/1005914999 conn(0x7f8f5008bf70 msgr2=0x7f8f5009d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:37.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.634+0000 7f8f4fe00640 1 -- 172.21.15.155:0/1005914999 shutdown_connections 2024-04-24T05:45:37.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.634+0000 7f8f4fe00640 1 -- 172.21.15.155:0/1005914999 wait complete. 2024-04-24T05:45:37.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.637+0000 7f8f4fe00640 1 Processor -- start 2024-04-24T05:45:37.638 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.637+0000 7f8f4fe00640 1 -- start start 2024-04-24T05:45:37.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.638+0000 7f8f4fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f500b6420 0x7f8f5015f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f4fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500ceb40 0x7f8f500cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f4fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500cfdb0 0x7f8f500c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f4fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f500d1c20 con 0x7f8f500b6420 2024-04-24T05:45:37.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f4fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f500d3070 con 0x7f8f500ceb40 2024-04-24T05:45:37.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f4fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8f500d1da0 con 0x7f8f500cfdb0 2024-04-24T05:45:37.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f46000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500ceb40 0x7f8f500cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f4f400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500cfdb0 0x7f8f500c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.642 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f46000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500ceb40 0x7f8f500cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:60608/0 (socket says 172.21.15.155:60608) 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f46000640 1 -- 172.21.15.155:0/782024010 learned_addr learned my addr 172.21.15.155:0/782024010 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f4ea00640 1 --2- 172.21.15.155:0/782024010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f500b6420 0x7f8f5015f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f4f400640 1 -- 172.21.15.155:0/782024010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500ceb40 msgr2=0x7f8f500cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f4f400640 1 --2- 172.21.15.155:0/782024010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500ceb40 0x7f8f500cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f4f400640 1 -- 172.21.15.155:0/782024010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f500b6420 msgr2=0x7f8f5015f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f4f400640 1 --2- 172.21.15.155:0/782024010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f500b6420 0x7f8f5015f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f4f400640 1 -- 172.21.15.155:0/782024010 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5015fe50 con 0x7f8f500cfdb0 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.639+0000 7f8f46000640 1 --2- 172.21.15.155:0/782024010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500ceb40 0x7f8f500cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.640+0000 7f8f4f400640 1 --2- 172.21.15.155:0/782024010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500cfdb0 0x7f8f500c8ca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800efc0 tx=0x7f8f4800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.640+0000 7f8f47400640 1 -- 172.21.15.155:0/782024010 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48018030 con 0x7f8f500cfdb0 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.640+0000 7f8f47400640 1 -- 172.21.15.155:0/782024010 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4800cd30 con 0x7f8f500cfdb0 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.640+0000 7f8f47400640 1 -- 172.21.15.155:0/782024010 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48003bb0 con 0x7f8f500cfdb0 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.640+0000 7f8f4fe00640 1 -- 172.21.15.155:0/782024010 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f50160130 con 0x7f8f500cfdb0 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.640+0000 7f8f4fe00640 1 -- 172.21.15.155:0/782024010 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f501606f0 con 0x7f8f500cfdb0 2024-04-24T05:45:37.643 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.640+0000 7f8f47400640 1 -- 172.21.15.155:0/782024010 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8f4801b070 con 0x7f8f500cfdb0 2024-04-24T05:45:37.644 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.641+0000 7f8f47400640 1 -- 172.21.15.155:0/782024010 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f8f48004530 con 0x7f8f500cfdb0 2024-04-24T05:45:37.644 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.641+0000 7f8f4fe00640 1 -- 172.21.15.155:0/782024010 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f50068a20 con 0x7f8f500cfdb0 2024-04-24T05:45:37.645 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.644+0000 7f8f47400640 1 -- 172.21.15.155:0/782024010 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8f48010040 con 0x7f8f500cfdb0 2024-04-24T05:45:37.650 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:37.649+0000 7f203f7b8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T05:45:37.686 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:37.685+0000 7f3669693040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T05:45:37.717 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:37.715+0000 7f203f7b8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T05:45:37.719 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.717+0000 7f8f4fe00640 1 -- 172.21.15.155:0/782024010 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1fe968d4-69f8-4f2c-a1ad-455c263fd3c3", "id": 4} v 0) -- 0x7f8f500b5700 con 0x7f8f500cfdb0 2024-04-24T05:45:37.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.725+0000 7f8f47400640 1 -- 172.21.15.155:0/782024010 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1fe968d4-69f8-4f2c-a1ad-455c263fd3c3", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f8f48003d50 con 0x7f8f500cfdb0 2024-04-24T05:45:37.727 INFO:teuthology.orchestra.run.smithi155.stdout:4 2024-04-24T05:45:37.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.727+0000 7f8f4fe00640 1 -- 172.21.15.155:0/782024010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500cfdb0 msgr2=0x7f8f500c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.727+0000 7f8f4fe00640 1 --2- 172.21.15.155:0/782024010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500cfdb0 0x7f8f500c8ca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800efc0 tx=0x7f8f4800c490 comp rx=0 tx=0).stop 2024-04-24T05:45:37.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.727+0000 7f8f4fe00640 1 -- 172.21.15.155:0/782024010 shutdown_connections 2024-04-24T05:45:37.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.727+0000 7f8f4fe00640 1 --2- 172.21.15.155:0/782024010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500cfdb0 0x7f8f500c8ca0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.727+0000 7f8f4fe00640 1 --2- 172.21.15.155:0/782024010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500ceb40 0x7f8f500cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.727+0000 7f8f4fe00640 1 --2- 172.21.15.155:0/782024010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f500b6420 0x7f8f5015f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.727+0000 7f8f4fe00640 1 -- 172.21.15.155:0/782024010 >> 172.21.15.155:0/782024010 conn(0x7f8f5008bf70 msgr2=0x7f8f500b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:37.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.727+0000 7f8f4fe00640 1 -- 172.21.15.155:0/782024010 shutdown_connections 2024-04-24T05:45:37.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.727+0000 7f8f4fe00640 1 -- 172.21.15.155:0/782024010 wait complete. 2024-04-24T05:45:37.745 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new b3f5abd3-ba17-4227-a466-b75c3b10ba79 5 2024-04-24T05:45:37.752 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:37.750+0000 7f3669693040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T05:45:37.845 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:37.844+0000 7f203f7b8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T05:45:37.886 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:37.885+0000 7f3669693040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T05:45:37.924 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.923+0000 7f1e7aa00640 1 Processor -- start 2024-04-24T05:45:37.924 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.923+0000 7f1e7aa00640 1 -- start start 2024-04-24T05:45:37.926 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.924+0000 7f1e7aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740cfdb0 0x7f1e740d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.926 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.924+0000 7f1e7aa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1e740b6420 0x7f1e740d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.926 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.924+0000 7f1e7aa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e740ceb40 0x7f1e740cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.926 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.924+0000 7f1e7aa00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1e740d3760 con 0x7f1e740b6420 2024-04-24T05:45:37.926 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.924+0000 7f1e7aa00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1e740b6910 con 0x7f1e740ceb40 2024-04-24T05:45:37.926 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.924+0000 7f1e7aa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e740b6bb0 con 0x7f1e740cfdb0 2024-04-24T05:45:37.926 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.924+0000 7f1e79600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740cfdb0 0x7f1e740d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.926 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.924+0000 7f1e7a000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e740ceb40 0x7f1e740cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.926 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.924+0000 7f1e7a000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e740ceb40 0x7f1e740cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:60612/0 (socket says 172.21.15.155:60612) 2024-04-24T05:45:37.926 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.924+0000 7f1e7a000640 1 -- 172.21.15.155:0/800209822 learned_addr learned my addr 172.21.15.155:0/800209822 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-24T05:45:37.926 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.924+0000 7f1e78c00640 1 --2- 172.21.15.155:0/800209822 >> v2:172.21.15.123:3300/0 conn(0x7f1e740b6420 0x7f1e740d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.927 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.925+0000 7f1e7a000640 1 -- 172.21.15.155:0/800209822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740cfdb0 msgr2=0x7f1e740d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:37.927 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.925+0000 7f1e7a000640 1 --2- 172.21.15.155:0/800209822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740cfdb0 0x7f1e740d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.925+0000 7f1e7a000640 1 -- 172.21.15.155:0/800209822 >> v2:172.21.15.123:3300/0 conn(0x7f1e740b6420 msgr2=0x7f1e740d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:37.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.925+0000 7f1e7a000640 1 --2- 172.21.15.155:0/800209822 >> v2:172.21.15.123:3300/0 conn(0x7f1e740b6420 0x7f1e740d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.925+0000 7f1e7a000640 1 -- 172.21.15.155:0/800209822 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e740d06d0 con 0x7f1e740ceb40 2024-04-24T05:45:37.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.925+0000 7f1e78c00640 1 --2- 172.21.15.155:0/800209822 >> v2:172.21.15.123:3300/0 conn(0x7f1e740b6420 0x7f1e740d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:45:37.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.926+0000 7f1e7a000640 1 --2- 172.21.15.155:0/800209822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e740ceb40 0x7f1e740cefd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1e740b9660 tx=0x7f1e6c02a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27ed5e7201f24f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:37.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.926+0000 7f1e73e00640 1 -- 172.21.15.155:0/800209822 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c036070 con 0x7f1e740ceb40 2024-04-24T05:45:37.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.926+0000 7f1e73e00640 1 -- 172.21.15.155:0/800209822 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e6c031600 con 0x7f1e740ceb40 2024-04-24T05:45:37.928 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.926+0000 7f1e73e00640 1 -- 172.21.15.155:0/800209822 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c0319d0 con 0x7f1e740ceb40 2024-04-24T05:45:37.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.927+0000 7f1e7aa00640 1 -- 172.21.15.155:0/800209822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e740ceb40 msgr2=0x7f1e740cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.927+0000 7f1e7aa00640 1 --2- 172.21.15.155:0/800209822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e740ceb40 0x7f1e740cefd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1e740b9660 tx=0x7f1e6c02a610 comp rx=0 tx=0).stop 2024-04-24T05:45:37.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.927+0000 7f1e7aa00640 1 -- 172.21.15.155:0/800209822 shutdown_connections 2024-04-24T05:45:37.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.927+0000 7f1e7aa00640 1 --2- 172.21.15.155:0/800209822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e740ceb40 0x7f1e740cefd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.927+0000 7f1e7aa00640 1 --2- 172.21.15.155:0/800209822 >> v2:172.21.15.123:3300/0 conn(0x7f1e740b6420 0x7f1e740d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.927+0000 7f1e7aa00640 1 --2- 172.21.15.155:0/800209822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740cfdb0 0x7f1e740d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.927+0000 7f1e7aa00640 1 -- 172.21.15.155:0/800209822 >> 172.21.15.155:0/800209822 conn(0x7f1e7408bf70 msgr2=0x7f1e7409d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:37.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.927+0000 7f1e7aa00640 1 -- 172.21.15.155:0/800209822 shutdown_connections 2024-04-24T05:45:37.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.927+0000 7f1e7aa00640 1 -- 172.21.15.155:0/800209822 wait complete. 2024-04-24T05:45:37.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.929+0000 7f1e7aa00640 1 Processor -- start 2024-04-24T05:45:37.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.929+0000 7f1e7aa00640 1 -- start start 2024-04-24T05:45:37.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e7aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e740b6420 0x7f1e741610f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e7aa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740ceb40 0x7f1e740cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e7aa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e740cfdb0 0x7f1e7406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:37.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e7aa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1e740d1c20 con 0x7f1e740b6420 2024-04-24T05:45:37.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e7aa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1e740d3070 con 0x7f1e740cfdb0 2024-04-24T05:45:37.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e7aa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e740d1da0 con 0x7f1e740ceb40 2024-04-24T05:45:37.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e78c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740ceb40 0x7f1e740cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e78c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740ceb40 0x7f1e740cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:42934/0 (socket says 172.21.15.155:42934) 2024-04-24T05:45:37.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e78c00640 1 -- 172.21.15.155:0/234342486 learned_addr learned my addr 172.21.15.155:0/234342486 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-24T05:45:37.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e7a000640 1 --2- 172.21.15.155:0/234342486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e740cfdb0 0x7f1e7406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e79600640 1 --2- 172.21.15.155:0/234342486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e740b6420 0x7f1e741610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:37.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e7a000640 1 -- 172.21.15.155:0/234342486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740ceb40 msgr2=0x7f1e740cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e7a000640 1 --2- 172.21.15.155:0/234342486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740ceb40 0x7f1e740cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e7a000640 1 -- 172.21.15.155:0/234342486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e740b6420 msgr2=0x7f1e741610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:37.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e7a000640 1 --2- 172.21.15.155:0/234342486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e740b6420 0x7f1e741610f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:37.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e7a000640 1 -- 172.21.15.155:0/234342486 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7406e060 con 0x7f1e740cfdb0 2024-04-24T05:45:37.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.930+0000 7f1e78c00640 1 --2- 172.21.15.155:0/234342486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740ceb40 0x7f1e740cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:45:37.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.931+0000 7f1e7a000640 1 --2- 172.21.15.155:0/234342486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e740cfdb0 0x7f1e7406d1e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c030000 tx=0x7f1e6c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:37.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.931+0000 7f1e71600640 1 -- 172.21.15.155:0/234342486 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c041020 con 0x7f1e740cfdb0 2024-04-24T05:45:37.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.931+0000 7f1e71600640 1 -- 172.21.15.155:0/234342486 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e6c0077a0 con 0x7f1e740cfdb0 2024-04-24T05:45:37.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.931+0000 7f1e71600640 1 -- 172.21.15.155:0/234342486 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c036040 con 0x7f1e740cfdb0 2024-04-24T05:45:37.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.931+0000 7f1e7aa00640 1 -- 172.21.15.155:0/234342486 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7406e340 con 0x7f1e740cfdb0 2024-04-24T05:45:37.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.931+0000 7f1e7aa00640 1 -- 172.21.15.155:0/234342486 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e74161630 con 0x7f1e740cfdb0 2024-04-24T05:45:37.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.931+0000 7f1e71600640 1 -- 172.21.15.155:0/234342486 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1e6c044020 con 0x7f1e740cfdb0 2024-04-24T05:45:37.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.932+0000 7f1e71600640 1 -- 172.21.15.155:0/234342486 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f1e6c007b90 con 0x7f1e740cfdb0 2024-04-24T05:45:37.937 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.932+0000 7f1e7aa00640 1 -- 172.21.15.155:0/234342486 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e7415ea80 con 0x7f1e740cfdb0 2024-04-24T05:45:37.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:37.935+0000 7f1e71600640 1 -- 172.21.15.155:0/234342486 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1e6c04b030 con 0x7f1e740cfdb0 2024-04-24T05:45:38.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.013+0000 7f1e7aa00640 1 -- 172.21.15.155:0/234342486 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b3f5abd3-ba17-4227-a466-b75c3b10ba79", "id": 5} v 0) -- 0x7f1e74068a20 con 0x7f1e740cfdb0 2024-04-24T05:45:38.022 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.020+0000 7f1e71600640 1 -- 172.21.15.155:0/234342486 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b3f5abd3-ba17-4227-a466-b75c3b10ba79", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f1e6c03c650 con 0x7f1e740cfdb0 2024-04-24T05:45:38.022 INFO:teuthology.orchestra.run.smithi155.stdout:5 2024-04-24T05:45:38.024 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.022+0000 7f1e7aa00640 1 -- 172.21.15.155:0/234342486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e740cfdb0 msgr2=0x7f1e7406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.024 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.022+0000 7f1e7aa00640 1 --2- 172.21.15.155:0/234342486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e740cfdb0 0x7f1e7406d1e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c030000 tx=0x7f1e6c004a40 comp rx=0 tx=0).stop 2024-04-24T05:45:38.024 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.022+0000 7f1e7aa00640 1 -- 172.21.15.155:0/234342486 shutdown_connections 2024-04-24T05:45:38.024 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.022+0000 7f1e7aa00640 1 --2- 172.21.15.155:0/234342486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e740cfdb0 0x7f1e7406d1e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.024 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.022+0000 7f1e7aa00640 1 --2- 172.21.15.155:0/234342486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740ceb40 0x7f1e740cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.024 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.022+0000 7f1e7aa00640 1 --2- 172.21.15.155:0/234342486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e740b6420 0x7f1e741610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.024 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.022+0000 7f1e7aa00640 1 -- 172.21.15.155:0/234342486 >> 172.21.15.155:0/234342486 conn(0x7f1e7408bf70 msgr2=0x7f1e740b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:38.024 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.022+0000 7f1e7aa00640 1 -- 172.21.15.155:0/234342486 shutdown_connections 2024-04-24T05:45:38.024 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.022+0000 7f1e7aa00640 1 -- 172.21.15.155:0/234342486 wait complete. 2024-04-24T05:45:38.040 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 6973cbac-812d-478e-aec9-add25fd81549 6 2024-04-24T05:45:38.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.263+0000 7fa6cca00640 1 Processor -- start 2024-04-24T05:45:38.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.263+0000 7fa6cca00640 1 -- start start 2024-04-24T05:45:38.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.265+0000 7fa6cca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6c8158b80 0x7fa6c815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.265+0000 7fa6cca00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa6c8073400 0x7fa6c80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.265+0000 7fa6cca00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6c815f690 con 0x7fa6c815f110 2024-04-24T05:45:38.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.265+0000 7fa6cca00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa6c815f970 con 0x7fa6c8073400 2024-04-24T05:45:38.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.265+0000 7fa6cca00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c80cefd0 con 0x7fa6c8158b80 2024-04-24T05:45:38.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.265+0000 7fa6c6800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6c8158b80 0x7fa6c815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.265+0000 7fa6c6800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6c8158b80 0x7fa6c815ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:42938/0 (socket says 172.21.15.155:42938) 2024-04-24T05:45:38.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.265+0000 7fa6c6800640 1 -- 172.21.15.155:0/2827753994 learned_addr learned my addr 172.21.15.155:0/2827753994 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-24T05:45:38.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.265+0000 7fa6c6800640 1 -- 172.21.15.155:0/2827753994 >> v2:172.21.15.155:3300/0 conn(0x7fa6c8073400 msgr2=0x7fa6c80b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:38.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.265+0000 7fa6c7200640 1 --2- 172.21.15.155:0/2827753994 >> v2:172.21.15.155:3300/0 conn(0x7fa6c8073400 0x7fa6c80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.266+0000 7fa6c6800640 1 --2- 172.21.15.155:0/2827753994 >> v2:172.21.15.155:3300/0 conn(0x7fa6c8073400 0x7fa6c80b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.266+0000 7fa6c6800640 1 -- 172.21.15.155:0/2827753994 >> v1:172.21.15.123:6789/0 conn(0x7fa6c815f110 legacy=0x7fa6c8072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.266+0000 7fa6c6800640 1 -- 172.21.15.155:0/2827753994 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c80cf2b0 con 0x7fa6c8158b80 2024-04-24T05:45:38.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.266+0000 7fa6c7200640 1 --2- 172.21.15.155:0/2827753994 >> v2:172.21.15.155:3300/0 conn(0x7fa6c8073400 0x7fa6c80b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:45:38.269 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.267+0000 7fa6c6800640 1 --2- 172.21.15.155:0/2827753994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6c8158b80 0x7fa6c815ebd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009870 tx=0x7fa6b802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a885ada09e800bf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:38.269 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.267+0000 7fa6c5400640 1 -- 172.21.15.155:0/2827753994 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8037070 con 0x7fa6c8158b80 2024-04-24T05:45:38.269 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.267+0000 7fa6c5400640 1 -- 172.21.15.155:0/2827753994 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b8032440 con 0x7fa6c8158b80 2024-04-24T05:45:38.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.267+0000 7fa6c5400640 1 -- 172.21.15.155:0/2827753994 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8032810 con 0x7fa6c8158b80 2024-04-24T05:45:38.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.267+0000 7fa6cca00640 1 -- 172.21.15.155:0/2827753994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6c8158b80 msgr2=0x7fa6c815ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.267+0000 7fa6cca00640 1 --2- 172.21.15.155:0/2827753994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6c8158b80 0x7fa6c815ebd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009870 tx=0x7fa6b802a250 comp rx=0 tx=0).stop 2024-04-24T05:45:38.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.268+0000 7fa6cca00640 1 -- 172.21.15.155:0/2827753994 shutdown_connections 2024-04-24T05:45:38.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.268+0000 7fa6cca00640 1 --2- 172.21.15.155:0/2827753994 >> v2:172.21.15.155:3300/0 conn(0x7fa6c8073400 0x7fa6c80b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.268+0000 7fa6cca00640 1 --2- 172.21.15.155:0/2827753994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6c8158b80 0x7fa6c815ebd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.268+0000 7fa6cca00640 1 -- 172.21.15.155:0/2827753994 >> 172.21.15.155:0/2827753994 conn(0x7fa6c80d1280 msgr2=0x7fa6c80d1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:38.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.268+0000 7fa6cca00640 1 -- 172.21.15.155:0/2827753994 shutdown_connections 2024-04-24T05:45:38.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.268+0000 7fa6cca00640 1 -- 172.21.15.155:0/2827753994 wait complete. 2024-04-24T05:45:38.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.269+0000 7fa6cca00640 1 Processor -- start 2024-04-24T05:45:38.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.270+0000 7fa6cca00640 1 -- start start 2024-04-24T05:45:38.273 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.270+0000 7fa6cca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6c8073400 0x7fa6c815d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.273 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.270+0000 7fa6cca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6c8158b80 0x7fa6c80ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.270+0000 7fa6cca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6c815f110 0x7fa6c81f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6cca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8074090 con 0x7fa6c815f110 2024-04-24T05:45:38.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6cca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8073f10 con 0x7fa6c8073400 2024-04-24T05:45:38.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6cca00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c80b5300 con 0x7fa6c8158b80 2024-04-24T05:45:38.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6c6800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6c8073400 0x7fa6c815d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6c5e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6c8158b80 0x7fa6c80ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6c5e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6c8158b80 0x7fa6c80ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:42946/0 (socket says 172.21.15.155:42946) 2024-04-24T05:45:38.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6c5e00640 1 -- 172.21.15.155:0/2461419023 learned_addr learned my addr 172.21.15.155:0/2461419023 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-24T05:45:38.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6c6800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6c8073400 0x7fa6c815d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:60636/0 (socket says 172.21.15.155:60636) 2024-04-24T05:45:38.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6c7200640 1 --2- 172.21.15.155:0/2461419023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6c815f110 0x7fa6c81f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6c5e00640 1 -- 172.21.15.155:0/2461419023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6c8073400 msgr2=0x7fa6c815d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6c5e00640 1 --2- 172.21.15.155:0/2461419023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6c8073400 0x7fa6c815d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6c5e00640 1 -- 172.21.15.155:0/2461419023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6c815f110 msgr2=0x7fa6c81f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6c5e00640 1 --2- 172.21.15.155:0/2461419023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6c815f110 0x7fa6c81f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6c5e00640 1 -- 172.21.15.155:0/2461419023 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c81f5020 con 0x7fa6c8158b80 2024-04-24T05:45:38.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.271+0000 7fa6c6800640 1 --2- 172.21.15.155:0/2461419023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6c8073400 0x7fa6c815d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:45:38.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.272+0000 7fa6c5e00640 1 --2- 172.21.15.155:0/2461419023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6c8158b80 0x7fa6c80ccc20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400c3f0 tx=0x7fa6b400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:38.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.272+0000 7fa6bea00640 1 -- 172.21.15.155:0/2461419023 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40052d0 con 0x7fa6c8158b80 2024-04-24T05:45:38.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.272+0000 7fa6bea00640 1 -- 172.21.15.155:0/2461419023 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b4005490 con 0x7fa6c8158b80 2024-04-24T05:45:38.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.272+0000 7fa6bea00640 1 -- 172.21.15.155:0/2461419023 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4007260 con 0x7fa6c8158b80 2024-04-24T05:45:38.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.272+0000 7fa6cca00640 1 -- 172.21.15.155:0/2461419023 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c815dc10 con 0x7fa6c8158b80 2024-04-24T05:45:38.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.272+0000 7fa6cca00640 1 -- 172.21.15.155:0/2461419023 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c815e1d0 con 0x7fa6c8158b80 2024-04-24T05:45:38.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.272+0000 7fa6bea00640 1 -- 172.21.15.155:0/2461419023 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa6b4007400 con 0x7fa6c8158b80 2024-04-24T05:45:38.275 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.272+0000 7fa6bea00640 1 -- 172.21.15.155:0/2461419023 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fa6b40067d0 con 0x7fa6c8158b80 2024-04-24T05:45:38.276 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.273+0000 7fa6cca00640 1 -- 172.21.15.155:0/2461419023 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c8069130 con 0x7fa6c8158b80 2024-04-24T05:45:38.276 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.275+0000 7fa6bea00640 1 -- 172.21.15.155:0/2461419023 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa6c81f5020 con 0x7fa6c8158b80 2024-04-24T05:45:38.299 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:38.297+0000 7f203f7b8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T05:45:38.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.354+0000 7fa6cca00640 1 -- 172.21.15.155:0/2461419023 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6973cbac-812d-478e-aec9-add25fd81549", "id": 6} v 0) -- 0x7fa6c80738f0 con 0x7fa6c8158b80 2024-04-24T05:45:38.363 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.361+0000 7fa6bea00640 1 -- 172.21.15.155:0/2461419023 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6973cbac-812d-478e-aec9-add25fd81549", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fa6b40035f0 con 0x7fa6c8158b80 2024-04-24T05:45:38.363 INFO:teuthology.orchestra.run.smithi155.stdout:6 2024-04-24T05:45:38.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.363+0000 7fa6cca00640 1 -- 172.21.15.155:0/2461419023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6c8158b80 msgr2=0x7fa6c80ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.363+0000 7fa6cca00640 1 --2- 172.21.15.155:0/2461419023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6c8158b80 0x7fa6c80ccc20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400c3f0 tx=0x7fa6b400c8b0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.363+0000 7fa6cca00640 1 -- 172.21.15.155:0/2461419023 shutdown_connections 2024-04-24T05:45:38.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.363+0000 7fa6cca00640 1 --2- 172.21.15.155:0/2461419023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6c815f110 0x7fa6c81f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.363+0000 7fa6cca00640 1 --2- 172.21.15.155:0/2461419023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6c8158b80 0x7fa6c80ccc20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.363+0000 7fa6cca00640 1 --2- 172.21.15.155:0/2461419023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6c8073400 0x7fa6c815d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.363+0000 7fa6cca00640 1 -- 172.21.15.155:0/2461419023 >> 172.21.15.155:0/2461419023 conn(0x7fa6c80d1280 msgr2=0x7fa6c8161380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:38.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.363+0000 7fa6cca00640 1 -- 172.21.15.155:0/2461419023 shutdown_connections 2024-04-24T05:45:38.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.363+0000 7fa6cca00640 1 -- 172.21.15.155:0/2461419023 wait complete. 2024-04-24T05:45:38.366 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:38.364+0000 7f3669693040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T05:45:38.380 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 95ff107b-7dfa-449b-a448-6d3573feb2c9 7 2024-04-24T05:45:38.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.552+0000 7f3017e00640 1 Processor -- start 2024-04-24T05:45:38.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.553+0000 7f3017e00640 1 -- start start 2024-04-24T05:45:38.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.554+0000 7f3017e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30180ce310 0x7f30180d0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.554+0000 7f3017e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f30180b4340 0x7f30180d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.554+0000 7f3017e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30180cd1f0 con 0x7f30180b3f60 2024-04-24T05:45:38.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.554+0000 7f3017e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f301815d030 con 0x7f30180b4340 2024-04-24T05:45:38.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.554+0000 7f3017e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f301815d310 con 0x7f30180ce310 2024-04-24T05:45:38.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.554+0000 7f300e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30180ce310 0x7f30180d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.554+0000 7f3016a00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f30180b3f60 0x7f30180ccae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:51022/0 (socket says 172.21.15.155:51022) 2024-04-24T05:45:38.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.554+0000 7f3016a00640 1 -- 172.21.15.155:0/3642519142 learned_addr learned my addr 172.21.15.155:0/3642519142 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-04-24T05:45:38.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.555+0000 7f300e000640 1 -- 172.21.15.155:0/3642519142 >> v2:172.21.15.123:3300/0 conn(0x7f30180b4340 msgr2=0x7f30180d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:38.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.555+0000 7f3017400640 1 --2- 172.21.15.155:0/3642519142 >> v2:172.21.15.123:3300/0 conn(0x7f30180b4340 0x7f30180d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.555+0000 7f3016000640 1 -- 172.21.15.155:0/3642519142 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3033393377 0 0) 0x7f30180cd1f0 con 0x7f30180b3f60 2024-04-24T05:45:38.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.555+0000 7f300e000640 1 --2- 172.21.15.155:0/3642519142 >> v2:172.21.15.123:3300/0 conn(0x7f30180b4340 0x7f30180d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.555+0000 7f300e000640 1 -- 172.21.15.155:0/3642519142 >> v1:172.21.15.155:6789/0 conn(0x7f30180b3f60 legacy=0x7f30180ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.555+0000 7f300e000640 1 -- 172.21.15.155:0/3642519142 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301815d5f0 con 0x7f30180ce310 2024-04-24T05:45:38.558 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.555+0000 7f3017400640 1 --2- 172.21.15.155:0/3642519142 >> v2:172.21.15.123:3300/0 conn(0x7f30180b4340 0x7f30180d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:45:38.559 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.556+0000 7f300e000640 1 --2- 172.21.15.155:0/3642519142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30180ce310 0x7f30180d0730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f30080097a0 tx=0x7f300802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e3cfd86d35eb4e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:38.559 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.556+0000 7f3016000640 1 -- 172.21.15.155:0/3642519142 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008037070 con 0x7f30180ce310 2024-04-24T05:45:38.559 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.556+0000 7f3016000640 1 -- 172.21.15.155:0/3642519142 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f300802ae50 con 0x7f30180ce310 2024-04-24T05:45:38.559 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.556+0000 7f3016000640 1 -- 172.21.15.155:0/3642519142 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008032660 con 0x7f30180ce310 2024-04-24T05:45:38.559 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.557+0000 7f3017e00640 1 -- 172.21.15.155:0/3642519142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30180ce310 msgr2=0x7f30180d0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.559 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.557+0000 7f3017e00640 1 --2- 172.21.15.155:0/3642519142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30180ce310 0x7f30180d0730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f30080097a0 tx=0x7f300802a100 comp rx=0 tx=0).stop 2024-04-24T05:45:38.559 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.557+0000 7f3017e00640 1 -- 172.21.15.155:0/3642519142 shutdown_connections 2024-04-24T05:45:38.560 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.557+0000 7f3017e00640 1 --2- 172.21.15.155:0/3642519142 >> v2:172.21.15.123:3300/0 conn(0x7f30180b4340 0x7f30180d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.560 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.557+0000 7f3017e00640 1 --2- 172.21.15.155:0/3642519142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30180ce310 0x7f30180d0730 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.560 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.557+0000 7f3017e00640 1 -- 172.21.15.155:0/3642519142 >> 172.21.15.155:0/3642519142 conn(0x7f30180cecf0 msgr2=0x7f30180cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:38.560 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.557+0000 7f3017e00640 1 -- 172.21.15.155:0/3642519142 shutdown_connections 2024-04-24T05:45:38.560 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.557+0000 7f3017e00640 1 -- 172.21.15.155:0/3642519142 wait complete. 2024-04-24T05:45:38.560 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.559+0000 7f3017e00640 1 Processor -- start 2024-04-24T05:45:38.560 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.559+0000 7f3017e00640 1 -- start start 2024-04-24T05:45:38.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3017e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30180b3f60 0x7f30180b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3017e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30180b4340 0x7f301815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3017e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30180ce310 0x7f30181506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3017e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30180d1aa0 con 0x7f30180b3f60 2024-04-24T05:45:38.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3017e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f301815d8d0 con 0x7f30180b4340 2024-04-24T05:45:38.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3017e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f30180d1c20 con 0x7f30180ce310 2024-04-24T05:45:38.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3017400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30180ce310 0x7f30181506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f300e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30180b4340 0x7f301815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f300e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30180b4340 0x7f301815e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:60644/0 (socket says 172.21.15.155:60644) 2024-04-24T05:45:38.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f300e000640 1 -- 172.21.15.155:0/1724001673 learned_addr learned my addr 172.21.15.155:0/1724001673 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-04-24T05:45:38.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3017400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30180ce310 0x7f30181506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:42964/0 (socket says 172.21.15.155:42964) 2024-04-24T05:45:38.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3016a00640 1 --2- 172.21.15.155:0/1724001673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30180b3f60 0x7f30180b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3017400640 1 -- 172.21.15.155:0/1724001673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30180b4340 msgr2=0x7f301815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3017400640 1 --2- 172.21.15.155:0/1724001673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30180b4340 0x7f301815e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3017400640 1 -- 172.21.15.155:0/1724001673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30180b3f60 msgr2=0x7f30180b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3017400640 1 --2- 172.21.15.155:0/1724001673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30180b3f60 0x7f30180b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3017400640 1 -- 172.21.15.155:0/1724001673 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301815ecf0 con 0x7f30180ce310 2024-04-24T05:45:38.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f3016a00640 1 --2- 172.21.15.155:0/1724001673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30180b3f60 0x7f30180b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:45:38.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.560+0000 7f300e000640 1 --2- 172.21.15.155:0/1724001673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30180b4340 0x7f301815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:45:38.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.561+0000 7f3017400640 1 --2- 172.21.15.155:0/1724001673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30180ce310 0x7f30181506f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f301000cdf0 tx=0x7f301000a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:38.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.561+0000 7f300f400640 1 -- 172.21.15.155:0/1724001673 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301001a070 con 0x7f30180ce310 2024-04-24T05:45:38.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.561+0000 7f300f400640 1 -- 172.21.15.155:0/1724001673 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301000d840 con 0x7f30180ce310 2024-04-24T05:45:38.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.561+0000 7f300f400640 1 -- 172.21.15.155:0/1724001673 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301000dc70 con 0x7f30180ce310 2024-04-24T05:45:38.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.561+0000 7f3017e00640 1 -- 172.21.15.155:0/1724001673 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30180b7840 con 0x7f30180ce310 2024-04-24T05:45:38.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.561+0000 7f3017e00640 1 -- 172.21.15.155:0/1724001673 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f30181e6040 con 0x7f30180ce310 2024-04-24T05:45:38.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.561+0000 7f300f400640 1 -- 172.21.15.155:0/1724001673 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f301001d070 con 0x7f30180ce310 2024-04-24T05:45:38.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.561+0000 7f300f400640 1 -- 172.21.15.155:0/1724001673 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f3010015aa0 con 0x7f30180ce310 2024-04-24T05:45:38.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.562+0000 7f3017e00640 1 -- 172.21.15.155:0/1724001673 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30180b7d60 con 0x7f30180ce310 2024-04-24T05:45:38.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.565+0000 7f300f400640 1 -- 172.21.15.155:0/1724001673 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3010024020 con 0x7f30180ce310 2024-04-24T05:45:38.593 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:38.592+0000 7f203f7b8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T05:45:38.645 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.644+0000 7f3017e00640 1 -- 172.21.15.155:0/1724001673 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "95ff107b-7dfa-449b-a448-6d3573feb2c9", "id": 7} v 0) -- 0x7f3018069130 con 0x7f30180ce310 2024-04-24T05:45:38.653 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.651+0000 7f300f400640 1 -- 172.21.15.155:0/1724001673 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "95ff107b-7dfa-449b-a448-6d3573feb2c9", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f3010015440 con 0x7f30180ce310 2024-04-24T05:45:38.653 INFO:teuthology.orchestra.run.smithi155.stdout:7 2024-04-24T05:45:38.653 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:38.652+0000 7f203f7b8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T05:45:38.654 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.653+0000 7f3017e00640 1 -- 172.21.15.155:0/1724001673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30180ce310 msgr2=0x7f30181506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.653+0000 7f3017e00640 1 --2- 172.21.15.155:0/1724001673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30180ce310 0x7f30181506f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f301000cdf0 tx=0x7f301000a710 comp rx=0 tx=0).stop 2024-04-24T05:45:38.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.653+0000 7f3017e00640 1 -- 172.21.15.155:0/1724001673 shutdown_connections 2024-04-24T05:45:38.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.653+0000 7f3017e00640 1 --2- 172.21.15.155:0/1724001673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30180ce310 0x7f30181506f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.653+0000 7f3017e00640 1 --2- 172.21.15.155:0/1724001673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30180b4340 0x7f301815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.653+0000 7f3017e00640 1 --2- 172.21.15.155:0/1724001673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30180b3f60 0x7f30180b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.653+0000 7f3017e00640 1 -- 172.21.15.155:0/1724001673 >> 172.21.15.155:0/1724001673 conn(0x7f30180cecf0 msgr2=0x7f301806f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:38.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.653+0000 7f3017e00640 1 -- 172.21.15.155:0/1724001673 shutdown_connections 2024-04-24T05:45:38.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:45:38.653+0000 7f3017e00640 1 -- 172.21.15.155:0/1724001673 wait complete. 2024-04-24T05:45:38.671 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-24T05:45:38.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-24T05:45:38.677 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:38.676+0000 7f3669693040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T05:45:38.678 INFO:tasks.ceph.osd.0:Started 2024-04-24T05:45:38.678 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-24T05:45:38.678 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-24T05:45:38.688 INFO:tasks.ceph.osd.1:Started 2024-04-24T05:45:38.688 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-24T05:45:38.688 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-24T05:45:38.693 INFO:tasks.ceph.osd.2:Started 2024-04-24T05:45:38.693 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-24T05:45:38.693 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-24T05:45:38.699 INFO:tasks.ceph.osd.3:Started 2024-04-24T05:45:38.699 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-24T05:45:38.700 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-24T05:45:38.707 INFO:tasks.ceph.osd.4:Started 2024-04-24T05:45:38.707 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-24T05:45:38.707 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-24T05:45:38.714 INFO:tasks.ceph.osd.5:Started 2024-04-24T05:45:38.714 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-24T05:45:38.714 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-24T05:45:38.720 INFO:tasks.ceph.osd.6:Started 2024-04-24T05:45:38.720 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-24T05:45:38.720 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-24T05:45:38.727 INFO:tasks.ceph.osd.7:Started 2024-04-24T05:45:38.727 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:45:38.744 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:38.743+0000 7f3669693040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T05:45:38.816 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:38.815+0000 7f203f7b8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T05:45:38.921 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:38.919+0000 7f3669693040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T05:45:38.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.963+0000 7f07af000640 1 Processor -- start 2024-04-24T05:45:38.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.963+0000 7f07af000640 1 -- start start 2024-04-24T05:45:38.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.964+0000 7f07af000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f07a80736d0 0x7f07a8151e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.964+0000 7f07af000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07a806f470 0x7f07a8152350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.964+0000 7f07af000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07a8071cd0 con 0x7f07a806f090 2024-04-24T05:45:38.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.964+0000 7f07af000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f07a8071fb0 con 0x7f07a80736d0 2024-04-24T05:45:38.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.964+0000 7f07af000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f07a8072290 con 0x7f07a806f470 2024-04-24T05:45:38.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.965+0000 7f07a7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f07a806f090 0x7f07a8072fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:51582/0 (socket says 172.21.15.123:51582) 2024-04-24T05:45:38.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.965+0000 7f07a7e00640 1 -- 172.21.15.123:0/2302514298 learned_addr learned my addr 172.21.15.123:0/2302514298 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:45:38.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.965+0000 7f079fe00640 1 --2- 172.21.15.123:0/2302514298 >> v2:172.21.15.155:3300/0 conn(0x7f07a80736d0 0x7f07a8151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.965+0000 7f07ad000640 1 --2- 172.21.15.123:0/2302514298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07a806f470 0x7f07a8152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.965+0000 7f07a7400640 1 -- 172.21.15.123:0/2302514298 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 843045313 0 0) 0x7f07a8071cd0 con 0x7f07a806f090 2024-04-24T05:45:38.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.965+0000 7f07a7400640 1 -- 172.21.15.123:0/2302514298 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f078c003520 con 0x7f07a806f090 2024-04-24T05:45:38.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.965+0000 7f079fe00640 1 -- 172.21.15.123:0/2302514298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07a806f470 msgr2=0x7f07a8152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:38.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.965+0000 7f079fe00640 1 --2- 172.21.15.123:0/2302514298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07a806f470 0x7f07a8152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.965+0000 7f079fe00640 1 -- 172.21.15.123:0/2302514298 >> v1:172.21.15.123:6789/0 conn(0x7f07a806f090 legacy=0x7f07a8072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.965+0000 7f079fe00640 1 -- 172.21.15.123:0/2302514298 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a8072570 con 0x7f07a80736d0 2024-04-24T05:45:38.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.967+0000 7f079fe00640 1 --2- 172.21.15.123:0/2302514298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a80736d0 0x7f07a8151e10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f07900097a0 tx=0x7f079002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=511d591177135c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:38.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.967+0000 7f07a7400640 1 -- 172.21.15.123:0/2302514298 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790037070 con 0x7f07a80736d0 2024-04-24T05:45:38.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.967+0000 7f07a7400640 1 -- 172.21.15.123:0/2302514298 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079002ae50 con 0x7f07a80736d0 2024-04-24T05:45:38.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.967+0000 7f07a7400640 1 -- 172.21.15.123:0/2302514298 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790032660 con 0x7f07a80736d0 2024-04-24T05:45:38.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.967+0000 7f07af000640 1 -- 172.21.15.123:0/2302514298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a80736d0 msgr2=0x7f07a8151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.967+0000 7f07af000640 1 --2- 172.21.15.123:0/2302514298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a80736d0 0x7f07a8151e10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f07900097a0 tx=0x7f079002a100 comp rx=0 tx=0).stop 2024-04-24T05:45:38.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.968+0000 7f07af000640 1 -- 172.21.15.123:0/2302514298 shutdown_connections 2024-04-24T05:45:38.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.968+0000 7f07af000640 1 --2- 172.21.15.123:0/2302514298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07a806f470 0x7f07a8152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.968+0000 7f07af000640 1 --2- 172.21.15.123:0/2302514298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a80736d0 0x7f07a8151e10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.968+0000 7f07af000640 1 -- 172.21.15.123:0/2302514298 >> 172.21.15.123:0/2302514298 conn(0x7f07a80b1e70 msgr2=0x7f07a80b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:38.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.968+0000 7f07af000640 1 -- 172.21.15.123:0/2302514298 shutdown_connections 2024-04-24T05:45:38.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.968+0000 7f07af000640 1 -- 172.21.15.123:0/2302514298 wait complete. 2024-04-24T05:45:38.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.969+0000 7f07af000640 1 Processor -- start 2024-04-24T05:45:38.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.969+0000 7f07af000640 1 -- start start 2024-04-24T05:45:38.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.970+0000 7f07af000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07a806f090 0x7f07a813b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.970+0000 7f07af000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a806f470 0x7f07a80ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.970+0000 7f07af000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07a80736d0 0x7f07a80cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:38.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.970+0000 7f07af000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07a815c510 con 0x7f07a806f090 2024-04-24T05:45:38.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.970+0000 7f07af000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f07a815c8d0 con 0x7f07a806f470 2024-04-24T05:45:38.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.970+0000 7f07af000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f07a815c690 con 0x7f07a80736d0 2024-04-24T05:45:38.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.970+0000 7f07a7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07a806f090 0x7f07a813b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.971+0000 7f07a7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07a806f090 0x7f07a813b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47074/0 (socket says 172.21.15.123:47074) 2024-04-24T05:45:38.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.971+0000 7f07a7e00640 1 -- 172.21.15.123:0/3050470985 learned_addr learned my addr 172.21.15.123:0/3050470985 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:38.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.971+0000 7f079fe00640 1 --2- 172.21.15.123:0/3050470985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a806f470 0x7f07a80ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.971+0000 7f07ad000640 1 --2- 172.21.15.123:0/3050470985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07a80736d0 0x7f07a80cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:38.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.971+0000 7f07a7e00640 1 -- 172.21.15.123:0/3050470985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07a80736d0 msgr2=0x7f07a80cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.971+0000 7f07a7e00640 1 --2- 172.21.15.123:0/3050470985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07a80736d0 0x7f07a80cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.971+0000 7f07a7e00640 1 -- 172.21.15.123:0/3050470985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a806f470 msgr2=0x7f07a80ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:38.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.971+0000 7f07a7e00640 1 --2- 172.21.15.123:0/3050470985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a806f470 0x7f07a80ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:38.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.971+0000 7f07a7e00640 1 -- 172.21.15.123:0/3050470985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a80cd5c0 con 0x7f07a806f090 2024-04-24T05:45:38.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.971+0000 7f07ad000640 1 --2- 172.21.15.123:0/3050470985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07a80736d0 0x7f07a80cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:45:38.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.971+0000 7f07a7e00640 1 --2- 172.21.15.123:0/3050470985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07a806f090 0x7f07a813b520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f079800c390 tx=0x7f079800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:38.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.971+0000 7f07a4c00640 1 -- 172.21.15.123:0/3050470985 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0798017070 con 0x7f07a806f090 2024-04-24T05:45:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.971+0000 7f07a4c00640 1 -- 172.21.15.123:0/3050470985 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0798005ce0 con 0x7f07a806f090 2024-04-24T05:45:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.972+0000 7f07a4c00640 1 -- 172.21.15.123:0/3050470985 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07980072f0 con 0x7f07a806f090 2024-04-24T05:45:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.972+0000 7f07af000640 1 -- 172.21.15.123:0/3050470985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a80cd8a0 con 0x7f07a806f090 2024-04-24T05:45:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.972+0000 7f07af000640 1 -- 172.21.15.123:0/3050470985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a80cde60 con 0x7f07a806f090 2024-04-24T05:45:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.972+0000 7f07a4c00640 1 -- 172.21.15.123:0/3050470985 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f079801a070 con 0x7f07a806f090 2024-04-24T05:45:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.972+0000 7f07a4c00640 1 -- 172.21.15.123:0/3050470985 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f0798007490 con 0x7f07a806f090 2024-04-24T05:45:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.973+0000 7f07af000640 1 -- 172.21.15.123:0/3050470985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a8068a20 con 0x7f07a806f090 2024-04-24T05:45:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:38.975+0000 7f07a4c00640 1 -- 172.21.15.123:0/3050470985 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0798021020 con 0x7f07a806f090 2024-04-24T05:45:39.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:39.041+0000 7f07af000640 1 -- 172.21.15.123:0/3050470985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07a80ad870 con 0x7f07a806f090 2024-04-24T05:45:39.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:39.041+0000 7f07a4c00640 1 -- 172.21.15.123:0/3050470985 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f079801c440 con 0x7f07a806f090 2024-04-24T05:45:39.043 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:45:39.043 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":10,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:45:38.647182+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:45:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:39.042+0000 7f07af000640 1 -- 172.21.15.123:0/3050470985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07a806f090 msgr2=0x7f07a813b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:39.042+0000 7f07af000640 1 --2- 172.21.15.123:0/3050470985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07a806f090 0x7f07a813b520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f079800c390 tx=0x7f079800c850 comp rx=0 tx=0).stop 2024-04-24T05:45:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:39.042+0000 7f07af000640 1 -- 172.21.15.123:0/3050470985 shutdown_connections 2024-04-24T05:45:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:39.043+0000 7f07af000640 1 --2- 172.21.15.123:0/3050470985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07a80736d0 0x7f07a80cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:39.043+0000 7f07af000640 1 --2- 172.21.15.123:0/3050470985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a806f470 0x7f07a80ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:39.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:39.043+0000 7f07af000640 1 --2- 172.21.15.123:0/3050470985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07a806f090 0x7f07a813b520 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:39.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:39.043+0000 7f07af000640 1 -- 172.21.15.123:0/3050470985 >> 172.21.15.123:0/3050470985 conn(0x7f07a80b1e70 msgr2=0x7f07a806e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:39.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:39.043+0000 7f07af000640 1 -- 172.21.15.123:0/3050470985 shutdown_connections 2024-04-24T05:45:39.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:39.043+0000 7f07af000640 1 -- 172.21.15.123:0/3050470985 wait complete. 2024-04-24T05:45:39.061 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-24T05:45:39.061 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-24T05:45:39.201 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:39.200+0000 7f203f7b8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T05:45:39.253 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:39.251+0000 7f3669693040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T05:45:39.256 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:39.255+0000 7f203f7b8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T05:45:39.308 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:39.306+0000 7f3669693040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T05:45:39.374 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T05:45:39.372+0000 7ff33e611640 -1 Falling back to public interface 2024-04-24T05:45:39.383 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T05:45:39.382+0000 7f3e619a7640 -1 Falling back to public interface 2024-04-24T05:45:39.388 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T05:45:39.387+0000 7fcdbe616640 -1 Falling back to public interface 2024-04-24T05:45:39.388 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T05:45:39.387+0000 7fed6140c640 -1 Falling back to public interface 2024-04-24T05:45:39.395 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T05:45:39.393+0000 7f80ec5c2640 -1 Falling back to public interface 2024-04-24T05:45:39.413 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T05:45:39.412+0000 7f3891432640 -1 Falling back to public interface 2024-04-24T05:45:39.421 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T05:45:39.420+0000 7ff1d7a15640 -1 Falling back to public interface 2024-04-24T05:45:39.430 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T05:45:39.428+0000 7f92d011a640 -1 Falling back to public interface 2024-04-24T05:45:39.444 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:39.443+0000 7f203f7b8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T05:45:39.495 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:39.493+0000 7f3669693040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T05:45:39.608 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:39.606+0000 7f203f7b8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T05:45:39.660 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:39.659+0000 7f3669693040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T05:45:39.667 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:39.665+0000 7f203f7b8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T05:45:39.720 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:39.719+0000 7f3669693040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T05:45:39.750 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:39.748+0000 7f203f7b8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T05:45:39.802 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:39.800+0000 7f3669693040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T05:45:39.802 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:39.800+0000 7f203f7b8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T05:45:39.855 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:39.853+0000 7f3669693040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T05:45:39.867 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T05:45:40.028 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.026+0000 7efe36e00640 1 Processor -- start 2024-04-24T05:45:40.028 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.026+0000 7efe36e00640 1 -- start start 2024-04-24T05:45:40.030 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.027+0000 7efe36e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efe30161d50 0x7efe300d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:40.030 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.027+0000 7efe36e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe3006be00 0x7efe300cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:40.030 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.027+0000 7efe36e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe3006c680 con 0x7efe300d00f0 2024-04-24T05:45:40.030 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.027+0000 7efe36e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7efe30162130 con 0x7efe30161d50 2024-04-24T05:45:40.030 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.027+0000 7efe36e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efe300d0520 con 0x7efe3006be00 2024-04-24T05:45:40.030 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.028+0000 7efe36400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7efe300d00f0 0x7efe300d2090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:51588/0 (socket says 172.21.15.123:51588) 2024-04-24T05:45:40.030 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.028+0000 7efe36400640 1 -- 172.21.15.123:0/2885093196 learned_addr learned my addr 172.21.15.123:0/2885093196 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:45:40.030 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.028+0000 7efe35a00640 1 --2- 172.21.15.123:0/2885093196 >> v2:172.21.15.155:3300/0 conn(0x7efe30161d50 0x7efe300d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:40.030 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.028+0000 7efe35000640 1 --2- 172.21.15.123:0/2885093196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe3006be00 0x7efe300cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:40.030 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.028+0000 7efe2fe00640 1 -- 172.21.15.123:0/2885093196 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1757728846 0 0) 0x7efe28001490 con 0x7efe300d00f0 2024-04-24T05:45:40.030 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.028+0000 7efe2fe00640 1 -- 172.21.15.123:0/2885093196 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe18003520 con 0x7efe300d00f0 2024-04-24T05:45:40.031 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.028+0000 7efe35a00640 1 -- 172.21.15.123:0/2885093196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe3006be00 msgr2=0x7efe300cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:40.031 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.028+0000 7efe35a00640 1 --2- 172.21.15.123:0/2885093196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe3006be00 0x7efe300cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:40.031 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.028+0000 7efe35a00640 1 -- 172.21.15.123:0/2885093196 >> v1:172.21.15.123:6789/0 conn(0x7efe300d00f0 legacy=0x7efe300d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:40.031 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.028+0000 7efe35a00640 1 -- 172.21.15.123:0/2885093196 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe300d3e20 con 0x7efe30161d50 2024-04-24T05:45:40.031 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.028+0000 7efe35000640 1 --2- 172.21.15.123:0/2885093196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe3006be00 0x7efe300cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:45:40.032 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.030+0000 7efe35a00640 1 --2- 172.21.15.123:0/2885093196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe30161d50 0x7efe300d0940 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efe240097d0 tx=0x7efe2402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c089e83e39d77c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:40.032 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.030+0000 7efe2fe00640 1 -- 172.21.15.123:0/2885093196 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24037070 con 0x7efe30161d50 2024-04-24T05:45:40.032 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.030+0000 7efe2fe00640 1 -- 172.21.15.123:0/2885093196 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2402ae40 con 0x7efe30161d50 2024-04-24T05:45:40.032 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.030+0000 7efe2fe00640 1 -- 172.21.15.123:0/2885093196 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24032640 con 0x7efe30161d50 2024-04-24T05:45:40.032 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.030+0000 7efe36e00640 1 -- 172.21.15.123:0/2885093196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe30161d50 msgr2=0x7efe300d0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:40.032 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.030+0000 7efe36e00640 1 --2- 172.21.15.123:0/2885093196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe30161d50 0x7efe300d0940 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efe240097d0 tx=0x7efe2402a130 comp rx=0 tx=0).stop 2024-04-24T05:45:40.032 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.031+0000 7efe36e00640 1 -- 172.21.15.123:0/2885093196 shutdown_connections 2024-04-24T05:45:40.032 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.031+0000 7efe36e00640 1 --2- 172.21.15.123:0/2885093196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe3006be00 0x7efe300cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:40.033 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.031+0000 7efe36e00640 1 --2- 172.21.15.123:0/2885093196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe30161d50 0x7efe300d0940 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:40.033 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.031+0000 7efe36e00640 1 -- 172.21.15.123:0/2885093196 >> 172.21.15.123:0/2885093196 conn(0x7efe300b6040 msgr2=0x7efe300b6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:40.033 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.031+0000 7efe36e00640 1 -- 172.21.15.123:0/2885093196 shutdown_connections 2024-04-24T05:45:40.033 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.031+0000 7efe36e00640 1 -- 172.21.15.123:0/2885093196 wait complete. 2024-04-24T05:45:40.033 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.032+0000 7efe36e00640 1 Processor -- start 2024-04-24T05:45:40.034 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.032+0000 7efe36e00640 1 -- start start 2024-04-24T05:45:40.036 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.033+0000 7efe36e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe3006be00 0x7efe301f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:40.036 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.033+0000 7efe36e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe300d00f0 0x7efe300b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:40.036 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.033+0000 7efe36e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe30161d50 0x7efe300b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:40.036 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.033+0000 7efe36e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe300b7b80 con 0x7efe30161d50 2024-04-24T05:45:40.036 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.033+0000 7efe36e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe300b78a0 con 0x7efe300d00f0 2024-04-24T05:45:40.036 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.033+0000 7efe36e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efe300b7e60 con 0x7efe3006be00 2024-04-24T05:45:40.036 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.033+0000 7efe36400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe30161d50 0x7efe300b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:40.036 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe36400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe30161d50 0x7efe300b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47090/0 (socket says 172.21.15.123:47090) 2024-04-24T05:45:40.036 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe36400640 1 -- 172.21.15.123:0/3419219857 learned_addr learned my addr 172.21.15.123:0/3419219857 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:40.036 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe35000640 1 --2- 172.21.15.123:0/3419219857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe300d00f0 0x7efe300b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:40.037 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe35a00640 1 --2- 172.21.15.123:0/3419219857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe3006be00 0x7efe301f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:40.037 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe36400640 1 -- 172.21.15.123:0/3419219857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe3006be00 msgr2=0x7efe301f4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:40.037 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe36400640 1 --2- 172.21.15.123:0/3419219857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe3006be00 0x7efe301f4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:40.037 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe36400640 1 -- 172.21.15.123:0/3419219857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe300d00f0 msgr2=0x7efe300b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:40.037 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe36400640 1 --2- 172.21.15.123:0/3419219857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe300d00f0 0x7efe300b6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:40.037 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe36400640 1 -- 172.21.15.123:0/3419219857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe301493e0 con 0x7efe30161d50 2024-04-24T05:45:40.037 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe35000640 1 --2- 172.21.15.123:0/3419219857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe300d00f0 0x7efe300b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:45:40.037 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe36400640 1 --2- 172.21.15.123:0/3419219857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe30161d50 0x7efe300b69d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efe2800cb50 tx=0x7efe2800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:40.037 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe2d600640 1 -- 172.21.15.123:0/3419219857 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28017070 con 0x7efe30161d50 2024-04-24T05:45:40.038 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe2d600640 1 -- 172.21.15.123:0/3419219857 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2800ad40 con 0x7efe30161d50 2024-04-24T05:45:40.038 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe2d600640 1 -- 172.21.15.123:0/3419219857 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28012620 con 0x7efe30161d50 2024-04-24T05:45:40.038 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.034+0000 7efe36e00640 1 -- 172.21.15.123:0/3419219857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe301496a0 con 0x7efe30161d50 2024-04-24T05:45:40.038 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.035+0000 7efe36e00640 1 -- 172.21.15.123:0/3419219857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe301f4850 con 0x7efe30161d50 2024-04-24T05:45:40.038 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.035+0000 7efe2d600640 1 -- 172.21.15.123:0/3419219857 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efe2801a070 con 0x7efe30161d50 2024-04-24T05:45:40.039 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.035+0000 7efe2d600640 1 -- 172.21.15.123:0/3419219857 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7efe28007740 con 0x7efe30161d50 2024-04-24T05:45:40.039 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.036+0000 7efe36e00640 1 -- 172.21.15.123:0/3419219857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3004fe00 con 0x7efe30161d50 2024-04-24T05:45:40.039 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.038+0000 7efe2d600640 1 -- 172.21.15.123:0/3419219857 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7efe28021020 con 0x7efe30161d50 2024-04-24T05:45:40.106 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.105+0000 7efe36e00640 1 -- 172.21.15.123:0/3419219857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe300ce650 con 0x7efe30161d50 2024-04-24T05:45:40.107 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.106+0000 7efe2d600640 1 -- 172.21.15.123:0/3419219857 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7efe3006c680 con 0x7efe30161d50 2024-04-24T05:45:40.108 INFO:teuthology.misc.health.smithi123.stdout: 2024-04-24T05:45:40.108 INFO:teuthology.misc.health.smithi123.stdout:{"epoch":10,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:45:38.647182+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:45:40.110 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.107+0000 7efe36e00640 1 -- 172.21.15.123:0/3419219857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe30161d50 msgr2=0x7efe300b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:40.110 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.107+0000 7efe36e00640 1 --2- 172.21.15.123:0/3419219857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe30161d50 0x7efe300b69d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efe2800cb50 tx=0x7efe2800a430 comp rx=0 tx=0).stop 2024-04-24T05:45:40.110 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.107+0000 7efe36e00640 1 -- 172.21.15.123:0/3419219857 shutdown_connections 2024-04-24T05:45:40.110 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.107+0000 7efe36e00640 1 --2- 172.21.15.123:0/3419219857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe30161d50 0x7efe300b69d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:40.110 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.107+0000 7efe36e00640 1 --2- 172.21.15.123:0/3419219857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe300d00f0 0x7efe300b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:40.111 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.107+0000 7efe36e00640 1 --2- 172.21.15.123:0/3419219857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe3006be00 0x7efe301f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:40.111 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.107+0000 7efe36e00640 1 -- 172.21.15.123:0/3419219857 >> 172.21.15.123:0/3419219857 conn(0x7efe300b6040 msgr2=0x7efe301609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:40.111 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.108+0000 7efe36e00640 1 -- 172.21.15.123:0/3419219857 shutdown_connections 2024-04-24T05:45:40.111 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:40.108+0000 7efe36e00640 1 -- 172.21.15.123:0/3419219857 wait complete. 2024-04-24T05:45:40.121 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-24T05:45:40.353 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:40.351+0000 7f203f7b8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T05:45:40.402 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:40.400+0000 7f3669693040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T05:45:40.484 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:40.483+0000 7f203f7b8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T05:45:40.539 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-04-24T05:45:40.537+0000 7f3669693040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T05:45:41.524 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:41.523+0000 7f1fe0800640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-24T05:45:41.529 INFO:tasks.ceph.mgr.y.smithi123.stderr:2024-04-24T05:45:41.528+0000 7f1fe0800640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-24T05:45:42.723 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T05:45:42.722+0000 7f3e619a7640 -1 osd.0 0 log_to_monitors true 2024-04-24T05:45:42.803 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T05:45:42.802+0000 7f3891432640 -1 osd.3 0 log_to_monitors true 2024-04-24T05:45:42.843 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T05:45:42.842+0000 7ff1d7a15640 -1 osd.7 0 log_to_monitors true 2024-04-24T05:45:42.980 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T05:45:42.979+0000 7ff33e611640 -1 osd.2 0 log_to_monitors true 2024-04-24T05:45:42.990 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T05:45:42.989+0000 7fed6140c640 -1 osd.1 0 log_to_monitors true 2024-04-24T05:45:43.019 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T05:45:43.017+0000 7f80ec5c2640 -1 osd.6 0 log_to_monitors true 2024-04-24T05:45:43.032 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T05:45:43.030+0000 7fcdbe616640 -1 osd.4 0 log_to_monitors true 2024-04-24T05:45:43.075 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T05:45:43.073+0000 7f92d011a640 -1 osd.5 0 log_to_monitors true 2024-04-24T05:45:46.827 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T05:45:46.825+0000 7f92cba00640 -1 osd.5 0 waiting for initial osdmap 2024-04-24T05:45:46.844 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T05:45:46.843+0000 7f92c1000640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T05:45:46.864 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T05:45:46.864+0000 7f80e7400640 -1 osd.6 0 waiting for initial osdmap 2024-04-24T05:45:46.870 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T05:45:46.869+0000 7f80dd400640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T05:45:46.916 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T05:45:46.915+0000 7ff1d2c00640 -1 osd.7 0 waiting for initial osdmap 2024-04-24T05:45:46.923 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T05:45:46.923+0000 7ff1c8200640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T05:45:46.928 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T05:45:46.931 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T05:45:46.927+0000 7f3e5c800640 -1 osd.0 0 waiting for initial osdmap 2024-04-24T05:45:46.939 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T05:45:46.938+0000 7fcdb8e00640 -1 osd.4 0 waiting for initial osdmap 2024-04-24T05:45:46.945 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T05:45:46.943+0000 7f3e52800640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T05:45:46.947 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T05:45:46.945+0000 7f388c200640 -1 osd.3 0 waiting for initial osdmap 2024-04-24T05:45:46.950 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T05:45:46.950+0000 7fcdaee00640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T05:45:46.958 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T05:45:46.957+0000 7f3882200640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T05:45:46.991 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T05:45:46.989+0000 7ff339800640 -1 osd.2 0 waiting for initial osdmap 2024-04-24T05:45:46.997 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T05:45:46.996+0000 7fed5bc00640 -1 osd.1 0 waiting for initial osdmap 2024-04-24T05:45:47.001 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T05:45:47.000+0000 7ff32ee00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T05:45:47.010 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T05:45:47.009+0000 7fed51c00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T05:45:47.086 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.084+0000 7f5375e00640 1 Processor -- start 2024-04-24T05:45:47.086 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.084+0000 7f5375e00640 1 -- start start 2024-04-24T05:45:47.088 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.085+0000 7f5375e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f537006dc20 0x7f537006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:47.088 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.085+0000 7f5375e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5370072310 0x7f5370072760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:47.088 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.085+0000 7f5375e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53700af160 con 0x7f537015b5d0 2024-04-24T05:45:47.088 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.085+0000 7f5375e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f53700af430 con 0x7f537006dc20 2024-04-24T05:45:47.088 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.085+0000 7f5375e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f53700af710 con 0x7f5370072310 2024-04-24T05:45:47.088 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.086+0000 7f536e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f537006dc20 0x7f537006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:47.088 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.086+0000 7f536e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f537006dc20 0x7f537006e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47206/0 (socket says 172.21.15.123:47206) 2024-04-24T05:45:47.088 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.086+0000 7f536e000640 1 -- 172.21.15.123:0/3178923977 learned_addr learned my addr 172.21.15.123:0/3178923977 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:47.088 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.086+0000 7f536f400640 1 --2- 172.21.15.123:0/3178923977 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5370072310 0x7f5370072760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:47.088 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.086+0000 7f536e000640 1 -- 172.21.15.123:0/3178923977 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5370072310 msgr2=0x7f5370072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:47.089 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.086+0000 7f536e000640 1 --2- 172.21.15.123:0/3178923977 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5370072310 0x7f5370072760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:47.089 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.086+0000 7f536e000640 1 -- 172.21.15.123:0/3178923977 >> v1:172.21.15.155:6789/0 conn(0x7f537015b5d0 legacy=0x7f53700aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:47.089 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.086+0000 7f536e000640 1 -- 172.21.15.123:0/3178923977 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53700af9f0 con 0x7f537006dc20 2024-04-24T05:45:47.090 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.087+0000 7f536e000640 1 --2- 172.21.15.123:0/3178923977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537006dc20 0x7f537006e0b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f535800bb10 tx=0x7f535802c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5969fde815927f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:47.090 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.087+0000 7f536d600640 1 -- 172.21.15.123:0/3178923977 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535803a070 con 0x7f537006dc20 2024-04-24T05:45:47.090 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.087+0000 7f536d600640 1 -- 172.21.15.123:0/3178923977 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53580326f0 con 0x7f537006dc20 2024-04-24T05:45:47.090 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.087+0000 7f536d600640 1 -- 172.21.15.123:0/3178923977 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358032ac0 con 0x7f537006dc20 2024-04-24T05:45:47.090 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.088+0000 7f5375e00640 1 -- 172.21.15.123:0/3178923977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537006dc20 msgr2=0x7f537006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:47.091 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.088+0000 7f5375e00640 1 --2- 172.21.15.123:0/3178923977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537006dc20 0x7f537006e0b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f535800bb10 tx=0x7f535802c830 comp rx=0 tx=0).stop 2024-04-24T05:45:47.091 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.088+0000 7f5375e00640 1 -- 172.21.15.123:0/3178923977 shutdown_connections 2024-04-24T05:45:47.091 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.088+0000 7f5375e00640 1 --2- 172.21.15.123:0/3178923977 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5370072310 0x7f5370072760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:47.091 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.088+0000 7f5375e00640 1 --2- 172.21.15.123:0/3178923977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537006dc20 0x7f537006e0b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:47.091 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.088+0000 7f5375e00640 1 -- 172.21.15.123:0/3178923977 >> 172.21.15.123:0/3178923977 conn(0x7f53700b3270 msgr2=0x7f53700b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:47.091 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.088+0000 7f5375e00640 1 -- 172.21.15.123:0/3178923977 shutdown_connections 2024-04-24T05:45:47.091 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.088+0000 7f5375e00640 1 -- 172.21.15.123:0/3178923977 wait complete. 2024-04-24T05:45:47.091 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.090+0000 7f5375e00640 1 Processor -- start 2024-04-24T05:45:47.092 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.090+0000 7f5375e00640 1 -- start start 2024-04-24T05:45:47.094 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f5375e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f537006dc20 0x7f537015d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:47.094 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f5375e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5370072310 0x7f537015b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:47.094 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f5375e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537015b5d0 0x7f537014dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:47.094 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f5375e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f537006f8b0 con 0x7f537015b5d0 2024-04-24T05:45:47.094 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f5375e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f537006f730 con 0x7f537006dc20 2024-04-24T05:45:47.094 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f5375e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5370072ca0 con 0x7f5370072310 2024-04-24T05:45:47.094 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f536f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537015b5d0 0x7f537014dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:47.094 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f536f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537015b5d0 0x7f537014dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47218/0 (socket says 172.21.15.123:47218) 2024-04-24T05:45:47.094 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f536f400640 1 -- 172.21.15.123:0/2529923879 learned_addr learned my addr 172.21.15.123:0/2529923879 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:47.094 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f536ea00640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f537006dc20 0x7f537015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:47.095 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f536e000640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5370072310 0x7f537015b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:47.095 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f536f400640 1 -- 172.21.15.123:0/2529923879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5370072310 msgr2=0x7f537015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:47.095 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f536f400640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5370072310 0x7f537015b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:47.095 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f536f400640 1 -- 172.21.15.123:0/2529923879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f537006dc20 msgr2=0x7f537015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:47.095 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f536f400640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f537006dc20 0x7f537015d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:47.095 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f536f400640 1 -- 172.21.15.123:0/2529923879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537015c090 con 0x7f537015b5d0 2024-04-24T05:45:47.095 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.091+0000 7f536ea00640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f537006dc20 0x7f537015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:45:47.095 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.092+0000 7f536f400640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537015b5d0 0x7f537014dfc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5368007e70 tx=0x7f536800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:47.095 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.092+0000 7f5367400640 1 -- 172.21.15.123:0/2529923879 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368014420 con 0x7f537015b5d0 2024-04-24T05:45:47.095 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.092+0000 7f5367400640 1 -- 172.21.15.123:0/2529923879 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5368019070 con 0x7f537015b5d0 2024-04-24T05:45:47.095 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.092+0000 7f5375e00640 1 -- 172.21.15.123:0/2529923879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537015c370 con 0x7f537015b5d0 2024-04-24T05:45:47.095 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.092+0000 7f5367400640 1 -- 172.21.15.123:0/2529923879 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368014820 con 0x7f537015b5d0 2024-04-24T05:45:47.095 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.092+0000 7f5375e00640 1 -- 172.21.15.123:0/2529923879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537015c930 con 0x7f537015b5d0 2024-04-24T05:45:47.096 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.093+0000 7f5375e00640 1 -- 172.21.15.123:0/2529923879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537014f380 con 0x7f537015b5d0 2024-04-24T05:45:47.096 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.094+0000 7f5367400640 1 -- 172.21.15.123:0/2529923879 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f53680149c0 con 0x7f537015b5d0 2024-04-24T05:45:47.096 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.095+0000 7f5367400640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f534c06c860 0x7f534c06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:47.097 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.095+0000 7f5367400640 1 -- 172.21.15.123:0/2529923879 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f536808b620 con 0x7f537015b5d0 2024-04-24T05:45:47.097 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.095+0000 7f536ea00640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f534c06c860 0x7f534c06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:47.097 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.095+0000 7f536ea00640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f534c06c860 0x7f534c06ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f536000a830 tx=0x7f5360008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:47.101 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.100+0000 7f5367400640 1 -- 172.21.15.123:0/2529923879 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f537014f380 con 0x7f537015b5d0 2024-04-24T05:45:47.294 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.292+0000 7f5375e00640 1 -- 172.21.15.123:0/2529923879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5370069350 con 0x7f537015b5d0 2024-04-24T05:45:47.295 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.294+0000 7f5367400640 1 -- 172.21.15.123:0/2529923879 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f536805a820 con 0x7f537015b5d0 2024-04-24T05:45:47.295 INFO:teuthology.misc.health.smithi123.stdout: 2024-04-24T05:45:47.296 INFO:teuthology.misc.health.smithi123.stdout:{"epoch":12,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:45:44.530403+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:45:47.298 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.297+0000 7f5375e00640 1 -- 172.21.15.123:0/2529923879 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f534c06c860 msgr2=0x7f534c06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:47.299 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.297+0000 7f5375e00640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f534c06c860 0x7f534c06ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f536000a830 tx=0x7f5360008040 comp rx=0 tx=0).stop 2024-04-24T05:45:47.299 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.297+0000 7f5375e00640 1 -- 172.21.15.123:0/2529923879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537015b5d0 msgr2=0x7f537014dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:47.299 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.297+0000 7f5375e00640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537015b5d0 0x7f537014dfc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5368007e70 tx=0x7f536800e4b0 comp rx=0 tx=0).stop 2024-04-24T05:45:47.299 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.297+0000 7f5375e00640 1 -- 172.21.15.123:0/2529923879 shutdown_connections 2024-04-24T05:45:47.299 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.297+0000 7f5375e00640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f534c06c860 0x7f534c06ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:47.299 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.297+0000 7f5375e00640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f537015b5d0 0x7f537014dfc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:47.299 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.297+0000 7f5375e00640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5370072310 0x7f537015b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:47.299 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.297+0000 7f5375e00640 1 --2- 172.21.15.123:0/2529923879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f537006dc20 0x7f537015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:47.299 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.297+0000 7f5375e00640 1 -- 172.21.15.123:0/2529923879 >> 172.21.15.123:0/2529923879 conn(0x7f53700b3270 msgr2=0x7f537006e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:47.299 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.297+0000 7f5375e00640 1 -- 172.21.15.123:0/2529923879 shutdown_connections 2024-04-24T05:45:47.299 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:47.297+0000 7f5375e00640 1 -- 172.21.15.123:0/2529923879 wait complete. 2024-04-24T05:45:47.311 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-24T05:45:52.450 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:45:52.458 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:45:54.118 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T05:45:54.300 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.299+0000 7f3878e00640 1 Processor -- start 2024-04-24T05:45:54.301 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.299+0000 7f3878e00640 1 -- start start 2024-04-24T05:45:54.303 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3878e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38741459f0 0x7f38740cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:54.303 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3878e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3874145dd0 0x7f38740cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:54.303 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3878e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f38740cff10 0x7f38740d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:54.304 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3878e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3874071960 con 0x7f3874145dd0 2024-04-24T05:45:54.304 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3878e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3874071c40 con 0x7f38740cff10 2024-04-24T05:45:54.304 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3878e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f38740cf670 con 0x7f38741459f0 2024-04-24T05:45:54.304 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3872a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3874145dd0 0x7f38740cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:54.304 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3872a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3874145dd0 0x7f38740cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52588/0 (socket says 172.21.15.123:52588) 2024-04-24T05:45:54.304 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3872a00640 1 -- 172.21.15.123:0/4101664243 learned_addr learned my addr 172.21.15.123:0/4101664243 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:54.304 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3873400640 1 --2- 172.21.15.123:0/4101664243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38741459f0 0x7f38740cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:54.304 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3873e00640 1 --2- 172.21.15.123:0/4101664243 >> v2:172.21.15.155:3300/0 conn(0x7f38740cff10 0x7f38740d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:54.304 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3872a00640 1 -- 172.21.15.123:0/4101664243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38741459f0 msgr2=0x7f38740cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:54.304 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3872a00640 1 --2- 172.21.15.123:0/4101664243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38741459f0 0x7f38740cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.304 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3872a00640 1 -- 172.21.15.123:0/4101664243 >> v2:172.21.15.155:3300/0 conn(0x7f38740cff10 msgr2=0x7f38740d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:54.304 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3872a00640 1 --2- 172.21.15.123:0/4101664243 >> v2:172.21.15.155:3300/0 conn(0x7f38740cff10 0x7f38740d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.304 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3872a00640 1 -- 172.21.15.123:0/4101664243 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38740cf950 con 0x7f3874145dd0 2024-04-24T05:45:54.304 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.301+0000 7f3873400640 1 --2- 172.21.15.123:0/4101664243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38741459f0 0x7f38740cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:45:54.305 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.302+0000 7f3873e00640 1 --2- 172.21.15.123:0/4101664243 >> v2:172.21.15.155:3300/0 conn(0x7f38740cff10 0x7f38740d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:45:54.306 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.302+0000 7f3872a00640 1 --2- 172.21.15.123:0/4101664243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3874145dd0 0x7f38740cefd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f387406d510 tx=0x7f386002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5a1fcaf054cfa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:54.306 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.303+0000 7f3872000640 1 -- 172.21.15.123:0/4101664243 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860037070 con 0x7f3874145dd0 2024-04-24T05:45:54.306 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.303+0000 7f3872000640 1 -- 172.21.15.123:0/4101664243 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3860032440 con 0x7f3874145dd0 2024-04-24T05:45:54.306 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.303+0000 7f3872000640 1 -- 172.21.15.123:0/4101664243 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860032810 con 0x7f3874145dd0 2024-04-24T05:45:54.306 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.303+0000 7f3878e00640 1 -- 172.21.15.123:0/4101664243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3874145dd0 msgr2=0x7f38740cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:54.306 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.303+0000 7f3878e00640 1 --2- 172.21.15.123:0/4101664243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3874145dd0 0x7f38740cefd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f387406d510 tx=0x7f386002a1e0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.306 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.304+0000 7f3878e00640 1 -- 172.21.15.123:0/4101664243 shutdown_connections 2024-04-24T05:45:54.306 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.304+0000 7f3878e00640 1 --2- 172.21.15.123:0/4101664243 >> v2:172.21.15.155:3300/0 conn(0x7f38740cff10 0x7f38740d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.306 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.304+0000 7f3878e00640 1 --2- 172.21.15.123:0/4101664243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3874145dd0 0x7f38740cefd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.306 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.304+0000 7f3878e00640 1 --2- 172.21.15.123:0/4101664243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38741459f0 0x7f38740cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.307 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.304+0000 7f3878e00640 1 -- 172.21.15.123:0/4101664243 >> 172.21.15.123:0/4101664243 conn(0x7f387406da30 msgr2=0x7f387406de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:54.307 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.304+0000 7f3878e00640 1 -- 172.21.15.123:0/4101664243 shutdown_connections 2024-04-24T05:45:54.307 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.304+0000 7f3878e00640 1 -- 172.21.15.123:0/4101664243 wait complete. 2024-04-24T05:45:54.307 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.305+0000 7f3878e00640 1 Processor -- start 2024-04-24T05:45:54.307 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.306+0000 7f3878e00640 1 -- start start 2024-04-24T05:45:54.308 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.306+0000 7f3878e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38740cff10 0x7f387413a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:54.308 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.306+0000 7f3878e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38741459f0 0x7f387415b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:54.309 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.306+0000 7f3878e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f387415efe0 0x7f38741442a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:54.309 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.306+0000 7f3878e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f38741600e0 con 0x7f38741459f0 2024-04-24T05:45:54.309 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.306+0000 7f3878e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f38740d08a0 con 0x7f387415efe0 2024-04-24T05:45:54.309 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.306+0000 7f3878e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f387406ba20 con 0x7f38740cff10 2024-04-24T05:45:54.309 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.307+0000 7f3872a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38741459f0 0x7f387415b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:54.309 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.307+0000 7f3872a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38741459f0 0x7f387415b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52598/0 (socket says 172.21.15.123:52598) 2024-04-24T05:45:54.309 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.307+0000 7f3872a00640 1 -- 172.21.15.123:0/3818605828 learned_addr learned my addr 172.21.15.123:0/3818605828 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:54.309 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.307+0000 7f3872a00640 1 -- 172.21.15.123:0/3818605828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38740cff10 msgr2=0x7f387413a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:54.309 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.307+0000 7f3872a00640 1 --2- 172.21.15.123:0/3818605828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38740cff10 0x7f387413a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.309 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.307+0000 7f3872a00640 1 -- 172.21.15.123:0/3818605828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f387415efe0 msgr2=0x7f38741442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:54.309 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.307+0000 7f3872a00640 1 --2- 172.21.15.123:0/3818605828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f387415efe0 0x7f38741442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.309 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.307+0000 7f3872a00640 1 -- 172.21.15.123:0/3818605828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387415f8c0 con 0x7f38741459f0 2024-04-24T05:45:54.310 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.308+0000 7f3872a00640 1 --2- 172.21.15.123:0/3818605828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38741459f0 0x7f387415b8b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f386002a220 tx=0x7f38600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:54.310 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.308+0000 7f386b400640 1 -- 172.21.15.123:0/3818605828 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860037040 con 0x7f38741459f0 2024-04-24T05:45:54.310 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.308+0000 7f386b400640 1 -- 172.21.15.123:0/3818605828 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3860042070 con 0x7f38741459f0 2024-04-24T05:45:54.310 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.308+0000 7f386b400640 1 -- 172.21.15.123:0/3818605828 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860030bd0 con 0x7f38741459f0 2024-04-24T05:45:54.310 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.308+0000 7f3878e00640 1 -- 172.21.15.123:0/3818605828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38740b3420 con 0x7f38741459f0 2024-04-24T05:45:54.311 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.308+0000 7f3878e00640 1 -- 172.21.15.123:0/3818605828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387415bdf0 con 0x7f38741459f0 2024-04-24T05:45:54.311 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.309+0000 7f3878e00640 1 -- 172.21.15.123:0/3818605828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387404fe00 con 0x7f38741459f0 2024-04-24T05:45:54.311 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.310+0000 7f386b400640 1 -- 172.21.15.123:0/3818605828 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3860047460 con 0x7f38741459f0 2024-04-24T05:45:54.313 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.311+0000 7f386b400640 1 --2- 172.21.15.123:0/3818605828 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f385406c8f0 0x7f385406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:54.313 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.311+0000 7f386b400640 1 -- 172.21.15.123:0/3818605828 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f38600ac430 con 0x7f38741459f0 2024-04-24T05:45:54.313 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.311+0000 7f3873400640 1 --2- 172.21.15.123:0/3818605828 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f385406c8f0 0x7f385406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:54.313 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.312+0000 7f3873400640 1 --2- 172.21.15.123:0/3818605828 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f385406c8f0 0x7f385406edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f38741f7d20 tx=0x7f3864008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:54.318 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.317+0000 7f386b400640 1 -- 172.21.15.123:0/3818605828 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f386007ac40 con 0x7f38741459f0 2024-04-24T05:45:54.519 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.517+0000 7f3878e00640 1 -- 172.21.15.123:0/3818605828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3874073c00 con 0x7f38741459f0 2024-04-24T05:45:54.521 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.519+0000 7f386b400640 1 -- 172.21.15.123:0/3818605828 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12920 (secure 0 0 0) 0x7f386007a540 con 0x7f38741459f0 2024-04-24T05:45:54.521 INFO:teuthology.misc.health.smithi123.stdout: 2024-04-24T05:45:54.521 INFO:teuthology.misc.health.smithi123.stdout:{"epoch":17,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:45:51.899740+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:45:54.524 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.522+0000 7f3878e00640 1 -- 172.21.15.123:0/3818605828 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f385406c8f0 msgr2=0x7f385406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:54.524 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.522+0000 7f3878e00640 1 --2- 172.21.15.123:0/3818605828 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f385406c8f0 0x7f385406edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f38741f7d20 tx=0x7f3864008040 comp rx=0 tx=0).stop 2024-04-24T05:45:54.524 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.522+0000 7f3878e00640 1 -- 172.21.15.123:0/3818605828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38741459f0 msgr2=0x7f387415b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:54.524 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.522+0000 7f3878e00640 1 --2- 172.21.15.123:0/3818605828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38741459f0 0x7f387415b8b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f386002a220 tx=0x7f38600046c0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.524 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.522+0000 7f3878e00640 1 -- 172.21.15.123:0/3818605828 shutdown_connections 2024-04-24T05:45:54.524 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.522+0000 7f3878e00640 1 --2- 172.21.15.123:0/3818605828 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f385406c8f0 0x7f385406edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.524 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.522+0000 7f3878e00640 1 --2- 172.21.15.123:0/3818605828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f387415efe0 0x7f38741442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.525 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.522+0000 7f3878e00640 1 --2- 172.21.15.123:0/3818605828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38741459f0 0x7f387415b8b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.525 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.522+0000 7f3878e00640 1 --2- 172.21.15.123:0/3818605828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38740cff10 0x7f387413a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.525 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.522+0000 7f3878e00640 1 -- 172.21.15.123:0/3818605828 >> 172.21.15.123:0/3818605828 conn(0x7f387406da30 msgr2=0x7f38740b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:54.525 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.523+0000 7f3878e00640 1 -- 172.21.15.123:0/3818605828 shutdown_connections 2024-04-24T05:45:54.525 INFO:teuthology.misc.health.smithi123.stderr:2024-04-24T05:45:54.523+0000 7f3878e00640 1 -- 172.21.15.123:0/3818605828 wait complete. 2024-04-24T05:45:54.539 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-24T05:45:54.539 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-24T05:45:54.539 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T05:45:54.539 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T05:45:54.544 INFO:tasks.ceph.mds.a:Started 2024-04-24T05:45:54.544 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T05:45:54.545 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T05:45:54.552 INFO:tasks.ceph.mds.c:Started 2024-04-24T05:45:54.552 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T05:45:54.552 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T05:45:54.560 INFO:tasks.ceph.mds.b:Started 2024-04-24T05:45:54.560 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T05:45:54.560 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T05:45:54.567 INFO:tasks.ceph.mds.d:Started 2024-04-24T05:45:54.567 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-24T05:45:54.568 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:45:54.690 INFO:tasks.ceph.mds.a.smithi123.stdout:starting mds.a at 2024-04-24T05:45:54.691 INFO:tasks.ceph.mds.c.smithi123.stdout:starting mds.c at 2024-04-24T05:45:54.715 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-24T05:45:54.716 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-24T05:45:54.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.741+0000 7f52e3200640 1 Processor -- start 2024-04-24T05:45:54.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.741+0000 7f52e3200640 1 -- start start 2024-04-24T05:45:54.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.742+0000 7f52e3200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f52dc06f090 0x7f52dc072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:54.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.743+0000 7f52e3200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f52dc06f470 0x7f52dc073500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:54.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.743+0000 7f52e3200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52dc071cd0 con 0x7f52dc0b2990 2024-04-24T05:45:54.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.743+0000 7f52e3200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f52dc071fb0 con 0x7f52dc06f470 2024-04-24T05:45:54.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.743+0000 7f52e3200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f52dc072290 con 0x7f52dc06f090 2024-04-24T05:45:54.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.743+0000 7f52d3400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f52dc06f470 0x7f52dc073500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:54.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.743+0000 7f52d3400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f52dc06f470 0x7f52dc073500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52618/0 (socket says 172.21.15.123:52618) 2024-04-24T05:45:54.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.743+0000 7f52d3400640 1 -- 172.21.15.123:0/3230852792 learned_addr learned my addr 172.21.15.123:0/3230852792 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:54.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.743+0000 7f52dbe00640 1 --2- 172.21.15.123:0/3230852792 >> v2:172.21.15.155:3300/0 conn(0x7f52dc06f090 0x7f52dc072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:54.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.743+0000 7f52d3400640 1 -- 172.21.15.123:0/3230852792 >> v2:172.21.15.155:3300/0 conn(0x7f52dc06f090 msgr2=0x7f52dc072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:54.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.743+0000 7f52d3400640 1 --2- 172.21.15.123:0/3230852792 >> v2:172.21.15.155:3300/0 conn(0x7f52dc06f090 0x7f52dc072fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.743+0000 7f52d3400640 1 -- 172.21.15.123:0/3230852792 >> v1:172.21.15.155:6789/0 conn(0x7f52dc0b2990 legacy=0x7f52dc151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:54.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.743+0000 7f52d3400640 1 -- 172.21.15.123:0/3230852792 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc072570 con 0x7f52dc06f470 2024-04-24T05:45:54.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.743+0000 7f52dbe00640 1 --2- 172.21.15.123:0/3230852792 >> v2:172.21.15.155:3300/0 conn(0x7f52dc06f090 0x7f52dc072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:45:54.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.744+0000 7f52d3400640 1 --2- 172.21.15.123:0/3230852792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52dc06f470 0x7f52dc073500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f52dc06fdd0 tx=0x7f52c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e993487562aad431 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:54.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.745+0000 7f52db400640 1 -- 172.21.15.123:0/3230852792 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4037070 con 0x7f52dc06f470 2024-04-24T05:45:54.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.745+0000 7f52db400640 1 -- 172.21.15.123:0/3230852792 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c4032440 con 0x7f52dc06f470 2024-04-24T05:45:54.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.745+0000 7f52db400640 1 -- 172.21.15.123:0/3230852792 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4032810 con 0x7f52dc06f470 2024-04-24T05:45:54.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.745+0000 7f52e3200640 1 -- 172.21.15.123:0/3230852792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52dc06f470 msgr2=0x7f52dc073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:54.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.745+0000 7f52e3200640 1 --2- 172.21.15.123:0/3230852792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52dc06f470 0x7f52dc073500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f52dc06fdd0 tx=0x7f52c402a1e0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.745+0000 7f52e3200640 1 -- 172.21.15.123:0/3230852792 shutdown_connections 2024-04-24T05:45:54.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.745+0000 7f52e3200640 1 --2- 172.21.15.123:0/3230852792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52dc06f470 0x7f52dc073500 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.745+0000 7f52e3200640 1 --2- 172.21.15.123:0/3230852792 >> v2:172.21.15.155:3300/0 conn(0x7f52dc06f090 0x7f52dc072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.745+0000 7f52e3200640 1 -- 172.21.15.123:0/3230852792 >> 172.21.15.123:0/3230852792 conn(0x7f52dc0b1e70 msgr2=0x7f52dc0b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:54.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.745+0000 7f52e3200640 1 -- 172.21.15.123:0/3230852792 shutdown_connections 2024-04-24T05:45:54.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.745+0000 7f52e3200640 1 -- 172.21.15.123:0/3230852792 wait complete. 2024-04-24T05:45:54.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.747+0000 7f52e3200640 1 Processor -- start 2024-04-24T05:45:54.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.747+0000 7f52e3200640 1 -- start start 2024-04-24T05:45:54.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.748+0000 7f52e3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52dc06f090 0x7f52dc0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:54.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.748+0000 7f52e3200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc06f470 0x7f52dc0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:54.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.748+0000 7f52e3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52dc0b2990 0x7f52dc151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:54.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.748+0000 7f52e3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc15c690 con 0x7f52dc06f090 2024-04-24T05:45:54.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.748+0000 7f52e3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc15c510 con 0x7f52dc0b2990 2024-04-24T05:45:54.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.748+0000 7f52e3200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f52dc072a90 con 0x7f52dc06f470 2024-04-24T05:45:54.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.748+0000 7f52dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52dc06f090 0x7f52dc0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:54.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.748+0000 7f52dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52dc06f090 0x7f52dc0ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52628/0 (socket says 172.21.15.123:52628) 2024-04-24T05:45:54.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.748+0000 7f52dbe00640 1 -- 172.21.15.123:0/3285226584 learned_addr learned my addr 172.21.15.123:0/3285226584 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:54.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.748+0000 7f52d3400640 1 --2- 172.21.15.123:0/3285226584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc06f470 0x7f52dc0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:54.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.748+0000 7f52e1200640 1 --2- 172.21.15.123:0/3285226584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52dc0b2990 0x7f52dc151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:54.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.749+0000 7f52dbe00640 1 -- 172.21.15.123:0/3285226584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc06f470 msgr2=0x7f52dc0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:54.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.749+0000 7f52dbe00640 1 --2- 172.21.15.123:0/3285226584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc06f470 0x7f52dc0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.749+0000 7f52dbe00640 1 -- 172.21.15.123:0/3285226584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52dc0b2990 msgr2=0x7f52dc151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:54.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.749+0000 7f52dbe00640 1 --2- 172.21.15.123:0/3285226584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52dc0b2990 0x7f52dc151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.749+0000 7f52dbe00640 1 -- 172.21.15.123:0/3285226584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc15e6a0 con 0x7f52dc06f090 2024-04-24T05:45:54.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.749+0000 7f52dbe00640 1 --2- 172.21.15.123:0/3285226584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52dc06f090 0x7f52dc0ce7d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009fc0 tx=0x7f52cc00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:54.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.749+0000 7f52d8c00640 1 -- 172.21.15.123:0/3285226584 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc01a070 con 0x7f52dc06f090 2024-04-24T05:45:54.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.749+0000 7f52d8c00640 1 -- 172.21.15.123:0/3285226584 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52cc003bb0 con 0x7f52dc06f090 2024-04-24T05:45:54.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.749+0000 7f52d8c00640 1 -- 172.21.15.123:0/3285226584 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc004030 con 0x7f52dc06f090 2024-04-24T05:45:54.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.749+0000 7f52e3200640 1 -- 172.21.15.123:0/3285226584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52dc15e980 con 0x7f52dc06f090 2024-04-24T05:45:54.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.749+0000 7f52e3200640 1 -- 172.21.15.123:0/3285226584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52dc15ef40 con 0x7f52dc06f090 2024-04-24T05:45:54.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.751+0000 7f52e3200640 1 -- 172.21.15.123:0/3285226584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52dc068a20 con 0x7f52dc06f090 2024-04-24T05:45:54.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.751+0000 7f52d8c00640 1 -- 172.21.15.123:0/3285226584 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f52cc01d070 con 0x7f52dc06f090 2024-04-24T05:45:54.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.752+0000 7f52d8c00640 1 --2- 172.21.15.123:0/3285226584 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f52b806c8f0 0x7f52b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:54.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.752+0000 7f52d8c00640 1 -- 172.21.15.123:0/3285226584 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f52cc08bb10 con 0x7f52dc06f090 2024-04-24T05:45:54.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.753+0000 7f52d3400640 1 --2- 172.21.15.123:0/3285226584 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f52b806c8f0 0x7f52b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:54.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.753+0000 7f52d3400640 1 --2- 172.21.15.123:0/3285226584 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f52b806c8f0 0x7f52b806edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f52c4034000 tx=0x7f52c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:54.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.757+0000 7f52d8c00640 1 -- 172.21.15.123:0/3285226584 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52cc010070 con 0x7f52dc06f090 2024-04-24T05:45:54.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.949+0000 7f52e3200640 1 -- 172.21.15.123:0/3285226584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52dc15a8e0 con 0x7f52dc06f090 2024-04-24T05:45:54.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.950+0000 7f52d8c00640 1 -- 172.21.15.123:0/3285226584 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12920 (secure 0 0 0) 0x7f52cc059ba0 con 0x7f52dc06f090 2024-04-24T05:45:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:45:54.952 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":17,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:45:51.899740+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:45:54.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.953+0000 7f52e3200640 1 -- 172.21.15.123:0/3285226584 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f52b806c8f0 msgr2=0x7f52b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:54.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.953+0000 7f52e3200640 1 --2- 172.21.15.123:0/3285226584 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f52b806c8f0 0x7f52b806edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f52c4034000 tx=0x7f52c402e000 comp rx=0 tx=0).stop 2024-04-24T05:45:54.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.953+0000 7f52e3200640 1 -- 172.21.15.123:0/3285226584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52dc06f090 msgr2=0x7f52dc0ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:54.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.953+0000 7f52e3200640 1 --2- 172.21.15.123:0/3285226584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52dc06f090 0x7f52dc0ce7d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009fc0 tx=0x7f52cc00d560 comp rx=0 tx=0).stop 2024-04-24T05:45:54.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.953+0000 7f52e3200640 1 -- 172.21.15.123:0/3285226584 shutdown_connections 2024-04-24T05:45:54.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.953+0000 7f52e3200640 1 --2- 172.21.15.123:0/3285226584 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f52b806c8f0 0x7f52b806edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.953+0000 7f52e3200640 1 --2- 172.21.15.123:0/3285226584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52dc0b2990 0x7f52dc151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.953+0000 7f52e3200640 1 --2- 172.21.15.123:0/3285226584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52dc06f470 0x7f52dc0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.953+0000 7f52e3200640 1 --2- 172.21.15.123:0/3285226584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52dc06f090 0x7f52dc0ce7d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:54.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.953+0000 7f52e3200640 1 -- 172.21.15.123:0/3285226584 >> 172.21.15.123:0/3285226584 conn(0x7f52dc0b1e70 msgr2=0x7f52dc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:54.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.953+0000 7f52e3200640 1 -- 172.21.15.123:0/3285226584 shutdown_connections 2024-04-24T05:45:54.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:54.953+0000 7f52e3200640 1 -- 172.21.15.123:0/3285226584 wait complete. 2024-04-24T05:45:54.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T05:45:54.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:45:55.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.159+0000 7f6479a00640 1 Processor -- start 2024-04-24T05:45:55.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.159+0000 7f6479a00640 1 -- start start 2024-04-24T05:45:55.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.160+0000 7f6479a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6474071c50 0x7f64740ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:55.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.160+0000 7f6479a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f64740cd0f0 0x7f64740d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:55.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.161+0000 7f6479a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64740cf170 con 0x7f647406c510 2024-04-24T05:45:55.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.161+0000 7f6479a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f64740cd520 con 0x7f6474071c50 2024-04-24T05:45:55.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.161+0000 7f6479a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f64740cf450 con 0x7f64740cd0f0 2024-04-24T05:45:55.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.161+0000 7f6472a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f647406c510 0x7f647406c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:47346/0 (socket says 172.21.15.123:47346) 2024-04-24T05:45:55.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.161+0000 7f6472a00640 1 -- 172.21.15.123:0/3673251445 learned_addr learned my addr 172.21.15.123:0/3673251445 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:45:55.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.161+0000 7f6472000640 1 --2- 172.21.15.123:0/3673251445 >> v2:172.21.15.123:3300/0 conn(0x7f6474071c50 0x7f64740ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:55.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.161+0000 7f6473400640 1 --2- 172.21.15.123:0/3673251445 >> v2:172.21.15.155:3300/0 conn(0x7f64740cd0f0 0x7f64740d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:55.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.161+0000 7f6471600640 1 -- 172.21.15.123:0/3673251445 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4190840256 0 0) 0x7f64740cf170 con 0x7f647406c510 2024-04-24T05:45:55.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.161+0000 7f6471600640 1 -- 172.21.15.123:0/3673251445 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6458003520 con 0x7f647406c510 2024-04-24T05:45:55.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.161+0000 7f6472000640 1 -- 172.21.15.123:0/3673251445 >> v2:172.21.15.155:3300/0 conn(0x7f64740cd0f0 msgr2=0x7f64740d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:55.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.161+0000 7f6472000640 1 --2- 172.21.15.123:0/3673251445 >> v2:172.21.15.155:3300/0 conn(0x7f64740cd0f0 0x7f64740d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.161+0000 7f6472000640 1 -- 172.21.15.123:0/3673251445 >> v1:172.21.15.123:6789/0 conn(0x7f647406c510 legacy=0x7f647406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:55.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.162+0000 7f6472000640 1 -- 172.21.15.123:0/3673251445 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64740cf730 con 0x7f6474071c50 2024-04-24T05:45:55.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.162+0000 7f6472000640 1 --2- 172.21.15.123:0/3673251445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6474071c50 0x7f64740ccbb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f647406da40 tx=0x7f646002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cb326d098b017f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:55.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.163+0000 7f6471600640 1 -- 172.21.15.123:0/3673251445 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6460037070 con 0x7f6474071c50 2024-04-24T05:45:55.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.163+0000 7f6471600640 1 -- 172.21.15.123:0/3673251445 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6460032440 con 0x7f6474071c50 2024-04-24T05:45:55.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.163+0000 7f6471600640 1 -- 172.21.15.123:0/3673251445 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6460032810 con 0x7f6474071c50 2024-04-24T05:45:55.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.163+0000 7f6479a00640 1 -- 172.21.15.123:0/3673251445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6474071c50 msgr2=0x7f64740ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:55.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.163+0000 7f6479a00640 1 --2- 172.21.15.123:0/3673251445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6474071c50 0x7f64740ccbb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f647406da40 tx=0x7f646002a1a0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.164+0000 7f6479a00640 1 -- 172.21.15.123:0/3673251445 shutdown_connections 2024-04-24T05:45:55.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.164+0000 7f6479a00640 1 --2- 172.21.15.123:0/3673251445 >> v2:172.21.15.155:3300/0 conn(0x7f64740cd0f0 0x7f64740d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.164+0000 7f6479a00640 1 --2- 172.21.15.123:0/3673251445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6474071c50 0x7f64740ccbb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.164+0000 7f6479a00640 1 -- 172.21.15.123:0/3673251445 >> 172.21.15.123:0/3673251445 conn(0x7f64740b51f0 msgr2=0x7f64740b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:55.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.164+0000 7f6479a00640 1 -- 172.21.15.123:0/3673251445 shutdown_connections 2024-04-24T05:45:55.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.164+0000 7f6479a00640 1 -- 172.21.15.123:0/3673251445 wait complete. 2024-04-24T05:45:55.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.165+0000 7f6479a00640 1 Processor -- start 2024-04-24T05:45:55.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.165+0000 7f6479a00640 1 -- start start 2024-04-24T05:45:55.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.166+0000 7f6479a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f647406c510 0x7f647409da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:55.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.166+0000 7f6479a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6474071c50 0x7f647415e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.166+0000 7f6479a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64740cd0f0 0x7f64740cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.166+0000 7f6479a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f647406d000 con 0x7f64740cd0f0 2024-04-24T05:45:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.166+0000 7f6479a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f64740cfcf0 con 0x7f6474071c50 2024-04-24T05:45:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.166+0000 7f6479a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f647406d180 con 0x7f647406c510 2024-04-24T05:45:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.167+0000 7f6473400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64740cd0f0 0x7f64740cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.167+0000 7f6473400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64740cd0f0 0x7f64740cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52652/0 (socket says 172.21.15.123:52652) 2024-04-24T05:45:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.167+0000 7f6473400640 1 -- 172.21.15.123:0/1245916518 learned_addr learned my addr 172.21.15.123:0/1245916518 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.167+0000 7f6472000640 1 --2- 172.21.15.123:0/1245916518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6474071c50 0x7f647415e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.167+0000 7f6472a00640 1 --2- 172.21.15.123:0/1245916518 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f647406c510 0x7f647409da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.167+0000 7f6473400640 1 -- 172.21.15.123:0/1245916518 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f647406c510 msgr2=0x7f647409da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.167+0000 7f6473400640 1 --2- 172.21.15.123:0/1245916518 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f647406c510 0x7f647409da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.167+0000 7f6473400640 1 -- 172.21.15.123:0/1245916518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6474071c50 msgr2=0x7f647415e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.167+0000 7f6473400640 1 --2- 172.21.15.123:0/1245916518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6474071c50 0x7f647415e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.167+0000 7f6473400640 1 -- 172.21.15.123:0/1245916518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647409dfb0 con 0x7f64740cd0f0 2024-04-24T05:45:55.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.167+0000 7f6473400640 1 --2- 172.21.15.123:0/1245916518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64740cd0f0 0x7f64740cb3e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f646c00efc0 tx=0x7f646c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:55.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.167+0000 7f646aa00640 1 -- 172.21.15.123:0/1245916518 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c018020 con 0x7f64740cd0f0 2024-04-24T05:45:55.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.168+0000 7f646aa00640 1 -- 172.21.15.123:0/1245916518 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f646c014480 con 0x7f64740cd0f0 2024-04-24T05:45:55.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.168+0000 7f646aa00640 1 -- 172.21.15.123:0/1245916518 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c014870 con 0x7f64740cd0f0 2024-04-24T05:45:55.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.168+0000 7f6479a00640 1 -- 172.21.15.123:0/1245916518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647415f4c0 con 0x7f64740cd0f0 2024-04-24T05:45:55.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.168+0000 7f6479a00640 1 -- 172.21.15.123:0/1245916518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6474159580 con 0x7f64740cd0f0 2024-04-24T05:45:55.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.169+0000 7f6479a00640 1 -- 172.21.15.123:0/1245916518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6474068a20 con 0x7f64740cd0f0 2024-04-24T05:45:55.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.170+0000 7f646aa00640 1 -- 172.21.15.123:0/1245916518 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f646c005d10 con 0x7f64740cd0f0 2024-04-24T05:45:55.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.171+0000 7f646aa00640 1 --2- 172.21.15.123:0/1245916518 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f645406c8f0 0x7f645406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:55.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.171+0000 7f646aa00640 1 -- 172.21.15.123:0/1245916518 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f646c08c650 con 0x7f64740cd0f0 2024-04-24T05:45:55.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.171+0000 7f6472a00640 1 --2- 172.21.15.123:0/1245916518 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f645406c8f0 0x7f645406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:55.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.172+0000 7f6472a00640 1 --2- 172.21.15.123:0/1245916518 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f645406c8f0 0x7f645406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6464005880 tx=0x7f6464009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:55.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.176+0000 7f646aa00640 1 -- 172.21.15.123:0/1245916518 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f646c022030 con 0x7f64740cd0f0 2024-04-24T05:45:55.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.378+0000 7f6479a00640 1 -- 172.21.15.123:0/1245916518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f64740b5e50 con 0x7f64740cd0f0 2024-04-24T05:45:55.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.379+0000 7f646aa00640 1 -- 172.21.15.123:0/1245916518 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f647409dfb0 con 0x7f64740cd0f0 2024-04-24T05:45:55.381 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:45:55.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.382+0000 7f6479a00640 1 -- 172.21.15.123:0/1245916518 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f645406c8f0 msgr2=0x7f645406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:55.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.382+0000 7f6479a00640 1 --2- 172.21.15.123:0/1245916518 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f645406c8f0 0x7f645406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6464005880 tx=0x7f6464009040 comp rx=0 tx=0).stop 2024-04-24T05:45:55.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.382+0000 7f6479a00640 1 -- 172.21.15.123:0/1245916518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64740cd0f0 msgr2=0x7f64740cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:55.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.382+0000 7f6479a00640 1 --2- 172.21.15.123:0/1245916518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64740cd0f0 0x7f64740cb3e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f646c00efc0 tx=0x7f646c00c490 comp rx=0 tx=0).stop 2024-04-24T05:45:55.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.382+0000 7f6479a00640 1 -- 172.21.15.123:0/1245916518 shutdown_connections 2024-04-24T05:45:55.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.382+0000 7f6479a00640 1 --2- 172.21.15.123:0/1245916518 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f645406c8f0 0x7f645406edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.382+0000 7f6479a00640 1 --2- 172.21.15.123:0/1245916518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64740cd0f0 0x7f64740cb3e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.382+0000 7f6479a00640 1 --2- 172.21.15.123:0/1245916518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6474071c50 0x7f647415e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.382+0000 7f6479a00640 1 --2- 172.21.15.123:0/1245916518 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f647406c510 0x7f647409da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.382+0000 7f6479a00640 1 -- 172.21.15.123:0/1245916518 >> 172.21.15.123:0/1245916518 conn(0x7f64740b51f0 msgr2=0x7f647406c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:55.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.382+0000 7f6479a00640 1 -- 172.21.15.123:0/1245916518 shutdown_connections 2024-04-24T05:45:55.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.382+0000 7f6479a00640 1 -- 172.21.15.123:0/1245916518 wait complete. 2024-04-24T05:45:55.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T05:45:55.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.592+0000 7f09c5400640 1 Processor -- start 2024-04-24T05:45:55.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.592+0000 7f09c5400640 1 -- start start 2024-04-24T05:45:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.593+0000 7f09c5400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09c00b4f70 0x7f09c006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.593+0000 7f09c5400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09c00cc1e0 0x7f09c006e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.594+0000 7f09c5400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09c00cf1c0 con 0x7f09c006e660 2024-04-24T05:45:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.594+0000 7f09c5400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f09c00cfd90 con 0x7f09c00cc1e0 2024-04-24T05:45:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.594+0000 7f09c5400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f09c00cf460 con 0x7f09c00b4f70 2024-04-24T05:45:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.594+0000 7f09b7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09c00cc1e0 0x7f09c006e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.594+0000 7f09b7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09c00cc1e0 0x7f09c006e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52664/0 (socket says 172.21.15.123:52664) 2024-04-24T05:45:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.594+0000 7f09b7400640 1 -- 172.21.15.123:0/512108604 learned_addr learned my addr 172.21.15.123:0/512108604 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.594+0000 7f09bfe00640 1 --2- 172.21.15.123:0/512108604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09c00b4f70 0x7f09c006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.594+0000 7f09b7400640 1 -- 172.21.15.123:0/512108604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09c00b4f70 msgr2=0x7f09c006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.594+0000 7f09b7400640 1 --2- 172.21.15.123:0/512108604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09c00b4f70 0x7f09c006dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.594+0000 7f09b7400640 1 -- 172.21.15.123:0/512108604 >> v1:172.21.15.155:6789/0 conn(0x7f09c006e660 legacy=0x7f09c00acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:55.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.595+0000 7f09b7400640 1 -- 172.21.15.123:0/512108604 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c00cf740 con 0x7f09c00cc1e0 2024-04-24T05:45:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.595+0000 7f09b7400640 1 --2- 172.21.15.123:0/512108604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c00cc1e0 0x7f09c006e120 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009870 tx=0x7f09a802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee87342fa2cd2b09 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.596+0000 7f09bf400640 1 -- 172.21.15.123:0/512108604 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8037070 con 0x7f09c00cc1e0 2024-04-24T05:45:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.596+0000 7f09bf400640 1 -- 172.21.15.123:0/512108604 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a8032440 con 0x7f09c00cc1e0 2024-04-24T05:45:55.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.596+0000 7f09bf400640 1 -- 172.21.15.123:0/512108604 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8032810 con 0x7f09c00cc1e0 2024-04-24T05:45:55.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.596+0000 7f09c5400640 1 -- 172.21.15.123:0/512108604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c00cc1e0 msgr2=0x7f09c006e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:55.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.596+0000 7f09c5400640 1 --2- 172.21.15.123:0/512108604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c00cc1e0 0x7f09c006e120 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009870 tx=0x7f09a802a1c0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.596+0000 7f09c5400640 1 -- 172.21.15.123:0/512108604 shutdown_connections 2024-04-24T05:45:55.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.596+0000 7f09c5400640 1 --2- 172.21.15.123:0/512108604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c00cc1e0 0x7f09c006e120 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.596+0000 7f09c5400640 1 --2- 172.21.15.123:0/512108604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09c00b4f70 0x7f09c006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.596+0000 7f09c5400640 1 -- 172.21.15.123:0/512108604 >> 172.21.15.123:0/512108604 conn(0x7f09c00d0070 msgr2=0x7f09c00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:55.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.597+0000 7f09c5400640 1 -- 172.21.15.123:0/512108604 shutdown_connections 2024-04-24T05:45:55.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.597+0000 7f09c5400640 1 -- 172.21.15.123:0/512108604 wait complete. 2024-04-24T05:45:55.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.598+0000 7f09c5400640 1 Processor -- start 2024-04-24T05:45:55.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.598+0000 7f09c5400640 1 -- start start 2024-04-24T05:45:55.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.599+0000 7f09c5400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c006e660 0x7f09c009d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:55.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.599+0000 7f09c5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c00b4f70 0x7f09c01e3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.599+0000 7f09c5400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09c00cc1e0 0x7f09c014b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.599+0000 7f09c5400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09c00d0e00 con 0x7f09c00b4f70 2024-04-24T05:45:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.599+0000 7f09c5400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f09c00b2f60 con 0x7f09c006e660 2024-04-24T05:45:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.599+0000 7f09c5400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f09c00d0f80 con 0x7f09c00cc1e0 2024-04-24T05:45:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.599+0000 7f09b7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c00b4f70 0x7f09c01e3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.599+0000 7f09b7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c00b4f70 0x7f09c01e3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52668/0 (socket says 172.21.15.123:52668) 2024-04-24T05:45:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.599+0000 7f09b7400640 1 -- 172.21.15.123:0/2508723602 learned_addr learned my addr 172.21.15.123:0/2508723602 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.600+0000 7f09c4a00640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09c00cc1e0 0x7f09c014b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.600+0000 7f09b7400640 1 -- 172.21.15.123:0/2508723602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09c00cc1e0 msgr2=0x7f09c014b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.600+0000 7f09b7400640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09c00cc1e0 0x7f09c014b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.600+0000 7f09bfe00640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c006e660 0x7f09c009d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.600+0000 7f09b7400640 1 -- 172.21.15.123:0/2508723602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c006e660 msgr2=0x7f09c009d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:55.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.600+0000 7f09b7400640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c006e660 0x7f09c009d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.600+0000 7f09b7400640 1 -- 172.21.15.123:0/2508723602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c009d7f0 con 0x7f09c00b4f70 2024-04-24T05:45:55.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.600+0000 7f09c4a00640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09c00cc1e0 0x7f09c014b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:45:55.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.600+0000 7f09b7400640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c00b4f70 0x7f09c01e3550 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009870 tx=0x7f09a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:55.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.600+0000 7f09bcc00640 1 -- 172.21.15.123:0/2508723602 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8041070 con 0x7f09c00b4f70 2024-04-24T05:45:55.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.601+0000 7f09bcc00640 1 -- 172.21.15.123:0/2508723602 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a803c890 con 0x7f09c00b4f70 2024-04-24T05:45:55.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.601+0000 7f09c5400640 1 -- 172.21.15.123:0/2508723602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c009ad30 con 0x7f09c00b4f70 2024-04-24T05:45:55.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.601+0000 7f09bcc00640 1 -- 172.21.15.123:0/2508723602 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8037040 con 0x7f09c00b4f70 2024-04-24T05:45:55.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.601+0000 7f09c5400640 1 -- 172.21.15.123:0/2508723602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c009b2f0 con 0x7f09c00b4f70 2024-04-24T05:45:55.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.602+0000 7f09c5400640 1 -- 172.21.15.123:0/2508723602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c015adc0 con 0x7f09c00b4f70 2024-04-24T05:45:55.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.603+0000 7f09bcc00640 1 -- 172.21.15.123:0/2508723602 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09a8044070 con 0x7f09c00b4f70 2024-04-24T05:45:55.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.604+0000 7f09bcc00640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f099c06c8f0 0x7f099c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:55.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.604+0000 7f09bcc00640 1 -- 172.21.15.123:0/2508723602 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f09a80ac200 con 0x7f09c00b4f70 2024-04-24T05:45:55.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.604+0000 7f09bfe00640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f099c06c8f0 0x7f099c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:55.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.605+0000 7f09bfe00640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f099c06c8f0 0x7f099c06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f09b0006fd0 tx=0x7f09b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:55.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.608+0000 7f09bcc00640 1 -- 172.21.15.123:0/2508723602 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f09c015adc0 con 0x7f09c00b4f70 2024-04-24T05:45:55.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.851+0000 7f09c5400640 1 -- 172.21.15.123:0/2508723602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09c0075b70 con 0x7f09c00b4f70 2024-04-24T05:45:55.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.852+0000 7f09bcc00640 1 -- 172.21.15.123:0/2508723602 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3876 (secure 0 0 0) 0x7f09a807aa10 con 0x7f09c00b4f70 2024-04-24T05:45:55.853 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:45:55.854 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2743935658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2743935658},{"type":"v1","addr":"172.21.15.155:6834","nonce":2743935658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1148561953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1148561953},{"type":"v1","addr":"172.21.15.123:6836","nonce":1148561953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2992052647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2992052647},{"type":"v1","addr":"172.21.15.155:6835","nonce":2992052647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3370584410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3370584410},{"type":"v1","addr":"172.21.15.123:6837","nonce":3370584410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T05:45:55.854 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 3 2024-04-24T05:45:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.854+0000 7f09c5400640 1 -- 172.21.15.123:0/2508723602 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f099c06c8f0 msgr2=0x7f099c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.854+0000 7f09c5400640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f099c06c8f0 0x7f099c06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f09b0006fd0 tx=0x7f09b0008040 comp rx=0 tx=0).stop 2024-04-24T05:45:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.854+0000 7f09c5400640 1 -- 172.21.15.123:0/2508723602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c00b4f70 msgr2=0x7f09c01e3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.854+0000 7f09c5400640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c00b4f70 0x7f09c01e3550 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009870 tx=0x7f09a80046c0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.854+0000 7f09c5400640 1 -- 172.21.15.123:0/2508723602 shutdown_connections 2024-04-24T05:45:55.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.854+0000 7f09c5400640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f099c06c8f0 0x7f099c06edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.854+0000 7f09c5400640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09c00cc1e0 0x7f09c014b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.854+0000 7f09c5400640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c00b4f70 0x7f09c01e3550 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.854+0000 7f09c5400640 1 --2- 172.21.15.123:0/2508723602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c006e660 0x7f09c009d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.855+0000 7f09c5400640 1 -- 172.21.15.123:0/2508723602 >> 172.21.15.123:0/2508723602 conn(0x7f09c00d0070 msgr2=0x7f09c0158b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.855+0000 7f09c5400640 1 -- 172.21.15.123:0/2508723602 shutdown_connections 2024-04-24T05:45:55.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:55.855+0000 7f09c5400640 1 -- 172.21.15.123:0/2508723602 wait complete. 2024-04-24T05:45:55.874 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:45:56.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.073+0000 7fda01e00640 1 Processor -- start 2024-04-24T05:45:56.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.073+0000 7fda01e00640 1 -- start start 2024-04-24T05:45:56.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.074+0000 7fda01e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd9fc15a2a0 0x7fd9fc0ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.074+0000 7fda01e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd9fc0af950 0x7fd9fc06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.074+0000 7fda01e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9fc15cdf0 con 0x7fd9fc0ad690 2024-04-24T05:45:56.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.074+0000 7fda01e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd9fc15a680 con 0x7fd9fc15a2a0 2024-04-24T05:45:56.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.074+0000 7fda01e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd9fc15da00 con 0x7fd9fc0af950 2024-04-24T05:45:56.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.075+0000 7fd9faa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd9fc15a2a0 0x7fd9fc0ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.075+0000 7fd9faa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd9fc15a2a0 0x7fd9fc0ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52684/0 (socket says 172.21.15.123:52684) 2024-04-24T05:45:56.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.075+0000 7fd9faa00640 1 -- 172.21.15.123:0/2105928478 learned_addr learned my addr 172.21.15.123:0/2105928478 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:56.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.075+0000 7fd9fb400640 1 --2- 172.21.15.123:0/2105928478 >> v2:172.21.15.155:3300/0 conn(0x7fd9fc0af950 0x7fd9fc06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.075+0000 7fd9faa00640 1 -- 172.21.15.123:0/2105928478 >> v2:172.21.15.155:3300/0 conn(0x7fd9fc0af950 msgr2=0x7fd9fc06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:56.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.075+0000 7fd9faa00640 1 --2- 172.21.15.123:0/2105928478 >> v2:172.21.15.155:3300/0 conn(0x7fd9fc0af950 0x7fd9fc06bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.075+0000 7fd9faa00640 1 -- 172.21.15.123:0/2105928478 >> v1:172.21.15.123:6789/0 conn(0x7fd9fc0ad690 legacy=0x7fd9fc0af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.075+0000 7fd9faa00640 1 -- 172.21.15.123:0/2105928478 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9fc15d0d0 con 0x7fd9fc15a2a0 2024-04-24T05:45:56.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.076+0000 7fd9faa00640 1 --2- 172.21.15.123:0/2105928478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc15a2a0 0x7fd9fc0ad150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc075290 tx=0x7fd9ec02a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9753f19db31fb10 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.077+0000 7fd9f9600640 1 -- 172.21.15.123:0/2105928478 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec037070 con 0x7fd9fc15a2a0 2024-04-24T05:45:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.077+0000 7fd9f9600640 1 -- 172.21.15.123:0/2105928478 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9ec032440 con 0x7fd9fc15a2a0 2024-04-24T05:45:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.077+0000 7fd9f9600640 1 -- 172.21.15.123:0/2105928478 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec032810 con 0x7fd9fc15a2a0 2024-04-24T05:45:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.077+0000 7fda01e00640 1 -- 172.21.15.123:0/2105928478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc15a2a0 msgr2=0x7fd9fc0ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.077+0000 7fda01e00640 1 --2- 172.21.15.123:0/2105928478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc15a2a0 0x7fd9fc0ad150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc075290 tx=0x7fd9ec02a1f0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.077+0000 7fda01e00640 1 -- 172.21.15.123:0/2105928478 shutdown_connections 2024-04-24T05:45:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.077+0000 7fda01e00640 1 --2- 172.21.15.123:0/2105928478 >> v2:172.21.15.155:3300/0 conn(0x7fd9fc0af950 0x7fd9fc06bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.077+0000 7fda01e00640 1 --2- 172.21.15.123:0/2105928478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc15a2a0 0x7fd9fc0ad150 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.077+0000 7fda01e00640 1 -- 172.21.15.123:0/2105928478 >> 172.21.15.123:0/2105928478 conn(0x7fd9fc0b1e70 msgr2=0x7fd9fc0b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.078+0000 7fda01e00640 1 -- 172.21.15.123:0/2105928478 shutdown_connections 2024-04-24T05:45:56.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.078+0000 7fda01e00640 1 -- 172.21.15.123:0/2105928478 wait complete. 2024-04-24T05:45:56.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.079+0000 7fda01e00640 1 Processor -- start 2024-04-24T05:45:56.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.079+0000 7fda01e00640 1 -- start start 2024-04-24T05:45:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.080+0000 7fda01e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc0ad690 0x7fd9fc072830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.080+0000 7fda01e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9fc0af950 0x7fd9fc147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.080+0000 7fda01e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9fc15a2a0 0x7fd9fc0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.080+0000 7fda01e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc06c040 con 0x7fd9fc0ad690 2024-04-24T05:45:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.080+0000 7fda01e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc15d690 con 0x7fd9fc0af950 2024-04-24T05:45:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.080+0000 7fda01e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd9fc06c1c0 con 0x7fd9fc15a2a0 2024-04-24T05:45:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.081+0000 7fd9faa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc0ad690 0x7fd9fc072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.081+0000 7fd9faa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc0ad690 0x7fd9fc072830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52696/0 (socket says 172.21.15.123:52696) 2024-04-24T05:45:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.081+0000 7fd9faa00640 1 -- 172.21.15.123:0/1203474393 learned_addr learned my addr 172.21.15.123:0/1203474393 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.081+0000 7fd9fa000640 1 --2- 172.21.15.123:0/1203474393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9fc0af950 0x7fd9fc147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.081+0000 7fd9faa00640 1 -- 172.21.15.123:0/1203474393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9fc15a2a0 msgr2=0x7fd9fc0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.081+0000 7fd9faa00640 1 --2- 172.21.15.123:0/1203474393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9fc15a2a0 0x7fd9fc0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.081+0000 7fd9faa00640 1 -- 172.21.15.123:0/1203474393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9fc0af950 msgr2=0x7fd9fc147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.081+0000 7fd9faa00640 1 --2- 172.21.15.123:0/1203474393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9fc0af950 0x7fd9fc147520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.081+0000 7fd9faa00640 1 -- 172.21.15.123:0/1203474393 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9fc0ca920 con 0x7fd9fc0ad690 2024-04-24T05:45:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.081+0000 7fd9fa000640 1 --2- 172.21.15.123:0/1203474393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9fc0af950 0x7fd9fc147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:45:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.081+0000 7fd9faa00640 1 --2- 172.21.15.123:0/1203474393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc0ad690 0x7fd9fc072830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec02a230 tx=0x7fd9ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:56.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.082+0000 7fd9f3400640 1 -- 172.21.15.123:0/1203474393 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec041020 con 0x7fd9fc0ad690 2024-04-24T05:45:56.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.082+0000 7fd9f3400640 1 -- 172.21.15.123:0/1203474393 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9ec0306c0 con 0x7fd9fc0ad690 2024-04-24T05:45:56.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.082+0000 7fda01e00640 1 -- 172.21.15.123:0/1203474393 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9fc0cac00 con 0x7fd9fc0ad690 2024-04-24T05:45:56.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.082+0000 7fd9f3400640 1 -- 172.21.15.123:0/1203474393 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec037040 con 0x7fd9fc0ad690 2024-04-24T05:45:56.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.082+0000 7fda01e00640 1 -- 172.21.15.123:0/1203474393 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9fc0cb1c0 con 0x7fd9fc0ad690 2024-04-24T05:45:56.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.083+0000 7fda01e00640 1 -- 172.21.15.123:0/1203474393 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9fc068a20 con 0x7fd9fc0ad690 2024-04-24T05:45:56.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.084+0000 7fd9f3400640 1 -- 172.21.15.123:0/1203474393 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9ec044020 con 0x7fd9fc0ad690 2024-04-24T05:45:56.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.085+0000 7fd9f3400640 1 --2- 172.21.15.123:0/1203474393 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd9dc06c8f0 0x7fd9dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.085+0000 7fd9f3400640 1 -- 172.21.15.123:0/1203474393 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd9ec0ac1b0 con 0x7fd9fc0ad690 2024-04-24T05:45:56.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.085+0000 7fd9fa000640 1 --2- 172.21.15.123:0/1203474393 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd9dc06c8f0 0x7fd9dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.086+0000 7fd9fa000640 1 --2- 172.21.15.123:0/1203474393 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd9dc06c8f0 0x7fd9dc06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc1f8200 tx=0x7fd9e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:56.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.089+0000 7fd9f3400640 1 -- 172.21.15.123:0/1203474393 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd9ec04b030 con 0x7fd9fc0ad690 2024-04-24T05:45:56.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.294+0000 7fda01e00640 1 -- 172.21.15.123:0/1203474393 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9fc161070 con 0x7fd9fc0ad690 2024-04-24T05:45:56.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.296+0000 7fd9f3400640 1 -- 172.21.15.123:0/1203474393 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12920 (secure 0 0 0) 0x7fd9ec07a800 con 0x7fd9fc0ad690 2024-04-24T05:45:56.297 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:45:56.298 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":17,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:45:51.899740+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:45:56.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.298+0000 7fda01e00640 1 -- 172.21.15.123:0/1203474393 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd9dc06c8f0 msgr2=0x7fd9dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.298+0000 7fda01e00640 1 --2- 172.21.15.123:0/1203474393 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd9dc06c8f0 0x7fd9dc06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc1f8200 tx=0x7fd9e8009040 comp rx=0 tx=0).stop 2024-04-24T05:45:56.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.298+0000 7fda01e00640 1 -- 172.21.15.123:0/1203474393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc0ad690 msgr2=0x7fd9fc072830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.299+0000 7fda01e00640 1 --2- 172.21.15.123:0/1203474393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc0ad690 0x7fd9fc072830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec02a230 tx=0x7fd9ec0046c0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.299+0000 7fda01e00640 1 -- 172.21.15.123:0/1203474393 shutdown_connections 2024-04-24T05:45:56.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.299+0000 7fda01e00640 1 --2- 172.21.15.123:0/1203474393 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd9dc06c8f0 0x7fd9dc06edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.299+0000 7fda01e00640 1 --2- 172.21.15.123:0/1203474393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9fc15a2a0 0x7fd9fc0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.299+0000 7fda01e00640 1 --2- 172.21.15.123:0/1203474393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9fc0af950 0x7fd9fc147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.299+0000 7fda01e00640 1 --2- 172.21.15.123:0/1203474393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9fc0ad690 0x7fd9fc072830 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.299+0000 7fda01e00640 1 -- 172.21.15.123:0/1203474393 >> 172.21.15.123:0/1203474393 conn(0x7fd9fc0b1e70 msgr2=0x7fd9fc0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:56.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.299+0000 7fda01e00640 1 -- 172.21.15.123:0/1203474393 shutdown_connections 2024-04-24T05:45:56.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.299+0000 7fda01e00640 1 -- 172.21.15.123:0/1203474393 wait complete. 2024-04-24T05:45:56.320 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T05:45:56.320 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:45:56.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.523+0000 7f301a800640 1 Processor -- start 2024-04-24T05:45:56.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.523+0000 7f301a800640 1 -- start start 2024-04-24T05:45:56.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.524+0000 7f301a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f301415c2f0 0x7f3014141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.524+0000 7f301a800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f301406ba00 con 0x7f30140cc5d0 2024-04-24T05:45:56.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.524+0000 7f301a800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3014142530 con 0x7f30140ad6e0 2024-04-24T05:45:56.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.524+0000 7f301a800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f301406bd30 con 0x7f301415c2f0 2024-04-24T05:45:56.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.524+0000 7f3018a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f30140cc5d0 0x7f30140acfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:47368/0 (socket says 172.21.15.123:47368) 2024-04-24T05:45:56.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.524+0000 7f3018a00640 1 -- 172.21.15.123:0/4139743449 learned_addr learned my addr 172.21.15.123:0/4139743449 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:45:56.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3019400640 1 --2- 172.21.15.123:0/4139743449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f301415c2f0 0x7f3014141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 -- 172.21.15.123:0/4139743449 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 9655030 0 0) 0x7f301406ba00 con 0x7f30140cc5d0 2024-04-24T05:45:56.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 -- 172.21.15.123:0/4139743449 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ffc003520 con 0x7f30140cc5d0 2024-04-24T05:45:56.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 -- 172.21.15.123:0/4139743449 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2138713163 0 0) 0x7f2ffc003520 con 0x7f30140cc5d0 2024-04-24T05:45:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 -- 172.21.15.123:0/4139743449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f301415c2f0 msgr2=0x7f3014141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 --2- 172.21.15.123:0/4139743449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f301415c2f0 0x7f3014141ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 -- 172.21.15.123:0/4139743449 >> v1:172.21.15.155:6789/0 conn(0x7f30140ad6e0 legacy=0x7f301406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 -- 172.21.15.123:0/4139743449 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3014142810 con 0x7f30140cc5d0 2024-04-24T05:45:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 -- 172.21.15.123:0/4139743449 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f3000001670 con 0x7f30140cc5d0 2024-04-24T05:45:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 -- 172.21.15.123:0/4139743449 >> v1:172.21.15.123:6789/0 conn(0x7f30140cc5d0 legacy=0x7f30140acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 --2- 172.21.15.123:0/4139743449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ffc0039e0 0x7f2ffc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 --2- 172.21.15.123:0/4139743449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f301415c2f0 0x7f2ffc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 --2- 172.21.15.123:0/4139743449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ffc004d40 0x7f2ffc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 -- 172.21.15.123:0/4139743449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f301406bd30 con 0x7f2ffc0039e0 2024-04-24T05:45:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 -- 172.21.15.123:0/4139743449 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00baf0 con 0x7f2ffc004d40 2024-04-24T05:45:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.525+0000 7f3013e00640 1 -- 172.21.15.123:0/4139743449 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2ffc00bcf0 con 0x7f301415c2f0 2024-04-24T05:45:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.526+0000 7f3019400640 1 --2- 172.21.15.123:0/4139743449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ffc0039e0 0x7f2ffc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.526+0000 7f3019400640 1 -- 172.21.15.123:0/4139743449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f301415c2f0 msgr2=0x7f2ffc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:56.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.526+0000 7f3019400640 1 --2- 172.21.15.123:0/4139743449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f301415c2f0 0x7f2ffc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.526+0000 7f3019400640 1 -- 172.21.15.123:0/4139743449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ffc004d40 msgr2=0x7f2ffc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:56.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.526+0000 7f3019e00640 1 --2- 172.21.15.123:0/4139743449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ffc004d40 0x7f2ffc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.526+0000 7f3019400640 1 --2- 172.21.15.123:0/4139743449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ffc004d40 0x7f2ffc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.526+0000 7f3019400640 1 -- 172.21.15.123:0/4139743449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc003520 con 0x7f2ffc0039e0 2024-04-24T05:45:56.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.527+0000 7f3019400640 1 --2- 172.21.15.123:0/4139743449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ffc0039e0 0x7f2ffc003e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3008009ac0 tx=0x7f2ffc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1db4535fdf9653 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:56.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.528+0000 7f3013e00640 1 -- 172.21.15.123:0/4139743449 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008036070 con 0x7f2ffc0039e0 2024-04-24T05:45:56.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.528+0000 7f3013e00640 1 -- 172.21.15.123:0/4139743449 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3008031400 con 0x7f2ffc0039e0 2024-04-24T05:45:56.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.528+0000 7f3013e00640 1 -- 172.21.15.123:0/4139743449 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008009690 con 0x7f2ffc0039e0 2024-04-24T05:45:56.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.528+0000 7f301a800640 1 -- 172.21.15.123:0/4139743449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ffc0039e0 msgr2=0x7f2ffc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.528+0000 7f301a800640 1 --2- 172.21.15.123:0/4139743449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ffc0039e0 0x7f2ffc003e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3008009ac0 tx=0x7f2ffc005830 comp rx=0 tx=0).stop 2024-04-24T05:45:56.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.528+0000 7f3019e00640 1 -- 172.21.15.123:0/4139743449 reap_dead start 2024-04-24T05:45:56.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.528+0000 7f301a800640 1 -- 172.21.15.123:0/4139743449 shutdown_connections 2024-04-24T05:45:56.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.528+0000 7f301a800640 1 -- 172.21.15.123:0/4139743449 >> 172.21.15.123:0/4139743449 conn(0x7f30140b1cf0 msgr2=0x7f30140b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:56.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.528+0000 7f301a800640 1 -- 172.21.15.123:0/4139743449 shutdown_connections 2024-04-24T05:45:56.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.528+0000 7f301a800640 1 -- 172.21.15.123:0/4139743449 wait complete. 2024-04-24T05:45:56.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.530+0000 7f301a800640 1 Processor -- start 2024-04-24T05:45:56.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.530+0000 7f301a800640 1 -- start start 2024-04-24T05:45:56.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.531+0000 7f301a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ffc0039e0 0x7f30140cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.531+0000 7f301a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301415e550 0x7f30141568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.531+0000 7f301a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30140afe00 0x7f30140b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.531+0000 7f301a800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f301415c850 con 0x7f301415e550 2024-04-24T05:45:56.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.531+0000 7f301a800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f301415c6d0 con 0x7f30140afe00 2024-04-24T05:45:56.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.531+0000 7f301a800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3014141ce0 con 0x7f2ffc0039e0 2024-04-24T05:45:56.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.531+0000 7f3018a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301415e550 0x7f30141568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.531+0000 7f3018a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301415e550 0x7f30141568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52724/0 (socket says 172.21.15.123:52724) 2024-04-24T05:45:56.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.531+0000 7f3018a00640 1 -- 172.21.15.123:0/1691758191 learned_addr learned my addr 172.21.15.123:0/1691758191 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:56.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.532+0000 7f3019400640 1 --2- 172.21.15.123:0/1691758191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ffc0039e0 0x7f30140cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.532+0000 7f3018a00640 1 -- 172.21.15.123:0/1691758191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ffc0039e0 msgr2=0x7f30140cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.532+0000 7f3018a00640 1 --2- 172.21.15.123:0/1691758191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ffc0039e0 0x7f30140cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.532+0000 7f3019e00640 1 --2- 172.21.15.123:0/1691758191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30140afe00 0x7f30140b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.532+0000 7f3018a00640 1 -- 172.21.15.123:0/1691758191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30140afe00 msgr2=0x7f30140b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.532+0000 7f3018a00640 1 --2- 172.21.15.123:0/1691758191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30140afe00 0x7f30140b0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.532+0000 7f3018a00640 1 -- 172.21.15.123:0/1691758191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301415e9d0 con 0x7f301415e550 2024-04-24T05:45:56.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.532+0000 7f3018a00640 1 --2- 172.21.15.123:0/1691758191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301415e550 0x7f30141568b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f300000c7a0 tx=0x7f300000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:56.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.532+0000 7f3011600640 1 -- 172.21.15.123:0/1691758191 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000018070 con 0x7f301415e550 2024-04-24T05:45:56.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.532+0000 7f3011600640 1 -- 172.21.15.123:0/1691758191 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30000074e0 con 0x7f301415e550 2024-04-24T05:45:56.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.532+0000 7f3011600640 1 -- 172.21.15.123:0/1691758191 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000004a00 con 0x7f301415e550 2024-04-24T05:45:56.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.532+0000 7f301a800640 1 -- 172.21.15.123:0/1691758191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30140cd110 con 0x7f301415e550 2024-04-24T05:45:56.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.533+0000 7f301a800640 1 -- 172.21.15.123:0/1691758191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30140cd6d0 con 0x7f301415e550 2024-04-24T05:45:56.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.534+0000 7f301a800640 1 -- 172.21.15.123:0/1691758191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f301415bb10 con 0x7f301415e550 2024-04-24T05:45:56.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.535+0000 7f3011600640 1 -- 172.21.15.123:0/1691758191 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3000004ba0 con 0x7f301415e550 2024-04-24T05:45:56.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.536+0000 7f3011600640 1 --2- 172.21.15.123:0/1691758191 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2ff406c920 0x7f2ff406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.536+0000 7f3011600640 1 -- 172.21.15.123:0/1691758191 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f300008ad70 con 0x7f301415e550 2024-04-24T05:45:56.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.536+0000 7f3019400640 1 --2- 172.21.15.123:0/1691758191 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2ff406c920 0x7f2ff406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.537+0000 7f3019400640 1 --2- 172.21.15.123:0/1691758191 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2ff406c920 0x7f2ff406ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3008033000 tx=0x7f3008008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:56.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.540+0000 7f3011600640 1 -- 172.21.15.123:0/1691758191 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f301415bb10 con 0x7f301415e550 2024-04-24T05:45:56.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.738+0000 7f301a800640 1 -- 172.21.15.123:0/1691758191 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3014068a20 con 0x7f301415e550 2024-04-24T05:45:56.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.739+0000 7f3011600640 1 -- 172.21.15.123:0/1691758191 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f3000059450 con 0x7f301415e550 2024-04-24T05:45:56.741 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:45:56.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.742+0000 7f301a800640 1 -- 172.21.15.123:0/1691758191 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2ff406c920 msgr2=0x7f2ff406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.742+0000 7f301a800640 1 --2- 172.21.15.123:0/1691758191 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2ff406c920 0x7f2ff406ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3008033000 tx=0x7f3008008030 comp rx=0 tx=0).stop 2024-04-24T05:45:56.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.742+0000 7f301a800640 1 -- 172.21.15.123:0/1691758191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301415e550 msgr2=0x7f30141568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.742+0000 7f301a800640 1 --2- 172.21.15.123:0/1691758191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301415e550 0x7f30141568b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f300000c7a0 tx=0x7f300000cc60 comp rx=0 tx=0).stop 2024-04-24T05:45:56.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.742+0000 7f301a800640 1 -- 172.21.15.123:0/1691758191 shutdown_connections 2024-04-24T05:45:56.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.742+0000 7f301a800640 1 --2- 172.21.15.123:0/1691758191 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2ff406c920 0x7f2ff406ede0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.742+0000 7f301a800640 1 --2- 172.21.15.123:0/1691758191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30140afe00 0x7f30140b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.742+0000 7f301a800640 1 --2- 172.21.15.123:0/1691758191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301415e550 0x7f30141568b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.742+0000 7f301a800640 1 --2- 172.21.15.123:0/1691758191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ffc0039e0 0x7f30140cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.742+0000 7f301a800640 1 -- 172.21.15.123:0/1691758191 >> 172.21.15.123:0/1691758191 conn(0x7f30140b1cf0 msgr2=0x7f301406ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:56.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.742+0000 7f301a800640 1 -- 172.21.15.123:0/1691758191 shutdown_connections 2024-04-24T05:45:56.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.742+0000 7f301a800640 1 -- 172.21.15.123:0/1691758191 wait complete. 2024-04-24T05:45:56.762 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T05:45:56.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.957+0000 7f7cf3e00640 1 Processor -- start 2024-04-24T05:45:56.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.957+0000 7f7cf3e00640 1 -- start start 2024-04-24T05:45:56.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.959+0000 7f7cf3e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7cf40ad950 0x7f7cf406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.959+0000 7f7cf3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7cf40cf7b0 0x7f7cf406c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.959+0000 7f7cf3e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cf40d0de0 con 0x7f7cf406e660 2024-04-24T05:45:56.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.959+0000 7f7cf3e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7cf415ec00 con 0x7f7cf40cf7b0 2024-04-24T05:45:56.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.959+0000 7f7cf3e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7cf40ade50 con 0x7f7cf40ad950 2024-04-24T05:45:56.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.959+0000 7f7cf3400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7cf40cf7b0 0x7f7cf406c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.959+0000 7f7cf3400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7cf40cf7b0 0x7f7cf406c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52732/0 (socket says 172.21.15.123:52732) 2024-04-24T05:45:56.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.959+0000 7f7cf2a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7cf406e660 0x7f7cf406ba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:47370/0 (socket says 172.21.15.123:47370) 2024-04-24T05:45:56.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.959+0000 7f7cf3400640 1 -- 172.21.15.123:0/3777948574 learned_addr learned my addr 172.21.15.123:0/3777948574 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:56.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.959+0000 7f7cebe00640 1 --2- 172.21.15.123:0/3777948574 >> v2:172.21.15.155:3300/0 conn(0x7f7cf40ad950 0x7f7cf406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.960+0000 7f7cf3400640 1 -- 172.21.15.123:0/3777948574 >> v2:172.21.15.155:3300/0 conn(0x7f7cf40ad950 msgr2=0x7f7cf406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:56.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.960+0000 7f7cf3400640 1 --2- 172.21.15.123:0/3777948574 >> v2:172.21.15.155:3300/0 conn(0x7f7cf40ad950 0x7f7cf406ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.960+0000 7f7cf3400640 1 -- 172.21.15.123:0/3777948574 >> v1:172.21.15.123:6789/0 conn(0x7f7cf406e660 legacy=0x7f7cf406ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.960+0000 7f7cf3400640 1 -- 172.21.15.123:0/3777948574 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf406dc80 con 0x7f7cf40cf7b0 2024-04-24T05:45:56.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.960+0000 7f7cf2000640 1 -- 172.21.15.123:0/3777948574 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1500671539 0 0) 0x7f7cf40d0de0 con 0x7f7cf406e660 2024-04-24T05:45:56.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.961+0000 7f7cf3400640 1 --2- 172.21.15.123:0/3777948574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cf40cf7b0 0x7f7cf406c110 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7cf406fd20 tx=0x7f7cec02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba393a511d66a49 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:56.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.961+0000 7f7cf2000640 1 -- 172.21.15.123:0/3777948574 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec036070 con 0x7f7cf40cf7b0 2024-04-24T05:45:56.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.961+0000 7f7cf2000640 1 -- 172.21.15.123:0/3777948574 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cec031600 con 0x7f7cf40cf7b0 2024-04-24T05:45:56.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.961+0000 7f7cf2000640 1 -- 172.21.15.123:0/3777948574 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0319d0 con 0x7f7cf40cf7b0 2024-04-24T05:45:56.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.962+0000 7f7cf3e00640 1 -- 172.21.15.123:0/3777948574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cf40cf7b0 msgr2=0x7f7cf406c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.962+0000 7f7cf3e00640 1 --2- 172.21.15.123:0/3777948574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cf40cf7b0 0x7f7cf406c110 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7cf406fd20 tx=0x7f7cec02a730 comp rx=0 tx=0).stop 2024-04-24T05:45:56.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.962+0000 7f7cf3e00640 1 -- 172.21.15.123:0/3777948574 shutdown_connections 2024-04-24T05:45:56.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.962+0000 7f7cf3e00640 1 --2- 172.21.15.123:0/3777948574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cf40cf7b0 0x7f7cf406c110 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.962+0000 7f7cf3e00640 1 --2- 172.21.15.123:0/3777948574 >> v2:172.21.15.155:3300/0 conn(0x7f7cf40ad950 0x7f7cf406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.962+0000 7f7cf3e00640 1 -- 172.21.15.123:0/3777948574 >> 172.21.15.123:0/3777948574 conn(0x7f7cf40d0070 msgr2=0x7f7cf40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:56.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.962+0000 7f7cf3e00640 1 -- 172.21.15.123:0/3777948574 shutdown_connections 2024-04-24T05:45:56.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.962+0000 7f7cf3e00640 1 -- 172.21.15.123:0/3777948574 wait complete. 2024-04-24T05:45:56.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.964+0000 7f7cf3e00640 1 Processor -- start 2024-04-24T05:45:56.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.964+0000 7f7cf3e00640 1 -- start start 2024-04-24T05:45:56.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cf406e660 0x7f7cf41570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf40ad950 0x7f7cf40cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf3e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7cf40cf7b0 0x7f7cf409ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf409e0c0 con 0x7f7cf406e660 2024-04-24T05:45:56.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf3e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf409df40 con 0x7f7cf40ad950 2024-04-24T05:45:56.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf3e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf406c650 con 0x7f7cf40cf7b0 2024-04-24T05:45:56.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cf406e660 0x7f7cf41570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cf406e660 0x7f7cf41570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52734/0 (socket says 172.21.15.123:52734) 2024-04-24T05:45:56.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf2a00640 1 -- 172.21.15.123:0/4283767198 learned_addr learned my addr 172.21.15.123:0/4283767198 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:56.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf3400640 1 --2- 172.21.15.123:0/4283767198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7cf40cf7b0 0x7f7cf409ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cebe00640 1 --2- 172.21.15.123:0/4283767198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf40ad950 0x7f7cf40cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf2a00640 1 -- 172.21.15.123:0/4283767198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7cf40cf7b0 msgr2=0x7f7cf409ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf2a00640 1 --2- 172.21.15.123:0/4283767198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7cf40cf7b0 0x7f7cf409ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf2a00640 1 -- 172.21.15.123:0/4283767198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf40ad950 msgr2=0x7f7cf40cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:56.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf2a00640 1 --2- 172.21.15.123:0/4283767198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf40ad950 0x7f7cf40cbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:56.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.965+0000 7f7cf2a00640 1 -- 172.21.15.123:0/4283767198 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf41575f0 con 0x7f7cf406e660 2024-04-24T05:45:56.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.966+0000 7f7cf2a00640 1 --2- 172.21.15.123:0/4283767198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cf406e660 0x7f7cf41570b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400c4d0 tx=0x7f7ce400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:56.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.966+0000 7f7ceb400640 1 -- 172.21.15.123:0/4283767198 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4018070 con 0x7f7cf406e660 2024-04-24T05:45:56.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.966+0000 7f7ceb400640 1 -- 172.21.15.123:0/4283767198 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce4007500 con 0x7f7cf406e660 2024-04-24T05:45:56.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.966+0000 7f7cf3e00640 1 -- 172.21.15.123:0/4283767198 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf40cc0e0 con 0x7f7cf406e660 2024-04-24T05:45:56.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.966+0000 7f7ceb400640 1 -- 172.21.15.123:0/4283767198 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4004a20 con 0x7f7cf406e660 2024-04-24T05:45:56.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.966+0000 7f7cf3e00640 1 -- 172.21.15.123:0/4283767198 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf40cade0 con 0x7f7cf406e660 2024-04-24T05:45:56.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.967+0000 7f7cf3e00640 1 -- 172.21.15.123:0/4283767198 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cf4075b70 con 0x7f7cf406e660 2024-04-24T05:45:56.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.968+0000 7f7ceb400640 1 -- 172.21.15.123:0/4283767198 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ce401b070 con 0x7f7cf406e660 2024-04-24T05:45:56.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.969+0000 7f7ceb400640 1 --2- 172.21.15.123:0/4283767198 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7cd006c920 0x7f7cd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:56.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.969+0000 7f7ceb400640 1 -- 172.21.15.123:0/4283767198 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f7ce408aa60 con 0x7f7cf406e660 2024-04-24T05:45:56.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.969+0000 7f7cebe00640 1 --2- 172.21.15.123:0/4283767198 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7cd006c920 0x7f7cd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:56.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.970+0000 7f7cebe00640 1 --2- 172.21.15.123:0/4283767198 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7cd006c920 0x7f7cd006ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40b4fd0 tx=0x7f7cdc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:56.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:56.974+0000 7f7ceb400640 1 -- 172.21.15.123:0/4283767198 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ce4059270 con 0x7f7cf406e660 2024-04-24T05:45:57.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.175+0000 7f7cf3e00640 1 -- 172.21.15.123:0/4283767198 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7cf4073960 con 0x7f7cf406e660 2024-04-24T05:45:57.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.176+0000 7f7ceb400640 1 -- 172.21.15.123:0/4283767198 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f7ce4058b70 con 0x7f7cf406e660 2024-04-24T05:45:57.177 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:45:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:[ 2024-04-24T05:45:57.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:45:57.178 INFO:teuthology.orchestra.run.smithi123.stdout: "poolnum": 1, 2024-04-24T05:45:57.178 INFO:teuthology.orchestra.run.smithi123.stdout: "poolname": ".mgr" 2024-04-24T05:45:57.178 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:45:57.178 INFO:teuthology.orchestra.run.smithi123.stdout:] 2024-04-24T05:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.178+0000 7f7cf3e00640 1 -- 172.21.15.123:0/4283767198 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7cd006c920 msgr2=0x7f7cd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.178+0000 7f7cf3e00640 1 --2- 172.21.15.123:0/4283767198 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7cd006c920 0x7f7cd006ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40b4fd0 tx=0x7f7cdc008040 comp rx=0 tx=0).stop 2024-04-24T05:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.178+0000 7f7cf3e00640 1 -- 172.21.15.123:0/4283767198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cf406e660 msgr2=0x7f7cf41570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.178+0000 7f7cf3e00640 1 --2- 172.21.15.123:0/4283767198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cf406e660 0x7f7cf41570b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400c4d0 tx=0x7f7ce400c990 comp rx=0 tx=0).stop 2024-04-24T05:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.178+0000 7f7cf3e00640 1 -- 172.21.15.123:0/4283767198 shutdown_connections 2024-04-24T05:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.178+0000 7f7cf3e00640 1 --2- 172.21.15.123:0/4283767198 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7cd006c920 0x7f7cd006ede0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.178+0000 7f7cf3e00640 1 --2- 172.21.15.123:0/4283767198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7cf40cf7b0 0x7f7cf409ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:57.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.178+0000 7f7cf3e00640 1 --2- 172.21.15.123:0/4283767198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf40ad950 0x7f7cf40cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:57.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.178+0000 7f7cf3e00640 1 --2- 172.21.15.123:0/4283767198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cf406e660 0x7f7cf41570b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:57.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.178+0000 7f7cf3e00640 1 -- 172.21.15.123:0/4283767198 >> 172.21.15.123:0/4283767198 conn(0x7f7cf40d0070 msgr2=0x7f7cf406ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:57.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.179+0000 7f7cf3e00640 1 -- 172.21.15.123:0/4283767198 shutdown_connections 2024-04-24T05:45:57.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.179+0000 7f7cf3e00640 1 -- 172.21.15.123:0/4283767198 wait complete. 2024-04-24T05:45:57.198 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T05:45:57.198 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T05:45:57.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.401+0000 7f4c64c00640 1 Processor -- start 2024-04-24T05:45:57.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.402+0000 7f4c64c00640 1 -- start start 2024-04-24T05:45:57.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.403+0000 7f4c64c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c6015c620 0x7f4c60160a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:57.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.403+0000 7f4c64c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4c6006ba00 0x7f4c6006be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:57.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.403+0000 7f4c64c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c6015ffc0 con 0x7f4c6006f3d0 2024-04-24T05:45:57.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.403+0000 7f4c64c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4c6006c4a0 con 0x7f4c6006ba00 2024-04-24T05:45:57.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.403+0000 7f4c64c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6006c780 con 0x7f4c6015c620 2024-04-24T05:45:57.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.404+0000 7f4c57e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4c6006ba00 0x7f4c6006be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:57.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.404+0000 7f4c57e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4c6006ba00 0x7f4c6006be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52736/0 (socket says 172.21.15.123:52736) 2024-04-24T05:45:57.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.404+0000 7f4c57e00640 1 -- 172.21.15.123:0/2233023478 learned_addr learned my addr 172.21.15.123:0/2233023478 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:57.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.404+0000 7f4c5fe00640 1 --1- 172.21.15.123:0/2233023478 >> v1:172.21.15.123:6789/0 conn(0x7f4c6006f3d0 0x7f4c6006f820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:47378/0 (socket says 172.21.15.123:47378) 2024-04-24T05:45:57.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.404+0000 7f4c5f400640 1 --2- 172.21.15.123:0/2233023478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c6015c620 0x7f4c60160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:57.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.404+0000 7f4c57e00640 1 -- 172.21.15.123:0/2233023478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c6015c620 msgr2=0x7f4c60160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:57.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.404+0000 7f4c57e00640 1 --2- 172.21.15.123:0/2233023478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c6015c620 0x7f4c60160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:57.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.404+0000 7f4c57e00640 1 -- 172.21.15.123:0/2233023478 >> v1:172.21.15.123:6789/0 conn(0x7f4c6006f3d0 legacy=0x7f4c6006f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:57.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.404+0000 7f4c57e00640 1 -- 172.21.15.123:0/2233023478 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c60160280 con 0x7f4c6006ba00 2024-04-24T05:45:57.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.404+0000 7f4c5ea00640 1 -- 172.21.15.123:0/2233023478 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 916129864 0 0) 0x7f4c6015ffc0 con 0x7f4c6006f3d0 2024-04-24T05:45:57.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.405+0000 7f4c57e00640 1 --2- 172.21.15.123:0/2233023478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6006ba00 0x7f4c6006be40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009870 tx=0x7f4c4802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e27b3f9e89551a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:57.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.405+0000 7f4c5ea00640 1 -- 172.21.15.123:0/2233023478 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48037070 con 0x7f4c6006ba00 2024-04-24T05:45:57.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.405+0000 7f4c5ea00640 1 -- 172.21.15.123:0/2233023478 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c48032440 con 0x7f4c6006ba00 2024-04-24T05:45:57.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.405+0000 7f4c5ea00640 1 -- 172.21.15.123:0/2233023478 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48032810 con 0x7f4c6006ba00 2024-04-24T05:45:57.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.406+0000 7f4c64c00640 1 -- 172.21.15.123:0/2233023478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6006ba00 msgr2=0x7f4c6006be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:57.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.406+0000 7f4c64c00640 1 --2- 172.21.15.123:0/2233023478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6006ba00 0x7f4c6006be40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009870 tx=0x7f4c4802a250 comp rx=0 tx=0).stop 2024-04-24T05:45:57.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.406+0000 7f4c64c00640 1 -- 172.21.15.123:0/2233023478 shutdown_connections 2024-04-24T05:45:57.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.406+0000 7f4c64c00640 1 --2- 172.21.15.123:0/2233023478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6006ba00 0x7f4c6006be40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:57.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.406+0000 7f4c64c00640 1 --2- 172.21.15.123:0/2233023478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c6015c620 0x7f4c60160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:57.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.406+0000 7f4c64c00640 1 -- 172.21.15.123:0/2233023478 >> 172.21.15.123:0/2233023478 conn(0x7f4c600d00c0 msgr2=0x7f4c600d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:57.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.406+0000 7f4c64c00640 1 -- 172.21.15.123:0/2233023478 shutdown_connections 2024-04-24T05:45:57.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.406+0000 7f4c64c00640 1 -- 172.21.15.123:0/2233023478 wait complete. 2024-04-24T05:45:57.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.408+0000 7f4c64c00640 1 Processor -- start 2024-04-24T05:45:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.408+0000 7f4c64c00640 1 -- start start 2024-04-24T05:45:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.409+0000 7f4c64c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c6006ba00 0x7f4c60154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.409+0000 7f4c64c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6006f3d0 0x7f4c600b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.409+0000 7f4c64c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c6015c620 0x7f4c60153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.409+0000 7f4c64c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6015ca00 con 0x7f4c6006f3d0 2024-04-24T05:45:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.409+0000 7f4c64c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4c600ae4d0 con 0x7f4c6015c620 2024-04-24T05:45:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.409+0000 7f4c64c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6015cb80 con 0x7f4c6006ba00 2024-04-24T05:45:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.409+0000 7f4c57e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6006f3d0 0x7f4c600b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.409+0000 7f4c57e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6006f3d0 0x7f4c600b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52742/0 (socket says 172.21.15.123:52742) 2024-04-24T05:45:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.409+0000 7f4c57e00640 1 -- 172.21.15.123:0/2211117378 learned_addr learned my addr 172.21.15.123:0/2211117378 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.410+0000 7f4c57e00640 1 -- 172.21.15.123:0/2211117378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c6006ba00 msgr2=0x7f4c60154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.410+0000 7f4c57e00640 1 --2- 172.21.15.123:0/2211117378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c6006ba00 0x7f4c60154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.410+0000 7f4c57e00640 1 -- 172.21.15.123:0/2211117378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c6015c620 msgr2=0x7f4c60153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:57.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.410+0000 7f4c57e00640 1 --2- 172.21.15.123:0/2211117378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c6015c620 0x7f4c60153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:57.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.410+0000 7f4c57e00640 1 -- 172.21.15.123:0/2211117378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c600b35b0 con 0x7f4c6006f3d0 2024-04-24T05:45:57.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.410+0000 7f4c57e00640 1 --2- 172.21.15.123:0/2211117378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6006f3d0 0x7f4c600b2ef0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c48035040 tx=0x7f4c480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:57.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.410+0000 7f4c57400640 1 -- 172.21.15.123:0/2211117378 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48037040 con 0x7f4c6006f3d0 2024-04-24T05:45:57.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.410+0000 7f4c57400640 1 -- 172.21.15.123:0/2211117378 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c48004230 con 0x7f4c6006f3d0 2024-04-24T05:45:57.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.410+0000 7f4c57400640 1 -- 172.21.15.123:0/2211117378 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48003610 con 0x7f4c6006f3d0 2024-04-24T05:45:57.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.410+0000 7f4c64c00640 1 -- 172.21.15.123:0/2211117378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c600b3890 con 0x7f4c6006f3d0 2024-04-24T05:45:57.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.410+0000 7f4c64c00640 1 -- 172.21.15.123:0/2211117378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c600b3e50 con 0x7f4c6006f3d0 2024-04-24T05:45:57.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.411+0000 7f4c64c00640 1 -- 172.21.15.123:0/2211117378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c60075b70 con 0x7f4c6006f3d0 2024-04-24T05:45:57.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.412+0000 7f4c57400640 1 -- 172.21.15.123:0/2211117378 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c480037b0 con 0x7f4c6006f3d0 2024-04-24T05:45:57.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.413+0000 7f4c57400640 1 --2- 172.21.15.123:0/2211117378 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4c3c06c8f0 0x7f4c3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:57.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.413+0000 7f4c57400640 1 -- 172.21.15.123:0/2211117378 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f4c480ac640 con 0x7f4c6006f3d0 2024-04-24T05:45:57.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.414+0000 7f4c5f400640 1 --2- 172.21.15.123:0/2211117378 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4c3c06c8f0 0x7f4c3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:57.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.414+0000 7f4c5f400640 1 --2- 172.21.15.123:0/2211117378 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4c3c06c8f0 0x7f4c3c06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c50006fd0 tx=0x7f4c50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:57.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.418+0000 7f4c57400640 1 -- 172.21.15.123:0/2211117378 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c4807adb0 con 0x7f4c6006f3d0 2024-04-24T05:45:57.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:57.601+0000 7f4c64c00640 1 -- 172.21.15.123:0/2211117378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4c600adb00 con 0x7f4c6006f3d0 2024-04-24T05:45:58.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.156+0000 7f4c57400640 1 -- 172.21.15.123:0/2211117378 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f4c4807a6b0 con 0x7f4c6006f3d0 2024-04-24T05:45:58.158 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' created 2024-04-24T05:45:58.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.159+0000 7f4c64c00640 1 -- 172.21.15.123:0/2211117378 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4c3c06c8f0 msgr2=0x7f4c3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:58.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.159+0000 7f4c64c00640 1 --2- 172.21.15.123:0/2211117378 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4c3c06c8f0 0x7f4c3c06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c50006fd0 tx=0x7f4c50008040 comp rx=0 tx=0).stop 2024-04-24T05:45:58.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.159+0000 7f4c64c00640 1 -- 172.21.15.123:0/2211117378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6006f3d0 msgr2=0x7f4c600b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:58.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.159+0000 7f4c64c00640 1 --2- 172.21.15.123:0/2211117378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6006f3d0 0x7f4c600b2ef0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c48035040 tx=0x7f4c480046c0 comp rx=0 tx=0).stop 2024-04-24T05:45:58.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.160+0000 7f4c64c00640 1 -- 172.21.15.123:0/2211117378 shutdown_connections 2024-04-24T05:45:58.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.160+0000 7f4c64c00640 1 --2- 172.21.15.123:0/2211117378 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4c3c06c8f0 0x7f4c3c06edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:58.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.160+0000 7f4c64c00640 1 --2- 172.21.15.123:0/2211117378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c6015c620 0x7f4c60153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:58.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.160+0000 7f4c64c00640 1 --2- 172.21.15.123:0/2211117378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c6006f3d0 0x7f4c600b2ef0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:58.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.160+0000 7f4c64c00640 1 --2- 172.21.15.123:0/2211117378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c6006ba00 0x7f4c60154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:58.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.160+0000 7f4c64c00640 1 -- 172.21.15.123:0/2211117378 >> 172.21.15.123:0/2211117378 conn(0x7f4c600d00c0 msgr2=0x7f4c60073950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:58.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.160+0000 7f4c64c00640 1 -- 172.21.15.123:0/2211117378 shutdown_connections 2024-04-24T05:45:58.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.160+0000 7f4c64c00640 1 -- 172.21.15.123:0/2211117378 wait complete. 2024-04-24T05:45:58.192 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-24T05:45:58.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.384+0000 7f813ca00640 1 Processor -- start 2024-04-24T05:45:58.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.385+0000 7f813ca00640 1 -- start start 2024-04-24T05:45:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.386+0000 7f813ca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81380d3e10 0x7f81380d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.386+0000 7f813ca00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f81380d4c00 0x7f81380d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.386+0000 7f813ca00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81380cfdf0 con 0x7f813815e970 2024-04-24T05:45:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.386+0000 7f813ca00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f81380d09c0 con 0x7f81380d4c00 2024-04-24T05:45:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.386+0000 7f813ca00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f81380d0090 con 0x7f81380d3e10 2024-04-24T05:45:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.387+0000 7f8135400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81380d3e10 0x7f81380d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.387+0000 7f8135400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81380d3e10 0x7f81380d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:40162/0 (socket says 172.21.15.123:40162) 2024-04-24T05:45:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.387+0000 7f8135400640 1 -- 172.21.15.123:0/2566594892 learned_addr learned my addr 172.21.15.123:0/2566594892 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.387+0000 7f8136800640 1 --2- 172.21.15.123:0/2566594892 >> v2:172.21.15.155:3300/0 conn(0x7f81380d4c00 0x7f81380d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.388+0000 7f8134a00640 1 -- 172.21.15.123:0/2566594892 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1366265888 0 0) 0x7f81380cfdf0 con 0x7f813815e970 2024-04-24T05:45:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.388+0000 7f8134a00640 1 -- 172.21.15.123:0/2566594892 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f811c003520 con 0x7f813815e970 2024-04-24T05:45:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.388+0000 7f8135400640 1 -- 172.21.15.123:0/2566594892 >> v2:172.21.15.155:3300/0 conn(0x7f81380d4c00 msgr2=0x7f81380d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.388+0000 7f8135400640 1 --2- 172.21.15.123:0/2566594892 >> v2:172.21.15.155:3300/0 conn(0x7f81380d4c00 0x7f81380d1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.388+0000 7f8135400640 1 -- 172.21.15.123:0/2566594892 >> v1:172.21.15.155:6789/0 conn(0x7f813815e970 legacy=0x7f81380d44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.388+0000 7f8135400640 1 -- 172.21.15.123:0/2566594892 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81380d0370 con 0x7f81380d3e10 2024-04-24T05:45:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.389+0000 7f8135400640 1 --2- 172.21.15.123:0/2566594892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81380d3e10 0x7f81380d0e50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8124009870 tx=0x7f812402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea7781f3ee0f2f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:58.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.390+0000 7f8134a00640 1 -- 172.21.15.123:0/2566594892 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124037070 con 0x7f81380d3e10 2024-04-24T05:45:58.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.390+0000 7f8134a00640 1 -- 172.21.15.123:0/2566594892 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8124032440 con 0x7f81380d3e10 2024-04-24T05:45:58.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.390+0000 7f8134a00640 1 -- 172.21.15.123:0/2566594892 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124032810 con 0x7f81380d3e10 2024-04-24T05:45:58.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.391+0000 7f813ca00640 1 -- 172.21.15.123:0/2566594892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81380d3e10 msgr2=0x7f81380d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:58.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.391+0000 7f813ca00640 1 --2- 172.21.15.123:0/2566594892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81380d3e10 0x7f81380d0e50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8124009870 tx=0x7f812402a1c0 comp rx=0 tx=0).stop 2024-04-24T05:45:58.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.391+0000 7f813ca00640 1 -- 172.21.15.123:0/2566594892 shutdown_connections 2024-04-24T05:45:58.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.391+0000 7f813ca00640 1 --2- 172.21.15.123:0/2566594892 >> v2:172.21.15.155:3300/0 conn(0x7f81380d4c00 0x7f81380d1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:58.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.391+0000 7f813ca00640 1 --2- 172.21.15.123:0/2566594892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81380d3e10 0x7f81380d0e50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:58.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.391+0000 7f813ca00640 1 -- 172.21.15.123:0/2566594892 >> 172.21.15.123:0/2566594892 conn(0x7f81380b9a40 msgr2=0x7f81380b75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:58.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.391+0000 7f813ca00640 1 -- 172.21.15.123:0/2566594892 shutdown_connections 2024-04-24T05:45:58.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.391+0000 7f813ca00640 1 -- 172.21.15.123:0/2566594892 wait complete. 2024-04-24T05:45:58.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.393+0000 7f813ca00640 1 Processor -- start 2024-04-24T05:45:58.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.393+0000 7f813ca00640 1 -- start start 2024-04-24T05:45:58.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.394+0000 7f813ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81380d3e10 0x7f81380ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:58.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.395+0000 7f813ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81380d4c00 0x7f813806bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:58.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.395+0000 7f813ca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f813815e970 0x7f813806c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:58.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.395+0000 7f813ca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81380b12b0 con 0x7f81380d4c00 2024-04-24T05:45:58.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.395+0000 7f813ca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f813815ed50 con 0x7f81380d3e10 2024-04-24T05:45:58.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.395+0000 7f813ca00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f81380d53a0 con 0x7f813815e970 2024-04-24T05:45:58.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.395+0000 7f8135400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81380d4c00 0x7f813806bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:58.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.395+0000 7f8135400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81380d4c00 0x7f813806bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52754/0 (socket says 172.21.15.123:52754) 2024-04-24T05:45:58.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.395+0000 7f8135400640 1 -- 172.21.15.123:0/3949979315 learned_addr learned my addr 172.21.15.123:0/3949979315 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:58.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.395+0000 7f8136800640 1 --2- 172.21.15.123:0/3949979315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f813815e970 0x7f813806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:58.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.395+0000 7f8135e00640 1 --2- 172.21.15.123:0/3949979315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81380d3e10 0x7f81380ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:58.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.396+0000 7f8135400640 1 -- 172.21.15.123:0/3949979315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f813815e970 msgr2=0x7f813806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:58.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.396+0000 7f8135400640 1 --2- 172.21.15.123:0/3949979315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f813815e970 0x7f813806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:58.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.396+0000 7f8135400640 1 -- 172.21.15.123:0/3949979315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81380d3e10 msgr2=0x7f81380ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:58.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.396+0000 7f8135400640 1 --2- 172.21.15.123:0/3949979315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81380d3e10 0x7f81380ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:58.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.396+0000 7f8135400640 1 -- 172.21.15.123:0/3949979315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81380ccfe0 con 0x7f81380d4c00 2024-04-24T05:45:58.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.396+0000 7f8135400640 1 --2- 172.21.15.123:0/3949979315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81380d4c00 0x7f813806bc30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8124000c00 tx=0x7f81240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:58.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.396+0000 7f812e000640 1 -- 172.21.15.123:0/3949979315 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124041020 con 0x7f81380d4c00 2024-04-24T05:45:58.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.397+0000 7f812e000640 1 -- 172.21.15.123:0/3949979315 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f812403d6e0 con 0x7f81380d4c00 2024-04-24T05:45:58.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.397+0000 7f812e000640 1 -- 172.21.15.123:0/3949979315 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124037040 con 0x7f81380d4c00 2024-04-24T05:45:58.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.397+0000 7f813ca00640 1 -- 172.21.15.123:0/3949979315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81380cd2c0 con 0x7f81380d4c00 2024-04-24T05:45:58.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.397+0000 7f813ca00640 1 -- 172.21.15.123:0/3949979315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81380cd880 con 0x7f81380d4c00 2024-04-24T05:45:58.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.398+0000 7f813ca00640 1 -- 172.21.15.123:0/3949979315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8138068a20 con 0x7f81380d4c00 2024-04-24T05:45:58.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.399+0000 7f812e000640 1 -- 172.21.15.123:0/3949979315 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8124044020 con 0x7f81380d4c00 2024-04-24T05:45:58.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.400+0000 7f812e000640 1 --2- 172.21.15.123:0/3949979315 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f811806c920 0x7f811806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:58.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.400+0000 7f812e000640 1 -- 172.21.15.123:0/3949979315 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f81240abf80 con 0x7f81380d4c00 2024-04-24T05:45:58.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.401+0000 7f8135e00640 1 --2- 172.21.15.123:0/3949979315 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f811806c920 0x7f811806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:58.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.401+0000 7f8135e00640 1 --2- 172.21.15.123:0/3949979315 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f811806c920 0x7f811806ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8128000f60 tx=0x7f8128009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:58.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.406+0000 7f812e000640 1 -- 172.21.15.123:0/3949979315 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f812404b030 con 0x7f81380d4c00 2024-04-24T05:45:58.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:58.610+0000 7f813ca00640 1 -- 172.21.15.123:0/3949979315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f81380cb4a0 con 0x7f81380d4c00 2024-04-24T05:45:59.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.170+0000 7f812e000640 1 -- 172.21.15.123:0/3949979315 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7f812407a590 con 0x7f81380d4c00 2024-04-24T05:45:59.172 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' created 2024-04-24T05:45:59.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.175+0000 7f813ca00640 1 -- 172.21.15.123:0/3949979315 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f811806c920 msgr2=0x7f811806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:59.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.175+0000 7f813ca00640 1 --2- 172.21.15.123:0/3949979315 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f811806c920 0x7f811806ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8128000f60 tx=0x7f8128009040 comp rx=0 tx=0).stop 2024-04-24T05:45:59.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.175+0000 7f813ca00640 1 -- 172.21.15.123:0/3949979315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81380d4c00 msgr2=0x7f813806bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:59.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.175+0000 7f813ca00640 1 --2- 172.21.15.123:0/3949979315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81380d4c00 0x7f813806bc30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8124000c00 tx=0x7f81240046c0 comp rx=0 tx=0).stop 2024-04-24T05:45:59.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.176+0000 7f813ca00640 1 -- 172.21.15.123:0/3949979315 shutdown_connections 2024-04-24T05:45:59.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.176+0000 7f813ca00640 1 --2- 172.21.15.123:0/3949979315 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f811806c920 0x7f811806ede0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:59.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.176+0000 7f813ca00640 1 --2- 172.21.15.123:0/3949979315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f813815e970 0x7f813806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:59.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.176+0000 7f813ca00640 1 --2- 172.21.15.123:0/3949979315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81380d4c00 0x7f813806bc30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:59.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.176+0000 7f813ca00640 1 --2- 172.21.15.123:0/3949979315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81380d3e10 0x7f81380ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:59.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.176+0000 7f813ca00640 1 -- 172.21.15.123:0/3949979315 >> 172.21.15.123:0/3949979315 conn(0x7f81380b9a40 msgr2=0x7f813815f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:59.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.176+0000 7f813ca00640 1 -- 172.21.15.123:0/3949979315 shutdown_connections 2024-04-24T05:45:59.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.176+0000 7f813ca00640 1 -- 172.21.15.123:0/3949979315 wait complete. 2024-04-24T05:45:59.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T05:45:59.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.391+0000 7f8982200640 1 Processor -- start 2024-04-24T05:45:59.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.391+0000 7f8982200640 1 -- start start 2024-04-24T05:45:59.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.392+0000 7f8982200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c158e00 0x7f897c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:59.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.392+0000 7f8982200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f897c06ce30 0x7f897c06dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:59.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.392+0000 7f8982200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897c06e600 con 0x7f897c06e1f0 2024-04-24T05:45:59.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.392+0000 7f8982200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f897c06f590 con 0x7f897c06ce30 2024-04-24T05:45:59.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.392+0000 7f8982200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f897c06c460 con 0x7f897c158e00 2024-04-24T05:45:59.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.393+0000 7f8973e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f897c06ce30 0x7f897c06dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:59.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.393+0000 7f8973e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f897c06ce30 0x7f897c06dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55404/0 (socket says 172.21.15.123:55404) 2024-04-24T05:45:59.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.393+0000 7f8973e00640 1 -- 172.21.15.123:0/1611460560 learned_addr learned my addr 172.21.15.123:0/1611460560 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:59.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.393+0000 7f897b400640 1 --2- 172.21.15.123:0/1611460560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c158e00 0x7f897c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:59.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.393+0000 7f8973e00640 1 -- 172.21.15.123:0/1611460560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c158e00 msgr2=0x7f897c06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:45:59.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.393+0000 7f8973e00640 1 --2- 172.21.15.123:0/1611460560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c158e00 0x7f897c06c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:59.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.393+0000 7f8973e00640 1 -- 172.21.15.123:0/1611460560 >> v1:172.21.15.155:6789/0 conn(0x7f897c06e1f0 legacy=0x7f897c06e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:59.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.393+0000 7f8973e00640 1 -- 172.21.15.123:0/1611460560 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c06bb30 con 0x7f897c06ce30 2024-04-24T05:45:59.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.394+0000 7f8973e00640 1 --2- 172.21.15.123:0/1611460560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c06ce30 0x7f897c06dcb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f897c06f710 tx=0x7f896402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadb202124cc41a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:59.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.394+0000 7f897aa00640 1 -- 172.21.15.123:0/1611460560 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964037070 con 0x7f897c06ce30 2024-04-24T05:45:59.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.394+0000 7f897aa00640 1 -- 172.21.15.123:0/1611460560 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8964032440 con 0x7f897c06ce30 2024-04-24T05:45:59.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.394+0000 7f897aa00640 1 -- 172.21.15.123:0/1611460560 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964032810 con 0x7f897c06ce30 2024-04-24T05:45:59.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.395+0000 7f8982200640 1 -- 172.21.15.123:0/1611460560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c06ce30 msgr2=0x7f897c06dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:59.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.395+0000 7f8982200640 1 --2- 172.21.15.123:0/1611460560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c06ce30 0x7f897c06dcb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f897c06f710 tx=0x7f896402a1c0 comp rx=0 tx=0).stop 2024-04-24T05:45:59.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.395+0000 7f8982200640 1 -- 172.21.15.123:0/1611460560 shutdown_connections 2024-04-24T05:45:59.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.395+0000 7f8982200640 1 --2- 172.21.15.123:0/1611460560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c06ce30 0x7f897c06dcb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:59.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.395+0000 7f8982200640 1 --2- 172.21.15.123:0/1611460560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c158e00 0x7f897c06c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:59.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.395+0000 7f8982200640 1 -- 172.21.15.123:0/1611460560 >> 172.21.15.123:0/1611460560 conn(0x7f897c0b51f0 msgr2=0x7f897c0b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:45:59.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.395+0000 7f8982200640 1 -- 172.21.15.123:0/1611460560 shutdown_connections 2024-04-24T05:45:59.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.395+0000 7f8982200640 1 -- 172.21.15.123:0/1611460560 wait complete. 2024-04-24T05:45:59.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.397+0000 7f8982200640 1 Processor -- start 2024-04-24T05:45:59.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.397+0000 7f8982200640 1 -- start start 2024-04-24T05:45:59.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f8982200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c06ce30 0x7f897c1617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:59.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f8982200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c06e1f0 0x7f897c0994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:59.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f8982200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c158e00 0x7f897c0b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:59.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f8982200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f897c1591e0 con 0x7f897c158e00 2024-04-24T05:45:59.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f8982200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f897c06c0f0 con 0x7f897c06e1f0 2024-04-24T05:45:59.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f8982200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f897c0af210 con 0x7f897c06ce30 2024-04-24T05:45:59.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f897be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c158e00 0x7f897c0b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:59.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f897be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c158e00 0x7f897c0b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55412/0 (socket says 172.21.15.123:55412) 2024-04-24T05:45:59.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f897be00640 1 -- 172.21.15.123:0/2956167385 learned_addr learned my addr 172.21.15.123:0/2956167385 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:45:59.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f8973e00640 1 --2- 172.21.15.123:0/2956167385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c06e1f0 0x7f897c0994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:59.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f897b400640 1 --2- 172.21.15.123:0/2956167385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c06ce30 0x7f897c1617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:59.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f897be00640 1 -- 172.21.15.123:0/2956167385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c06ce30 msgr2=0x7f897c1617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:59.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f897be00640 1 --2- 172.21.15.123:0/2956167385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c06ce30 0x7f897c1617c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:59.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f897be00640 1 -- 172.21.15.123:0/2956167385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c06e1f0 msgr2=0x7f897c0994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:45:59.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f897be00640 1 --2- 172.21.15.123:0/2956167385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c06e1f0 0x7f897c0994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:45:59.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.398+0000 7f897be00640 1 -- 172.21.15.123:0/2956167385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c0ae600 con 0x7f897c158e00 2024-04-24T05:45:59.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.399+0000 7f897be00640 1 --2- 172.21.15.123:0/2956167385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c158e00 0x7f897c0b0bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f897400c7d0 tx=0x7f897400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:59.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.399+0000 7f8973400640 1 -- 172.21.15.123:0/2956167385 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974003b70 con 0x7f897c158e00 2024-04-24T05:45:59.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.399+0000 7f8973400640 1 -- 172.21.15.123:0/2956167385 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8974003d30 con 0x7f897c158e00 2024-04-24T05:45:59.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.399+0000 7f8982200640 1 -- 172.21.15.123:0/2956167385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897c0ae8e0 con 0x7f897c158e00 2024-04-24T05:45:59.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.399+0000 7f8973400640 1 -- 172.21.15.123:0/2956167385 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897400aa90 con 0x7f897c158e00 2024-04-24T05:45:59.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.399+0000 7f8982200640 1 -- 172.21.15.123:0/2956167385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897c0aeea0 con 0x7f897c158e00 2024-04-24T05:45:59.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.400+0000 7f8982200640 1 -- 172.21.15.123:0/2956167385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897c068a20 con 0x7f897c158e00 2024-04-24T05:45:59.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.401+0000 7f8973400640 1 -- 172.21.15.123:0/2956167385 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f897400ac30 con 0x7f897c158e00 2024-04-24T05:45:59.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.403+0000 7f8973400640 1 --2- 172.21.15.123:0/2956167385 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f895806c920 0x7f895806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:45:59.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.403+0000 7f8973400640 1 -- 172.21.15.123:0/2956167385 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f897408bbc0 con 0x7f897c158e00 2024-04-24T05:45:59.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.403+0000 7f897b400640 1 --2- 172.21.15.123:0/2956167385 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f895806c920 0x7f895806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:45:59.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.404+0000 7f897b400640 1 --2- 172.21.15.123:0/2956167385 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f895806c920 0x7f895806ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f896c006fd0 tx=0x7f896c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:45:59.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.408+0000 7f8973400640 1 -- 172.21.15.123:0/2956167385 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f897c06e600 con 0x7f897c158e00 2024-04-24T05:45:59.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:45:59.637+0000 7f8982200640 1 -- 172.21.15.123:0/2956167385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f897c071b50 con 0x7f897c158e00 2024-04-24T05:46:00.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.194+0000 7f8973400640 1 -- 172.21.15.123:0/2956167385 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f8974059f20 con 0x7f897c158e00 2024-04-24T05:46:00.197 INFO:teuthology.orchestra.run.smithi123.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T05:46:00.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.204+0000 7f8982200640 1 -- 172.21.15.123:0/2956167385 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f895806c920 msgr2=0x7f895806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:00.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.204+0000 7f8982200640 1 --2- 172.21.15.123:0/2956167385 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f895806c920 0x7f895806ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f896c006fd0 tx=0x7f896c008040 comp rx=0 tx=0).stop 2024-04-24T05:46:00.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.204+0000 7f8982200640 1 -- 172.21.15.123:0/2956167385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c158e00 msgr2=0x7f897c0b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:00.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.204+0000 7f8982200640 1 --2- 172.21.15.123:0/2956167385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c158e00 0x7f897c0b0bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f897400c7d0 tx=0x7f897400cc90 comp rx=0 tx=0).stop 2024-04-24T05:46:00.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.204+0000 7f8982200640 1 -- 172.21.15.123:0/2956167385 shutdown_connections 2024-04-24T05:46:00.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.204+0000 7f8982200640 1 --2- 172.21.15.123:0/2956167385 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f895806c920 0x7f895806ede0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:00.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.204+0000 7f8982200640 1 --2- 172.21.15.123:0/2956167385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f897c158e00 0x7f897c0b0bd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:00.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.204+0000 7f8982200640 1 --2- 172.21.15.123:0/2956167385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c06e1f0 0x7f897c0994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:00.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.204+0000 7f8982200640 1 --2- 172.21.15.123:0/2956167385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c06ce30 0x7f897c1617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:00.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.204+0000 7f8982200640 1 -- 172.21.15.123:0/2956167385 >> 172.21.15.123:0/2956167385 conn(0x7f897c0b51f0 msgr2=0x7f897c072e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:00.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.206+0000 7f8982200640 1 -- 172.21.15.123:0/2956167385 shutdown_connections 2024-04-24T05:46:00.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.206+0000 7f8982200640 1 -- 172.21.15.123:0/2956167385 wait complete. 2024-04-24T05:46:00.246 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-24T05:46:00.247 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-24T05:46:00.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.413+0000 7f9407e00640 1 Processor -- start 2024-04-24T05:46:00.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.413+0000 7f9407e00640 1 -- start start 2024-04-24T05:46:00.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.415+0000 7f9407e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f940815e680 0x7f94080d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:00.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.415+0000 7f9407e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f94080d4a90 0x7f94080d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:00.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.415+0000 7f9407e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94080cfd30 con 0x7f940815ea60 2024-04-24T05:46:00.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.415+0000 7f9407e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f94080d0010 con 0x7f94080d4a90 2024-04-24T05:46:00.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.415+0000 7f9407e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f94080d02f0 con 0x7f940815e680 2024-04-24T05:46:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.416+0000 7f9406a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f940815e680 0x7f94080d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.416+0000 7f9406a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f940815e680 0x7f94080d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:33720/0 (socket says 172.21.15.123:33720) 2024-04-24T05:46:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.416+0000 7f9406a00640 1 -- 172.21.15.123:0/1631606834 learned_addr learned my addr 172.21.15.123:0/1631606834 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.416+0000 7f9407400640 1 --2- 172.21.15.123:0/1631606834 >> v2:172.21.15.155:3300/0 conn(0x7f94080d4a90 0x7f94080d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.416+0000 7f9405600640 1 -- 172.21.15.123:0/1631606834 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3158658032 0 0) 0x7f94080cfd30 con 0x7f940815ea60 2024-04-24T05:46:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.416+0000 7f9405600640 1 -- 172.21.15.123:0/1631606834 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93ec003520 con 0x7f940815ea60 2024-04-24T05:46:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.416+0000 7f9406a00640 1 -- 172.21.15.123:0/1631606834 >> v2:172.21.15.155:3300/0 conn(0x7f94080d4a90 msgr2=0x7f94080d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:00.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.417+0000 7f9406a00640 1 --2- 172.21.15.123:0/1631606834 >> v2:172.21.15.155:3300/0 conn(0x7f94080d4a90 0x7f94080d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.417+0000 7f9406a00640 1 -- 172.21.15.123:0/1631606834 >> v1:172.21.15.155:6789/0 conn(0x7f940815ea60 legacy=0x7f94080d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.417+0000 7f9406a00640 1 -- 172.21.15.123:0/1631606834 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94080d05d0 con 0x7f940815e680 2024-04-24T05:46:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.417+0000 7f9407400640 1 --2- 172.21.15.123:0/1631606834 >> v2:172.21.15.155:3300/0 conn(0x7f94080d4a90 0x7f94080d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:46:00.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.418+0000 7f9406a00640 1 --2- 172.21.15.123:0/1631606834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f940815e680 0x7f94080d13e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009870 tx=0x7f93f802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73b7fb0e98a8bca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:00.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.418+0000 7f9405600640 1 -- 172.21.15.123:0/1631606834 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8037070 con 0x7f940815e680 2024-04-24T05:46:00.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.418+0000 7f9405600640 1 -- 172.21.15.123:0/1631606834 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f8032440 con 0x7f940815e680 2024-04-24T05:46:00.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.418+0000 7f9405600640 1 -- 172.21.15.123:0/1631606834 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f80327f0 con 0x7f940815e680 2024-04-24T05:46:00.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.419+0000 7f9407e00640 1 -- 172.21.15.123:0/1631606834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f940815e680 msgr2=0x7f94080d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.419+0000 7f9407e00640 1 --2- 172.21.15.123:0/1631606834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f940815e680 0x7f94080d13e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009870 tx=0x7f93f802a250 comp rx=0 tx=0).stop 2024-04-24T05:46:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.419+0000 7f9407e00640 1 -- 172.21.15.123:0/1631606834 shutdown_connections 2024-04-24T05:46:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.419+0000 7f9407e00640 1 --2- 172.21.15.123:0/1631606834 >> v2:172.21.15.155:3300/0 conn(0x7f94080d4a90 0x7f94080d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.419+0000 7f9407e00640 1 --2- 172.21.15.123:0/1631606834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f940815e680 0x7f94080d13e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.419+0000 7f9407e00640 1 -- 172.21.15.123:0/1631606834 >> 172.21.15.123:0/1631606834 conn(0x7f94080b9cd0 msgr2=0x7f940809d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.419+0000 7f9407e00640 1 -- 172.21.15.123:0/1631606834 shutdown_connections 2024-04-24T05:46:00.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.419+0000 7f9407e00640 1 -- 172.21.15.123:0/1631606834 wait complete. 2024-04-24T05:46:00.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.421+0000 7f9407e00640 1 Processor -- start 2024-04-24T05:46:00.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.421+0000 7f9407e00640 1 -- start start 2024-04-24T05:46:00.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9407e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94080d4a90 0x7f94080b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:00.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9407e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940815e680 0x7f94081f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:00.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9407e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f940815ea60 0x7f940815d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9407e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94080d21e0 con 0x7f940815e680 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9407e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f94080d2060 con 0x7f940815ea60 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9407e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f94080d2360 con 0x7f94080d4a90 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9406000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940815e680 0x7f94081f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9406000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940815e680 0x7f94081f50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55422/0 (socket says 172.21.15.123:55422) 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9406000640 1 -- 172.21.15.123:0/2502512954 learned_addr learned my addr 172.21.15.123:0/2502512954 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9406a00640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94080d4a90 0x7f94080b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9407400640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f940815ea60 0x7f940815d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9406000640 1 -- 172.21.15.123:0/2502512954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94080d4a90 msgr2=0x7f94080b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9406000640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94080d4a90 0x7f94080b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9406000640 1 -- 172.21.15.123:0/2502512954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f940815ea60 msgr2=0x7f940815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9406000640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f940815ea60 0x7f940815d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9406000640 1 -- 172.21.15.123:0/2502512954 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94080ce2c0 con 0x7f940815e680 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9406a00640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94080d4a90 0x7f94080b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.422+0000 7f9407400640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f940815ea60 0x7f940815d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:00.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.423+0000 7f9406000640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940815e680 0x7f94081f50e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c730 tx=0x7f93f400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.423+0000 7f93ff400640 1 -- 172.21.15.123:0/2502512954 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4003b70 con 0x7f940815e680 2024-04-24T05:46:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.423+0000 7f93ff400640 1 -- 172.21.15.123:0/2502512954 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f4003d10 con 0x7f940815e680 2024-04-24T05:46:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.423+0000 7f93ff400640 1 -- 172.21.15.123:0/2502512954 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f400aa30 con 0x7f940815e680 2024-04-24T05:46:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.423+0000 7f9407e00640 1 -- 172.21.15.123:0/2502512954 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940806c9c0 con 0x7f940815e680 2024-04-24T05:46:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.423+0000 7f9407e00640 1 -- 172.21.15.123:0/2502512954 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94080b5bb0 con 0x7f940815e680 2024-04-24T05:46:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.424+0000 7f9407e00640 1 -- 172.21.15.123:0/2502512954 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9408075b70 con 0x7f940815e680 2024-04-24T05:46:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.425+0000 7f93ff400640 1 -- 172.21.15.123:0/2502512954 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93f400abd0 con 0x7f940815e680 2024-04-24T05:46:00.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.426+0000 7f93ff400640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f93e806c8f0 0x7f93e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:00.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.426+0000 7f93ff400640 1 -- 172.21.15.123:0/2502512954 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f93f408ba80 con 0x7f940815e680 2024-04-24T05:46:00.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.426+0000 7f9406a00640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f93e806c8f0 0x7f93e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:00.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.427+0000 7f9406a00640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f93e806c8f0 0x7f93e806edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f93f8034000 tx=0x7f93f802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:00.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.430+0000 7f93ff400640 1 -- 172.21.15.123:0/2502512954 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f94080ce2c0 con 0x7f940815e680 2024-04-24T05:46:00.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:00.639+0000 7f9407e00640 1 -- 172.21.15.123:0/2502512954 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f9408069500 con 0x7f940815e680 2024-04-24T05:46:01.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.205+0000 7f93ff400640 1 -- 172.21.15.123:0/2502512954 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7f93f4060080 con 0x7f940815e680 2024-04-24T05:46:01.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.208+0000 7f9407e00640 1 -- 172.21.15.123:0/2502512954 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f93e806c8f0 msgr2=0x7f93e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:01.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.208+0000 7f9407e00640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f93e806c8f0 0x7f93e806edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f93f8034000 tx=0x7f93f802e000 comp rx=0 tx=0).stop 2024-04-24T05:46:01.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.208+0000 7f9407e00640 1 -- 172.21.15.123:0/2502512954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940815e680 msgr2=0x7f94081f50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:01.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.208+0000 7f9407e00640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940815e680 0x7f94081f50e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c730 tx=0x7f93f400cbf0 comp rx=0 tx=0).stop 2024-04-24T05:46:01.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.208+0000 7f9407e00640 1 -- 172.21.15.123:0/2502512954 shutdown_connections 2024-04-24T05:46:01.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.208+0000 7f9407e00640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f93e806c8f0 0x7f93e806edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:01.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.208+0000 7f9407e00640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f940815ea60 0x7f940815d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:01.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.208+0000 7f9407e00640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940815e680 0x7f94081f50e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:01.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.208+0000 7f9407e00640 1 --2- 172.21.15.123:0/2502512954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94080d4a90 0x7f94080b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:01.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.208+0000 7f9407e00640 1 -- 172.21.15.123:0/2502512954 >> 172.21.15.123:0/2502512954 conn(0x7f94080b9cd0 msgr2=0x7f94080b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:01.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.209+0000 7f9407e00640 1 -- 172.21.15.123:0/2502512954 shutdown_connections 2024-04-24T05:46:01.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.209+0000 7f9407e00640 1 -- 172.21.15.123:0/2502512954 wait complete. 2024-04-24T05:46:01.228 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T05:46:01.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.413+0000 7fbeabe00640 1 Processor -- start 2024-04-24T05:46:01.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.413+0000 7fbeabe00640 1 -- start start 2024-04-24T05:46:01.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.414+0000 7fbeabe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbeac15c620 0x7fbeac160a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:01.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.414+0000 7fbeabe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbeac06fae0 0x7fbeac06be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:01.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.414+0000 7fbeabe00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeac15ffc0 con 0x7fbeac06ba00 2024-04-24T05:46:01.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.414+0000 7fbeabe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbeac06c4a0 con 0x7fbeac15c620 2024-04-24T05:46:01.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.414+0000 7fbeabe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbeac06c780 con 0x7fbeac06fae0 2024-04-24T05:46:01.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.415+0000 7fbeaa000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbeac06ba00 0x7fbeac06f3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60638/0 (socket says 172.21.15.123:60638) 2024-04-24T05:46:01.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.415+0000 7fbeaa000640 1 -- 172.21.15.123:0/606309833 learned_addr learned my addr 172.21.15.123:0/606309833 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:01.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.415+0000 7fbeaaa00640 1 --2- 172.21.15.123:0/606309833 >> v2:172.21.15.155:3300/0 conn(0x7fbeac15c620 0x7fbeac160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:01.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.415+0000 7fbeab400640 1 --2- 172.21.15.123:0/606309833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbeac06fae0 0x7fbeac06be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:01.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.415+0000 7fbea9600640 1 -- 172.21.15.123:0/606309833 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 824546944 0 0) 0x7fbeac15ffc0 con 0x7fbeac06ba00 2024-04-24T05:46:01.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.415+0000 7fbea9600640 1 -- 172.21.15.123:0/606309833 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe90003520 con 0x7fbeac06ba00 2024-04-24T05:46:01.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.415+0000 7fbea9600640 1 -- 172.21.15.123:0/606309833 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3270945786 0 0) 0x7fbe90003520 con 0x7fbeac06ba00 2024-04-24T05:46:01.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbea9600640 1 -- 172.21.15.123:0/606309833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbeac06fae0 msgr2=0x7fbeac06be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:01.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbea9600640 1 --2- 172.21.15.123:0/606309833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbeac06fae0 0x7fbeac06be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:01.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbea9600640 1 -- 172.21.15.123:0/606309833 >> v2:172.21.15.155:3300/0 conn(0x7fbeac15c620 msgr2=0x7fbeac160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:01.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbea9600640 1 --2- 172.21.15.123:0/606309833 >> v2:172.21.15.155:3300/0 conn(0x7fbeac15c620 0x7fbeac160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:01.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbea9600640 1 -- 172.21.15.123:0/606309833 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac160280 con 0x7fbeac06ba00 2024-04-24T05:46:01.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbea9600640 1 -- 172.21.15.123:0/606309833 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7fbe98001670 con 0x7fbeac06ba00 2024-04-24T05:46:01.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbeab400640 1 --2- 172.21.15.123:0/606309833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbeac06fae0 0x7fbeac06be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:46:01.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbea9600640 1 -- 172.21.15.123:0/606309833 >> v1:172.21.15.123:6789/0 conn(0x7fbeac06ba00 legacy=0x7fbeac06f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:01.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbea9600640 1 --2- 172.21.15.123:0/606309833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe900039b0 0x7fbe90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:01.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbea9600640 1 --2- 172.21.15.123:0/606309833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbeac06fae0 0x7fbe90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:01.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbea9600640 1 --2- 172.21.15.123:0/606309833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe90004d40 0x7fbe900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:01.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbea9600640 1 -- 172.21.15.123:0/606309833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac06c4a0 con 0x7fbe90004d40 2024-04-24T05:46:01.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbea9600640 1 -- 172.21.15.123:0/606309833 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac06c780 con 0x7fbe900039b0 2024-04-24T05:46:01.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbea9600640 1 -- 172.21.15.123:0/606309833 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbe9000b9c0 con 0x7fbeac06fae0 2024-04-24T05:46:01.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbeab400640 1 --2- 172.21.15.123:0/606309833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe90004d40 0x7fbe900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:01.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbeab400640 1 -- 172.21.15.123:0/606309833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbeac06fae0 msgr2=0x7fbe90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:01.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbeab400640 1 --2- 172.21.15.123:0/606309833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbeac06fae0 0x7fbe90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:01.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbeab400640 1 -- 172.21.15.123:0/606309833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe900039b0 msgr2=0x7fbe90003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:01.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbeab400640 1 --2- 172.21.15.123:0/606309833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe900039b0 0x7fbe90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:01.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.416+0000 7fbeab400640 1 -- 172.21.15.123:0/606309833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe90003520 con 0x7fbe90004d40 2024-04-24T05:46:01.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.417+0000 7fbeab400640 1 --2- 172.21.15.123:0/606309833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe90004d40 0x7fbe900051c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbea40073e0 tx=0x7fbe90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed176c4905326d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:01.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.418+0000 7fbea9600640 1 -- 172.21.15.123:0/606309833 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4009c80 con 0x7fbe90004d40 2024-04-24T05:46:01.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.418+0000 7fbea9600640 1 -- 172.21.15.123:0/606309833 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea402c650 con 0x7fbe90004d40 2024-04-24T05:46:01.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.418+0000 7fbea9600640 1 -- 172.21.15.123:0/606309833 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea402ca00 con 0x7fbe90004d40 2024-04-24T05:46:01.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.418+0000 7fbeabe00640 1 -- 172.21.15.123:0/606309833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe90004d40 msgr2=0x7fbe900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:01.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.418+0000 7fbeabe00640 1 --2- 172.21.15.123:0/606309833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe90004d40 0x7fbe900051c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbea40073e0 tx=0x7fbe90005830 comp rx=0 tx=0).stop 2024-04-24T05:46:01.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.418+0000 7fbeab400640 1 -- 172.21.15.123:0/606309833 reap_dead start 2024-04-24T05:46:01.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.418+0000 7fbeabe00640 1 -- 172.21.15.123:0/606309833 shutdown_connections 2024-04-24T05:46:01.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.418+0000 7fbeabe00640 1 -- 172.21.15.123:0/606309833 >> 172.21.15.123:0/606309833 conn(0x7fbeac0d00c0 msgr2=0x7fbeac0d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:01.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.418+0000 7fbeabe00640 1 -- 172.21.15.123:0/606309833 shutdown_connections 2024-04-24T05:46:01.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.418+0000 7fbeabe00640 1 -- 172.21.15.123:0/606309833 wait complete. 2024-04-24T05:46:01.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.420+0000 7fbeabe00640 1 Processor -- start 2024-04-24T05:46:01.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.420+0000 7fbeabe00640 1 -- start start 2024-04-24T05:46:01.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.421+0000 7fbeabe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe90004d40 0x7fbeac09cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:01.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.421+0000 7fbeabe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbeac0cdc90 0x7fbeac152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:01.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.421+0000 7fbeabe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeac0c8870 0x7fbeac0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:01.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.421+0000 7fbeabe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac06be70 con 0x7fbeac0c8870 2024-04-24T05:46:01.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.421+0000 7fbeabe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac0ae4d0 con 0x7fbe90004d40 2024-04-24T05:46:01.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.421+0000 7fbeabe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbeac06bff0 con 0x7fbeac0cdc90 2024-04-24T05:46:01.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.421+0000 7fbeab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeac0c8870 0x7fbeac0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:01.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.421+0000 7fbeab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeac0c8870 0x7fbeac0c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55448/0 (socket says 172.21.15.123:55448) 2024-04-24T05:46:01.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.421+0000 7fbeab400640 1 -- 172.21.15.123:0/2438483315 learned_addr learned my addr 172.21.15.123:0/2438483315 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:01.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.422+0000 7fbeab400640 1 -- 172.21.15.123:0/2438483315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbeac0cdc90 msgr2=0x7fbeac152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:01.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.422+0000 7fbeab400640 1 --2- 172.21.15.123:0/2438483315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbeac0cdc90 0x7fbeac152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:01.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.422+0000 7fbeab400640 1 -- 172.21.15.123:0/2438483315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe90004d40 msgr2=0x7fbeac09cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:01.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.422+0000 7fbeab400640 1 --2- 172.21.15.123:0/2438483315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe90004d40 0x7fbeac09cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:01.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.422+0000 7fbeab400640 1 -- 172.21.15.123:0/2438483315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac0cf290 con 0x7fbeac0c8870 2024-04-24T05:46:01.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.422+0000 7fbeab400640 1 --2- 172.21.15.123:0/2438483315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeac0c8870 0x7fbeac0c8cf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbea4007500 tx=0x7fbea4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:01.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.422+0000 7fbea2a00640 1 -- 172.21.15.123:0/2438483315 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4009c80 con 0x7fbeac0c8870 2024-04-24T05:46:01.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.422+0000 7fbea2a00640 1 -- 172.21.15.123:0/2438483315 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea402c450 con 0x7fbeac0c8870 2024-04-24T05:46:01.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.422+0000 7fbeabe00640 1 -- 172.21.15.123:0/2438483315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac0ce1c0 con 0x7fbeac0c8870 2024-04-24T05:46:01.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.422+0000 7fbea2a00640 1 -- 172.21.15.123:0/2438483315 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4003bb0 con 0x7fbeac0c8870 2024-04-24T05:46:01.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.422+0000 7fbeabe00640 1 -- 172.21.15.123:0/2438483315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac0ce780 con 0x7fbeac0c8870 2024-04-24T05:46:01.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.423+0000 7fbeabe00640 1 -- 172.21.15.123:0/2438483315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeac075b70 con 0x7fbeac0c8870 2024-04-24T05:46:01.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.424+0000 7fbea2a00640 1 -- 172.21.15.123:0/2438483315 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbea402e880 con 0x7fbeac0c8870 2024-04-24T05:46:01.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.425+0000 7fbea2a00640 1 --2- 172.21.15.123:0/2438483315 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbe8c06c8f0 0x7fbe8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:01.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.425+0000 7fbea2a00640 1 -- 172.21.15.123:0/2438483315 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7fbea40ac500 con 0x7fbeac0c8870 2024-04-24T05:46:01.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.425+0000 7fbeaaa00640 1 --2- 172.21.15.123:0/2438483315 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbe8c06c8f0 0x7fbe8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:01.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.426+0000 7fbeaaa00640 1 --2- 172.21.15.123:0/2438483315 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbe8c06c8f0 0x7fbe8c06edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c006a90 tx=0x7fbe9c006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:01.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.429+0000 7fbea2a00640 1 -- 172.21.15.123:0/2438483315 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbea4032030 con 0x7fbeac0c8870 2024-04-24T05:46:01.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:01.628+0000 7fbeabe00640 1 -- 172.21.15.123:0/2438483315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbeac0adb00 con 0x7fbeac0c8870 2024-04-24T05:46:03.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.222+0000 7fbea2a00640 1 -- 172.21.15.123:0/2438483315 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7fbea407a6d0 con 0x7fbeac0c8870 2024-04-24T05:46:03.224 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data_ec' created 2024-04-24T05:46:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.225+0000 7fbeabe00640 1 -- 172.21.15.123:0/2438483315 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbe8c06c8f0 msgr2=0x7fbe8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.225+0000 7fbeabe00640 1 --2- 172.21.15.123:0/2438483315 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbe8c06c8f0 0x7fbe8c06edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c006a90 tx=0x7fbe9c006a20 comp rx=0 tx=0).stop 2024-04-24T05:46:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.225+0000 7fbeabe00640 1 -- 172.21.15.123:0/2438483315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeac0c8870 msgr2=0x7fbeac0c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.225+0000 7fbeabe00640 1 --2- 172.21.15.123:0/2438483315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeac0c8870 0x7fbeac0c8cf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbea4007500 tx=0x7fbea4004ab0 comp rx=0 tx=0).stop 2024-04-24T05:46:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.226+0000 7fbeabe00640 1 -- 172.21.15.123:0/2438483315 shutdown_connections 2024-04-24T05:46:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.226+0000 7fbeabe00640 1 --2- 172.21.15.123:0/2438483315 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbe8c06c8f0 0x7fbe8c06edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:03.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.226+0000 7fbeabe00640 1 --2- 172.21.15.123:0/2438483315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeac0c8870 0x7fbeac0c8cf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:03.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.226+0000 7fbeabe00640 1 --2- 172.21.15.123:0/2438483315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbeac0cdc90 0x7fbeac152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:03.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.226+0000 7fbeabe00640 1 --2- 172.21.15.123:0/2438483315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe90004d40 0x7fbeac09cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:03.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.226+0000 7fbeabe00640 1 -- 172.21.15.123:0/2438483315 >> 172.21.15.123:0/2438483315 conn(0x7fbeac0d00c0 msgr2=0x7fbeac15b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:03.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.226+0000 7fbeabe00640 1 -- 172.21.15.123:0/2438483315 shutdown_connections 2024-04-24T05:46:03.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.227+0000 7fbeabe00640 1 -- 172.21.15.123:0/2438483315 wait complete. 2024-04-24T05:46:03.256 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-24T05:46:03.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.433+0000 7f837ca00640 1 Processor -- start 2024-04-24T05:46:03.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.433+0000 7f837ca00640 1 -- start start 2024-04-24T05:46:03.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.435+0000 7f837ca00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f837815cb40 0x7f837815f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:03.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.435+0000 7f837ca00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83780b4400 con 0x7f837815cfd0 2024-04-24T05:46:03.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.435+0000 7f837ca00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83780b46e0 con 0x7f8378155530 2024-04-24T05:46:03.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.435+0000 7f837ca00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f83780b49c0 con 0x7f837815cb40 2024-04-24T05:46:03.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.435+0000 7f8376800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f837815cfd0 0x7f83780b54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60646/0 (socket says 172.21.15.123:60646) 2024-04-24T05:46:03.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.435+0000 7f8376800640 1 -- 172.21.15.123:0/2223624058 learned_addr learned my addr 172.21.15.123:0/2223624058 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:03.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f836d400640 1 --2- 172.21.15.123:0/2223624058 >> v2:172.21.15.155:3300/0 conn(0x7f837815cb40 0x7f837815f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:03.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 -- 172.21.15.123:0/2223624058 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2525387006 0 0) 0x7f83780b4400 con 0x7f837815cfd0 2024-04-24T05:46:03.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 -- 172.21.15.123:0/2223624058 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f835c003520 con 0x7f837815cfd0 2024-04-24T05:46:03.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 -- 172.21.15.123:0/2223624058 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 446252635 0 0) 0x7f835c003520 con 0x7f837815cfd0 2024-04-24T05:46:03.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 -- 172.21.15.123:0/2223624058 >> v2:172.21.15.155:3300/0 conn(0x7f837815cb40 msgr2=0x7f837815f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:03.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 --2- 172.21.15.123:0/2223624058 >> v2:172.21.15.155:3300/0 conn(0x7f837815cb40 0x7f837815f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:03.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 -- 172.21.15.123:0/2223624058 >> v1:172.21.15.155:6789/0 conn(0x7f8378155530 legacy=0x7f8378155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:03.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 -- 172.21.15.123:0/2223624058 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83780b4ca0 con 0x7f837815cfd0 2024-04-24T05:46:03.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 -- 172.21.15.123:0/2223624058 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f83700019f0 con 0x7f837815cfd0 2024-04-24T05:46:03.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 -- 172.21.15.123:0/2223624058 >> v1:172.21.15.123:6789/0 conn(0x7f837815cfd0 legacy=0x7f83780b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:03.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 --2- 172.21.15.123:0/2223624058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f835c003a00 0x7f835c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:03.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 --2- 172.21.15.123:0/2223624058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f835c004800 0x7f835c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:03.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 --2- 172.21.15.123:0/2223624058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f835c0051c0 0x7f835c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 -- 172.21.15.123:0/2223624058 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83780b49c0 con 0x7f835c004800 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 -- 172.21.15.123:0/2223624058 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f835c00c0c0 con 0x7f835c003a00 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.436+0000 7f8375400640 1 -- 172.21.15.123:0/2223624058 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f835c00c2c0 con 0x7f835c0051c0 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.437+0000 7f836d400640 1 --2- 172.21.15.123:0/2223624058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f835c004800 0x7f835c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.437+0000 7f8376800640 1 --2- 172.21.15.123:0/2223624058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f835c0051c0 0x7f835c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.437+0000 7f836d400640 1 -- 172.21.15.123:0/2223624058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f835c0051c0 msgr2=0x7f835c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.437+0000 7f836d400640 1 --2- 172.21.15.123:0/2223624058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f835c0051c0 0x7f835c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.437+0000 7f8375e00640 1 --2- 172.21.15.123:0/2223624058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f835c003a00 0x7f835c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.437+0000 7f836d400640 1 -- 172.21.15.123:0/2223624058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f835c003a00 msgr2=0x7f835c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.437+0000 7f836d400640 1 --2- 172.21.15.123:0/2223624058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f835c003a00 0x7f835c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.437+0000 7f836d400640 1 -- 172.21.15.123:0/2223624058 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c003520 con 0x7f835c004800 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.437+0000 7f8376800640 1 -- 172.21.15.123:0/2223624058 reap_dead start 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.437+0000 7f8375e00640 1 --2- 172.21.15.123:0/2223624058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f835c003a00 0x7f835c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.438+0000 7f836d400640 1 --2- 172.21.15.123:0/2223624058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f835c004800 0x7f835c004c80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8360009ac0 tx=0x7f835c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7028e3abd8548f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:03.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.438+0000 7f8375400640 1 -- 172.21.15.123:0/2223624058 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360036070 con 0x7f835c004800 2024-04-24T05:46:03.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.438+0000 7f8375400640 1 -- 172.21.15.123:0/2223624058 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8360031400 con 0x7f835c004800 2024-04-24T05:46:03.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.438+0000 7f8375400640 1 -- 172.21.15.123:0/2223624058 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360009690 con 0x7f835c004800 2024-04-24T05:46:03.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.439+0000 7f837ca00640 1 -- 172.21.15.123:0/2223624058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f835c004800 msgr2=0x7f835c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:03.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.439+0000 7f837ca00640 1 --2- 172.21.15.123:0/2223624058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f835c004800 0x7f835c004c80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8360009ac0 tx=0x7f835c005830 comp rx=0 tx=0).stop 2024-04-24T05:46:03.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.439+0000 7f837ca00640 1 -- 172.21.15.123:0/2223624058 shutdown_connections 2024-04-24T05:46:03.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.439+0000 7f837ca00640 1 --2- 172.21.15.123:0/2223624058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f835c004800 0x7f835c004c80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:03.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.439+0000 7f837ca00640 1 -- 172.21.15.123:0/2223624058 >> 172.21.15.123:0/2223624058 conn(0x7f8378072d20 msgr2=0x7f8378073130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:03.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.439+0000 7f837ca00640 1 -- 172.21.15.123:0/2223624058 shutdown_connections 2024-04-24T05:46:03.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.439+0000 7f837ca00640 1 -- 172.21.15.123:0/2223624058 wait complete. 2024-04-24T05:46:03.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.441+0000 7f837ca00640 1 Processor -- start 2024-04-24T05:46:03.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.441+0000 7f837ca00640 1 -- start start 2024-04-24T05:46:03.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f837ca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f835c004800 0x7f837815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:03.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f837ca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8378148330 0x7f8378148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:03.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f837ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83780cc410 0x7f83780cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:03.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f837ca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8378155a90 con 0x7f83780cc410 2024-04-24T05:46:03.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f837ca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8378155910 con 0x7f835c004800 2024-04-24T05:46:03.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f837ca00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8378155c10 con 0x7f8378148330 2024-04-24T05:46:03.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f8376800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83780cc410 0x7f83780cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:03.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f8376800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83780cc410 0x7f83780cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55474/0 (socket says 172.21.15.123:55474) 2024-04-24T05:46:03.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f8376800640 1 -- 172.21.15.123:0/1491484258 learned_addr learned my addr 172.21.15.123:0/1491484258 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:03.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f8375e00640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f835c004800 0x7f837815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:03.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f8376800640 1 -- 172.21.15.123:0/1491484258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8378148330 msgr2=0x7f8378148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:03.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f836d400640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8378148330 0x7f8378148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:03.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f8376800640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8378148330 0x7f8378148790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:03.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f8376800640 1 -- 172.21.15.123:0/1491484258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f835c004800 msgr2=0x7f837815a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f8376800640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f835c004800 0x7f837815a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f8376800640 1 -- 172.21.15.123:0/1491484258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83780ccdd0 con 0x7f83780cc410 2024-04-24T05:46:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.442+0000 7f8375e00640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f835c004800 0x7f837815a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.443+0000 7f8376800640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83780cc410 0x7f83780cc890 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8370001370 tx=0x7f837000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.443+0000 7f836ea00640 1 -- 172.21.15.123:0/1491484258 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370004d60 con 0x7f83780cc410 2024-04-24T05:46:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.443+0000 7f836ea00640 1 -- 172.21.15.123:0/1491484258 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f837000bb80 con 0x7f83780cc410 2024-04-24T05:46:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.443+0000 7f836ea00640 1 -- 172.21.15.123:0/1491484258 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370007110 con 0x7f83780cc410 2024-04-24T05:46:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.443+0000 7f837ca00640 1 -- 172.21.15.123:0/1491484258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83780c6520 con 0x7f83780cc410 2024-04-24T05:46:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.443+0000 7f837ca00640 1 -- 172.21.15.123:0/1491484258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83780c6ae0 con 0x7f83780cc410 2024-04-24T05:46:03.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.444+0000 7f837ca00640 1 -- 172.21.15.123:0/1491484258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f837814abd0 con 0x7f83780cc410 2024-04-24T05:46:03.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.445+0000 7f836ea00640 1 -- 172.21.15.123:0/1491484258 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f837001b020 con 0x7f83780cc410 2024-04-24T05:46:03.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.446+0000 7f836ea00640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f835406c920 0x7f835406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:03.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.446+0000 7f836ea00640 1 -- 172.21.15.123:0/1491484258 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f837008bfb0 con 0x7f83780cc410 2024-04-24T05:46:03.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.447+0000 7f8375e00640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f835406c920 0x7f835406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:03.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.447+0000 7f8375e00640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f835406c920 0x7f835406ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f837806a130 tx=0x7f8368009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:03.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.451+0000 7f836ea00640 1 -- 172.21.15.123:0/1491484258 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f837814abd0 con 0x7f83780cc410 2024-04-24T05:46:03.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:03.652+0000 7f837ca00640 1 -- 172.21.15.123:0/1491484258 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f8378069130 con 0x7f83780cc410 2024-04-24T05:46:04.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.235+0000 7f836ea00640 1 -- 172.21.15.123:0/1491484258 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7f837005a150 con 0x7f83780cc410 2024-04-24T05:46:04.237 INFO:teuthology.orchestra.run.smithi123.stderr:set pool 4 allow_ec_overwrites to true 2024-04-24T05:46:04.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.239+0000 7f837ca00640 1 -- 172.21.15.123:0/1491484258 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f835406c920 msgr2=0x7f835406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:04.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.239+0000 7f837ca00640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f835406c920 0x7f835406ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f837806a130 tx=0x7f8368009040 comp rx=0 tx=0).stop 2024-04-24T05:46:04.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.239+0000 7f837ca00640 1 -- 172.21.15.123:0/1491484258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83780cc410 msgr2=0x7f83780cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.239+0000 7f837ca00640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83780cc410 0x7f83780cc890 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8370001370 tx=0x7f837000b4a0 comp rx=0 tx=0).stop 2024-04-24T05:46:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.239+0000 7f837ca00640 1 -- 172.21.15.123:0/1491484258 shutdown_connections 2024-04-24T05:46:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.239+0000 7f837ca00640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f835406c920 0x7f835406ede0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.239+0000 7f837ca00640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83780cc410 0x7f83780cc890 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.239+0000 7f837ca00640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8378148330 0x7f8378148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.240+0000 7f837ca00640 1 --2- 172.21.15.123:0/1491484258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f835c004800 0x7f837815a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:04.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.240+0000 7f837ca00640 1 -- 172.21.15.123:0/1491484258 >> 172.21.15.123:0/1491484258 conn(0x7f8378072d20 msgr2=0x7f83780b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:04.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.240+0000 7f837ca00640 1 -- 172.21.15.123:0/1491484258 shutdown_connections 2024-04-24T05:46:04.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.240+0000 7f837ca00640 1 -- 172.21.15.123:0/1491484258 wait complete. 2024-04-24T05:46:04.279 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-24T05:46:04.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.450+0000 7f66b9a00640 1 Processor -- start 2024-04-24T05:46:04.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.450+0000 7f66b9a00640 1 -- start start 2024-04-24T05:46:04.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.451+0000 7f66b9a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b4071c50 0x7f66b40ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:04.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.451+0000 7f66b9a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b40cf170 con 0x7f66b406c510 2024-04-24T05:46:04.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.451+0000 7f66b9a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b40cf450 con 0x7f66b40cd0f0 2024-04-24T05:46:04.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.451+0000 7f66b9a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f66b40cf730 con 0x7f66b4071c50 2024-04-24T05:46:04.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.452+0000 7f66b2a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f66b406c510 0x7f66b406c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60662/0 (socket says 172.21.15.123:60662) 2024-04-24T05:46:04.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.452+0000 7f66b2a00640 1 -- 172.21.15.123:0/2148829665 learned_addr learned my addr 172.21.15.123:0/2148829665 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:04.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.452+0000 7f66b2000640 1 --2- 172.21.15.123:0/2148829665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b4071c50 0x7f66b40ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:04.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.452+0000 7f66b1600640 1 -- 172.21.15.123:0/2148829665 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1197820478 0 0) 0x7f66b40cf170 con 0x7f66b406c510 2024-04-24T05:46:04.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.452+0000 7f66b1600640 1 -- 172.21.15.123:0/2148829665 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6698003520 con 0x7f66b406c510 2024-04-24T05:46:04.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.452+0000 7f66b1600640 1 -- 172.21.15.123:0/2148829665 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3368379580 0 0) 0x7f6698003520 con 0x7f66b406c510 2024-04-24T05:46:04.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.452+0000 7f66b1600640 1 -- 172.21.15.123:0/2148829665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b4071c50 msgr2=0x7f66b40ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:04.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.452+0000 7f66b1600640 1 --2- 172.21.15.123:0/2148829665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b4071c50 0x7f66b40ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:04.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.452+0000 7f66b1600640 1 -- 172.21.15.123:0/2148829665 >> v1:172.21.15.155:6789/0 conn(0x7f66b40cd0f0 legacy=0x7f66b40d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:04.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.452+0000 7f66b1600640 1 -- 172.21.15.123:0/2148829665 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b40cfa10 con 0x7f66b406c510 2024-04-24T05:46:04.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b1600640 1 -- 172.21.15.123:0/2148829665 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f66a4001670 con 0x7f66b406c510 2024-04-24T05:46:04.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b1600640 1 -- 172.21.15.123:0/2148829665 >> v1:172.21.15.123:6789/0 conn(0x7f66b406c510 legacy=0x7f66b406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:04.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b1600640 1 --2- 172.21.15.123:0/2148829665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b4071c50 0x7f66980039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:04.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b1600640 1 --2- 172.21.15.123:0/2148829665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6698003f60 0x7f6698004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:04.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b1600640 1 --2- 172.21.15.123:0/2148829665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6698004d40 0x7f66980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:04.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b1600640 1 -- 172.21.15.123:0/2148829665 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66b40cf730 con 0x7f6698003f60 2024-04-24T05:46:04.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b1600640 1 -- 172.21.15.123:0/2148829665 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f669800baa0 con 0x7f6698004d40 2024-04-24T05:46:04.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b1600640 1 -- 172.21.15.123:0/2148829665 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f669800bca0 con 0x7f66b4071c50 2024-04-24T05:46:04.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b2000640 1 --2- 172.21.15.123:0/2148829665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6698003f60 0x7f6698004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:04.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b2a00640 1 --2- 172.21.15.123:0/2148829665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b4071c50 0x7f66980039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:04.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b3400640 1 --2- 172.21.15.123:0/2148829665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6698004d40 0x7f66980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:04.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b2000640 1 -- 172.21.15.123:0/2148829665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b4071c50 msgr2=0x7f66980039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:04.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b2000640 1 --2- 172.21.15.123:0/2148829665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66b4071c50 0x7f66980039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:04.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b2000640 1 -- 172.21.15.123:0/2148829665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6698004d40 msgr2=0x7f66980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:04.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b2000640 1 --2- 172.21.15.123:0/2148829665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6698004d40 0x7f66980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:04.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b2000640 1 -- 172.21.15.123:0/2148829665 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6698003520 con 0x7f6698003f60 2024-04-24T05:46:04.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.453+0000 7f66b3400640 1 --2- 172.21.15.123:0/2148829665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6698004d40 0x7f66980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:04.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.454+0000 7f66b2000640 1 --2- 172.21.15.123:0/2148829665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6698003f60 0x7f6698004800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009aa0 tx=0x7f6698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0535586fcf7a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:04.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.455+0000 7f66b1600640 1 -- 172.21.15.123:0/2148829665 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0036070 con 0x7f6698003f60 2024-04-24T05:46:04.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.455+0000 7f66b1600640 1 -- 172.21.15.123:0/2148829665 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a0031400 con 0x7f6698003f60 2024-04-24T05:46:04.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.455+0000 7f66b1600640 1 -- 172.21.15.123:0/2148829665 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0009670 con 0x7f6698003f60 2024-04-24T05:46:04.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.455+0000 7f66b9a00640 1 -- 172.21.15.123:0/2148829665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6698003f60 msgr2=0x7f6698004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:04.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.455+0000 7f66b9a00640 1 --2- 172.21.15.123:0/2148829665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6698003f60 0x7f6698004800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009aa0 tx=0x7f6698005830 comp rx=0 tx=0).stop 2024-04-24T05:46:04.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.455+0000 7f66b3400640 1 -- 172.21.15.123:0/2148829665 reap_dead start 2024-04-24T05:46:04.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.455+0000 7f66b9a00640 1 -- 172.21.15.123:0/2148829665 shutdown_connections 2024-04-24T05:46:04.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.455+0000 7f66b9a00640 1 -- 172.21.15.123:0/2148829665 >> 172.21.15.123:0/2148829665 conn(0x7f66b40b51f0 msgr2=0x7f66b40b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:04.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.455+0000 7f66b9a00640 1 -- 172.21.15.123:0/2148829665 shutdown_connections 2024-04-24T05:46:04.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.455+0000 7f66b9a00640 1 -- 172.21.15.123:0/2148829665 wait complete. 2024-04-24T05:46:04.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.457+0000 7f66b9a00640 1 Processor -- start 2024-04-24T05:46:04.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.457+0000 7f66b9a00640 1 -- start start 2024-04-24T05:46:04.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.458+0000 7f66b9a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6698003f60 0x7f66b40b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:04.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.458+0000 7f66b9a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66b415c890 0x7f66b41f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:04.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.458+0000 7f66b9a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66b41f4510 0x7f66b415eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:04.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.458+0000 7f66b9a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66b406d180 con 0x7f66b41f4510 2024-04-24T05:46:04.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.458+0000 7f66b9a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f66b406d000 con 0x7f66b415c890 2024-04-24T05:46:04.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.458+0000 7f66b9a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f66b40ced00 con 0x7f6698003f60 2024-04-24T05:46:04.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.459+0000 7f66b3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66b41f4510 0x7f66b415eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:04.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.459+0000 7f66b3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66b41f4510 0x7f66b415eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55488/0 (socket says 172.21.15.123:55488) 2024-04-24T05:46:04.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.459+0000 7f66b3400640 1 -- 172.21.15.123:0/161776517 learned_addr learned my addr 172.21.15.123:0/161776517 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:04.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.459+0000 7f66b2a00640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6698003f60 0x7f66b40b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:04.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.459+0000 7f66b2000640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66b415c890 0x7f66b41f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:04.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.459+0000 7f66b3400640 1 -- 172.21.15.123:0/161776517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6698003f60 msgr2=0x7f66b40b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:04.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.459+0000 7f66b3400640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6698003f60 0x7f66b40b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:04.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.459+0000 7f66b3400640 1 -- 172.21.15.123:0/161776517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66b415c890 msgr2=0x7f66b41f8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:04.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.459+0000 7f66b3400640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66b415c890 0x7f66b41f8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:04.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.459+0000 7f66b3400640 1 -- 172.21.15.123:0/161776517 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b40b1a20 con 0x7f66b41f4510 2024-04-24T05:46:04.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.459+0000 7f66b2a00640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6698003f60 0x7f66b40b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:46:04.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.459+0000 7f66b3400640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66b41f4510 0x7f66b415eb80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66b40cd4d0 tx=0x7f66ac00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:04.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.460+0000 7f66aaa00640 1 -- 172.21.15.123:0/161776517 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac006d50 con 0x7f66b41f4510 2024-04-24T05:46:04.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.460+0000 7f66aaa00640 1 -- 172.21.15.123:0/161776517 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac003970 con 0x7f66b41f4510 2024-04-24T05:46:04.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.460+0000 7f66aaa00640 1 -- 172.21.15.123:0/161776517 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac003d60 con 0x7f66b41f4510 2024-04-24T05:46:04.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.460+0000 7f66b9a00640 1 -- 172.21.15.123:0/161776517 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b40b1d00 con 0x7f66b41f4510 2024-04-24T05:46:04.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.460+0000 7f66b9a00640 1 -- 172.21.15.123:0/161776517 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b41f4990 con 0x7f66b41f4510 2024-04-24T05:46:04.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.461+0000 7f66b9a00640 1 -- 172.21.15.123:0/161776517 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b4068a20 con 0x7f66b41f4510 2024-04-24T05:46:04.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.462+0000 7f66aaa00640 1 -- 172.21.15.123:0/161776517 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66ac01b020 con 0x7f66b41f4510 2024-04-24T05:46:04.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.463+0000 7f66aaa00640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f669406c920 0x7f669406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:04.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.463+0000 7f66aaa00640 1 -- 172.21.15.123:0/161776517 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f66ac08ba20 con 0x7f66b41f4510 2024-04-24T05:46:04.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.463+0000 7f66b2a00640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f669406c920 0x7f669406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:04.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.464+0000 7f66b2a00640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f669406c920 0x7f669406ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f66a4001cd0 tx=0x7f66a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:04.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.467+0000 7f66aaa00640 1 -- 172.21.15.123:0/161776517 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66ac059bc0 con 0x7f66b41f4510 2024-04-24T05:46:04.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:04.714+0000 7f66b9a00640 1 -- 172.21.15.123:0/161776517 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f66b40b5e60 con 0x7f66b41f4510 2024-04-24T05:46:05.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.239+0000 7f66aaa00640 1 -- 172.21.15.123:0/161776517 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f66ac01ce30 con 0x7f66b41f4510 2024-04-24T05:46:05.241 INFO:teuthology.orchestra.run.smithi123.stderr:added data pool 4 to fsmap 2024-04-24T05:46:05.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.243+0000 7f66b9a00640 1 -- 172.21.15.123:0/161776517 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f669406c920 msgr2=0x7f669406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:05.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.243+0000 7f66b9a00640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f669406c920 0x7f669406ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f66a4001cd0 tx=0x7f66a4009040 comp rx=0 tx=0).stop 2024-04-24T05:46:05.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.243+0000 7f66b9a00640 1 -- 172.21.15.123:0/161776517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66b41f4510 msgr2=0x7f66b415eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:05.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.243+0000 7f66b9a00640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66b41f4510 0x7f66b415eb80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66b40cd4d0 tx=0x7f66ac00dc00 comp rx=0 tx=0).stop 2024-04-24T05:46:05.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.243+0000 7f66b9a00640 1 -- 172.21.15.123:0/161776517 shutdown_connections 2024-04-24T05:46:05.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.243+0000 7f66b9a00640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f669406c920 0x7f669406ede0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.243+0000 7f66b9a00640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66b41f4510 0x7f66b415eb80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.243+0000 7f66b9a00640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66b415c890 0x7f66b41f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.243+0000 7f66b9a00640 1 --2- 172.21.15.123:0/161776517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6698003f60 0x7f66b40b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.243+0000 7f66b9a00640 1 -- 172.21.15.123:0/161776517 >> 172.21.15.123:0/161776517 conn(0x7f66b40b51f0 msgr2=0x7f66b40aea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:05.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.243+0000 7f66b9a00640 1 -- 172.21.15.123:0/161776517 shutdown_connections 2024-04-24T05:46:05.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.243+0000 7f66b9a00640 1 -- 172.21.15.123:0/161776517 wait complete. 2024-04-24T05:46:05.268 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T05:46:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.460+0000 7f60e9600640 1 Processor -- start 2024-04-24T05:46:05.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.461+0000 7f60e9600640 1 -- start start 2024-04-24T05:46:05.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.462+0000 7f60e9600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60e415a2a0 0x7f60e40ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.462+0000 7f60e9600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f60e40ad690 0x7f60e40af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.462+0000 7f60e9600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60e40afbf0 con 0x7f60e40af780 2024-04-24T05:46:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.462+0000 7f60e9600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f60e40adaf0 con 0x7f60e40ad690 2024-04-24T05:46:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.462+0000 7f60e9600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f60e415a680 con 0x7f60e415a2a0 2024-04-24T05:46:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.463+0000 7f60e2a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60e415a2a0 0x7f60e40ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:05.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.463+0000 7f60e3400640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f60e40af780 0x7f60e4141e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.123:34742/0 (socket says 172.21.15.123:34742) 2024-04-24T05:46:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.463+0000 7f60e3400640 1 -- 172.21.15.123:0/1005672049 learned_addr learned my addr 172.21.15.123:0/1005672049 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.463+0000 7f60dbe00640 1 --2- 172.21.15.123:0/1005672049 >> v2:172.21.15.155:3300/0 conn(0x7f60e40ad690 0x7f60e40af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.464+0000 7f60e2a00640 1 -- 172.21.15.123:0/1005672049 >> v2:172.21.15.155:3300/0 conn(0x7f60e40ad690 msgr2=0x7f60e40af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.464+0000 7f60e2a00640 1 --2- 172.21.15.123:0/1005672049 >> v2:172.21.15.155:3300/0 conn(0x7f60e40ad690 0x7f60e40af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.464+0000 7f60e2a00640 1 -- 172.21.15.123:0/1005672049 >> v1:172.21.15.155:6789/0 conn(0x7f60e40af780 legacy=0x7f60e4141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.464+0000 7f60e2a00640 1 -- 172.21.15.123:0/1005672049 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e415cdf0 con 0x7f60e415a2a0 2024-04-24T05:46:05.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.464+0000 7f60e2000640 1 -- 172.21.15.123:0/1005672049 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3414286931 0 0) 0x7f60e40afbf0 con 0x7f60e40af780 2024-04-24T05:46:05.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.465+0000 7f60e2a00640 1 --2- 172.21.15.123:0/1005672049 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60e415a2a0 0x7f60e40ad150 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f60d40097a0 tx=0x7f60d402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7e31bb33ed55424 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:05.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.465+0000 7f60e2000640 1 -- 172.21.15.123:0/1005672049 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4037070 con 0x7f60e415a2a0 2024-04-24T05:46:05.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.465+0000 7f60e2000640 1 -- 172.21.15.123:0/1005672049 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d402ae50 con 0x7f60e415a2a0 2024-04-24T05:46:05.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.466+0000 7f60e2000640 1 -- 172.21.15.123:0/1005672049 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4032660 con 0x7f60e415a2a0 2024-04-24T05:46:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.466+0000 7f60e9600640 1 -- 172.21.15.123:0/1005672049 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60e415a2a0 msgr2=0x7f60e40ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.466+0000 7f60e9600640 1 --2- 172.21.15.123:0/1005672049 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60e415a2a0 0x7f60e40ad150 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f60d40097a0 tx=0x7f60d402a100 comp rx=0 tx=0).stop 2024-04-24T05:46:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.466+0000 7f60e9600640 1 -- 172.21.15.123:0/1005672049 shutdown_connections 2024-04-24T05:46:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.466+0000 7f60e9600640 1 --2- 172.21.15.123:0/1005672049 >> v2:172.21.15.155:3300/0 conn(0x7f60e40ad690 0x7f60e40af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.466+0000 7f60e9600640 1 --2- 172.21.15.123:0/1005672049 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60e415a2a0 0x7f60e40ad150 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.466+0000 7f60e9600640 1 -- 172.21.15.123:0/1005672049 >> 172.21.15.123:0/1005672049 conn(0x7f60e40b1e70 msgr2=0x7f60e40b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.466+0000 7f60e9600640 1 -- 172.21.15.123:0/1005672049 shutdown_connections 2024-04-24T05:46:05.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.466+0000 7f60e9600640 1 -- 172.21.15.123:0/1005672049 wait complete. 2024-04-24T05:46:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.468+0000 7f60e9600640 1 Processor -- start 2024-04-24T05:46:05.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.468+0000 7f60e9600640 1 -- start start 2024-04-24T05:46:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.469+0000 7f60e9600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60e40ad690 0x7f60e40cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.469+0000 7f60e9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e40af780 0x7f60e40cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.469+0000 7f60e9600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60e415a2a0 0x7f60e406ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.469+0000 7f60e9600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f60e415d3b0 con 0x7f60e40af780 2024-04-24T05:46:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.469+0000 7f60e9600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f60e415da00 con 0x7f60e415a2a0 2024-04-24T05:46:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.469+0000 7f60e9600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f60e415d690 con 0x7f60e40ad690 2024-04-24T05:46:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.469+0000 7f60dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e40af780 0x7f60e40cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.470+0000 7f60dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e40af780 0x7f60e40cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55496/0 (socket says 172.21.15.123:55496) 2024-04-24T05:46:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.470+0000 7f60dbe00640 1 -- 172.21.15.123:0/2257286694 learned_addr learned my addr 172.21.15.123:0/2257286694 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.470+0000 7f60e3400640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60e415a2a0 0x7f60e406ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.470+0000 7f60e2a00640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60e40ad690 0x7f60e40cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.470+0000 7f60dbe00640 1 -- 172.21.15.123:0/2257286694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60e40ad690 msgr2=0x7f60e40cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.470+0000 7f60dbe00640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60e40ad690 0x7f60e40cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.470+0000 7f60dbe00640 1 -- 172.21.15.123:0/2257286694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60e415a2a0 msgr2=0x7f60e406ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.470+0000 7f60dbe00640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60e415a2a0 0x7f60e406ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.470+0000 7f60dbe00640 1 -- 172.21.15.123:0/2257286694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e40ce760 con 0x7f60e40af780 2024-04-24T05:46:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.470+0000 7f60e3400640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60e415a2a0 0x7f60e406ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.470+0000 7f60e2a00640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60e40ad690 0x7f60e40cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.470+0000 7f60dbe00640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e40af780 0x7f60e40cf200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00e8a0 tx=0x7f60cc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.470+0000 7f60daa00640 1 -- 172.21.15.123:0/2257286694 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc017070 con 0x7f60e40af780 2024-04-24T05:46:05.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.471+0000 7f60daa00640 1 -- 172.21.15.123:0/2257286694 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60cc00c940 con 0x7f60e40af780 2024-04-24T05:46:05.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.471+0000 7f60daa00640 1 -- 172.21.15.123:0/2257286694 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc00cd30 con 0x7f60e40af780 2024-04-24T05:46:05.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.471+0000 7f60e9600640 1 -- 172.21.15.123:0/2257286694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e4158ff0 con 0x7f60e40af780 2024-04-24T05:46:05.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.471+0000 7f60e9600640 1 -- 172.21.15.123:0/2257286694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e40cea00 con 0x7f60e40af780 2024-04-24T05:46:05.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.472+0000 7f60e9600640 1 -- 172.21.15.123:0/2257286694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e4155310 con 0x7f60e40af780 2024-04-24T05:46:05.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.473+0000 7f60daa00640 1 -- 172.21.15.123:0/2257286694 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60cc01a070 con 0x7f60e40af780 2024-04-24T05:46:05.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.474+0000 7f60daa00640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f60c006c8f0 0x7f60c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:05.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.474+0000 7f60daa00640 1 -- 172.21.15.123:0/2257286694 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f60cc08c060 con 0x7f60e40af780 2024-04-24T05:46:05.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.474+0000 7f60e2a00640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f60c006c8f0 0x7f60c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:05.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.475+0000 7f60e2a00640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f60c006c8f0 0x7f60c006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f60d4034000 tx=0x7f60d402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:05.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.479+0000 7f60daa00640 1 -- 172.21.15.123:0/2257286694 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f60e4155310 con 0x7f60e40af780 2024-04-24T05:46:05.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.715+0000 7f60e9600640 1 -- 172.21.15.123:0/2257286694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60e40b1730 con 0x7f60e40af780 2024-04-24T05:46:05.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.716+0000 7f60daa00640 1 -- 172.21.15.123:0/2257286694 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7f60e40b1730 con 0x7f60e40af780 2024-04-24T05:46:05.718 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:05.718 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2743935658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2743935658},{"type":"v1","addr":"172.21.15.155:6834","nonce":2743935658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1148561953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1148561953},{"type":"v1","addr":"172.21.15.123:6836","nonce":1148561953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2992052647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2992052647},{"type":"v1","addr":"172.21.15.155:6835","nonce":2992052647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T05:45:59.639548+0000","modified":"2024-04-24T05:46:04.716074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/3370584410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3370584410},{"type":"v1","addr":"172.21.15.123:6837","nonce":3370584410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T05:46:05.719 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 7 2024-04-24T05:46:05.719 INFO:tasks.ceph.mds.a.smithi123.stderr:2024-04-24T05:46:05.717+0000 7f45ce000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T05:46:05.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.719+0000 7f60e9600640 1 -- 172.21.15.123:0/2257286694 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f60c006c8f0 msgr2=0x7f60c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:05.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.719+0000 7f60e9600640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f60c006c8f0 0x7f60c006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f60d4034000 tx=0x7f60d402e000 comp rx=0 tx=0).stop 2024-04-24T05:46:05.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.720+0000 7f60e9600640 1 -- 172.21.15.123:0/2257286694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e40af780 msgr2=0x7f60e40cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:05.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.720+0000 7f60e9600640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e40af780 0x7f60e40cf200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00e8a0 tx=0x7f60cc00ed60 comp rx=0 tx=0).stop 2024-04-24T05:46:05.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.720+0000 7f60e9600640 1 -- 172.21.15.123:0/2257286694 shutdown_connections 2024-04-24T05:46:05.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.720+0000 7f60e9600640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f60c006c8f0 0x7f60c006edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.720+0000 7f60e9600640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60e415a2a0 0x7f60e406ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.720+0000 7f60e9600640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60e40af780 0x7f60e40cf200 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.720+0000 7f60e9600640 1 --2- 172.21.15.123:0/2257286694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60e40ad690 0x7f60e40cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.720+0000 7f60e9600640 1 -- 172.21.15.123:0/2257286694 >> 172.21.15.123:0/2257286694 conn(0x7f60e40b1e70 msgr2=0x7f60e40b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:05.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.720+0000 7f60e9600640 1 -- 172.21.15.123:0/2257286694 shutdown_connections 2024-04-24T05:46:05.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.720+0000 7f60e9600640 1 -- 172.21.15.123:0/2257286694 wait complete. 2024-04-24T05:46:05.740 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:05.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.931+0000 7fda30400640 1 Processor -- start 2024-04-24T05:46:05.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.932+0000 7fda30400640 1 -- start start 2024-04-24T05:46:05.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.933+0000 7fda30400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fda2806f470 0x7fda28152520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:05.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.933+0000 7fda30400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda28071cd0 con 0x7fda2806f090 2024-04-24T05:46:05.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.933+0000 7fda30400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda28071fb0 con 0x7fda280736d0 2024-04-24T05:46:05.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.933+0000 7fda30400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fda28072290 con 0x7fda2806f470 2024-04-24T05:46:05.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.933+0000 7fda2da00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fda2806f090 0x7fda28072fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60670/0 (socket says 172.21.15.123:60670) 2024-04-24T05:46:05.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.933+0000 7fda2da00640 1 -- 172.21.15.123:0/2041356487 learned_addr learned my addr 172.21.15.123:0/2041356487 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:05.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.933+0000 7fda2e400640 1 --2- 172.21.15.123:0/2041356487 >> v2:172.21.15.123:3300/0 conn(0x7fda2806f470 0x7fda28152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:05.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.933+0000 7fda2e400640 1 -- 172.21.15.123:0/2041356487 >> v1:172.21.15.155:6789/0 conn(0x7fda280736d0 legacy=0x7fda28151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:05.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.933+0000 7fda2e400640 1 -- 172.21.15.123:0/2041356487 >> v1:172.21.15.123:6789/0 conn(0x7fda2806f090 legacy=0x7fda28072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:05.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.934+0000 7fda2e400640 1 -- 172.21.15.123:0/2041356487 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda28072570 con 0x7fda2806f470 2024-04-24T05:46:05.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.934+0000 7fda2e400640 1 --2- 172.21.15.123:0/2041356487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda2806f470 0x7fda28152520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fda20009d90 tx=0x7fda2002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c587d1b79ad188 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:05.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.935+0000 7fda27e00640 1 -- 172.21.15.123:0/2041356487 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20031600 con 0x7fda2806f470 2024-04-24T05:46:05.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.935+0000 7fda27e00640 1 -- 172.21.15.123:0/2041356487 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda200317a0 con 0x7fda2806f470 2024-04-24T05:46:05.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.935+0000 7fda27e00640 1 -- 172.21.15.123:0/2041356487 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20031b70 con 0x7fda2806f470 2024-04-24T05:46:05.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.935+0000 7fda30400640 1 -- 172.21.15.123:0/2041356487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda2806f470 msgr2=0x7fda28152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:05.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.935+0000 7fda30400640 1 --2- 172.21.15.123:0/2041356487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda2806f470 0x7fda28152520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fda20009d90 tx=0x7fda2002a610 comp rx=0 tx=0).stop 2024-04-24T05:46:05.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.935+0000 7fda30400640 1 -- 172.21.15.123:0/2041356487 shutdown_connections 2024-04-24T05:46:05.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.935+0000 7fda30400640 1 --2- 172.21.15.123:0/2041356487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda2806f470 0x7fda28152520 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.936+0000 7fda30400640 1 -- 172.21.15.123:0/2041356487 >> 172.21.15.123:0/2041356487 conn(0x7fda280b1e70 msgr2=0x7fda280b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:05.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.936+0000 7fda30400640 1 -- 172.21.15.123:0/2041356487 shutdown_connections 2024-04-24T05:46:05.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.936+0000 7fda30400640 1 -- 172.21.15.123:0/2041356487 wait complete. 2024-04-24T05:46:05.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.937+0000 7fda30400640 1 Processor -- start 2024-04-24T05:46:05.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.937+0000 7fda30400640 1 -- start start 2024-04-24T05:46:05.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.938+0000 7fda30400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda2806f090 0x7fda280cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:05.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.938+0000 7fda30400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda2806f470 0x7fda2815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:05.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.938+0000 7fda30400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda280736d0 0x7fda2815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:05.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.938+0000 7fda30400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda2815c690 con 0x7fda280736d0 2024-04-24T05:46:05.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.938+0000 7fda30400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda2815c510 con 0x7fda2806f470 2024-04-24T05:46:05.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.938+0000 7fda30400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda280b2990 con 0x7fda2806f090 2024-04-24T05:46:05.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.939+0000 7fda2e400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda280736d0 0x7fda2815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:05.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.939+0000 7fda2e400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda280736d0 0x7fda2815aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55528/0 (socket says 172.21.15.123:55528) 2024-04-24T05:46:05.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.939+0000 7fda2e400640 1 -- 172.21.15.123:0/1590160809 learned_addr learned my addr 172.21.15.123:0/1590160809 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:05.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.939+0000 7fda2da00640 1 --2- 172.21.15.123:0/1590160809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda2806f090 0x7fda280cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:05.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.939+0000 7fda2d000640 1 --2- 172.21.15.123:0/1590160809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda2806f470 0x7fda2815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:05.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.939+0000 7fda2e400640 1 -- 172.21.15.123:0/1590160809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda2806f090 msgr2=0x7fda280cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:05.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.939+0000 7fda2e400640 1 --2- 172.21.15.123:0/1590160809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda2806f090 0x7fda280cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.939+0000 7fda2e400640 1 -- 172.21.15.123:0/1590160809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda2806f470 msgr2=0x7fda2815a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:05.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.939+0000 7fda2e400640 1 --2- 172.21.15.123:0/1590160809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda2806f470 0x7fda2815a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:05.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.939+0000 7fda2e400640 1 -- 172.21.15.123:0/1590160809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda28146d80 con 0x7fda280736d0 2024-04-24T05:46:05.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.939+0000 7fda2e400640 1 --2- 172.21.15.123:0/1590160809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda280736d0 0x7fda2815aa10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fda20009d90 tx=0x7fda20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:05.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.940+0000 7fda25600640 1 -- 172.21.15.123:0/1590160809 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20040070 con 0x7fda280736d0 2024-04-24T05:46:05.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.940+0000 7fda25600640 1 -- 172.21.15.123:0/1590160809 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda20030b60 con 0x7fda280736d0 2024-04-24T05:46:05.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.940+0000 7fda25600640 1 -- 172.21.15.123:0/1590160809 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2003c420 con 0x7fda280736d0 2024-04-24T05:46:05.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.940+0000 7fda30400640 1 -- 172.21.15.123:0/1590160809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda28143ed0 con 0x7fda280736d0 2024-04-24T05:46:05.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.940+0000 7fda30400640 1 -- 172.21.15.123:0/1590160809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda28144490 con 0x7fda280736d0 2024-04-24T05:46:05.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.941+0000 7fda30400640 1 -- 172.21.15.123:0/1590160809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda28068a20 con 0x7fda280736d0 2024-04-24T05:46:05.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.942+0000 7fda25600640 1 -- 172.21.15.123:0/1590160809 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda20039040 con 0x7fda280736d0 2024-04-24T05:46:05.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.943+0000 7fda25600640 1 --2- 172.21.15.123:0/1590160809 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda0806c920 0x7fda0806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:05.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.943+0000 7fda25600640 1 -- 172.21.15.123:0/1590160809 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fda200ad220 con 0x7fda280736d0 2024-04-24T05:46:05.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.943+0000 7fda2da00640 1 --2- 172.21.15.123:0/1590160809 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda0806c920 0x7fda0806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:05.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.944+0000 7fda2da00640 1 --2- 172.21.15.123:0/1590160809 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda0806c920 0x7fda0806ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda1c000f60 tx=0x7fda1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:05.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:05.947+0000 7fda25600640 1 -- 172.21.15.123:0/1590160809 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda2007b3c0 con 0x7fda280736d0 2024-04-24T05:46:06.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.154+0000 7fda30400640 1 -- 172.21.15.123:0/1590160809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda28073cb0 con 0x7fda280736d0 2024-04-24T05:46:06.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.156+0000 7fda25600640 1 -- 172.21.15.123:0/1590160809 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7fda2007acc0 con 0x7fda280736d0 2024-04-24T05:46:06.158 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:06.158 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:04.715857+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:06.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.159+0000 7fda30400640 1 -- 172.21.15.123:0/1590160809 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda0806c920 msgr2=0x7fda0806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:06.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.159+0000 7fda30400640 1 --2- 172.21.15.123:0/1590160809 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda0806c920 0x7fda0806ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda1c000f60 tx=0x7fda1c009040 comp rx=0 tx=0).stop 2024-04-24T05:46:06.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.159+0000 7fda30400640 1 -- 172.21.15.123:0/1590160809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda280736d0 msgr2=0x7fda2815aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:06.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.159+0000 7fda30400640 1 --2- 172.21.15.123:0/1590160809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda280736d0 0x7fda2815aa10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fda20009d90 tx=0x7fda20004a40 comp rx=0 tx=0).stop 2024-04-24T05:46:06.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.159+0000 7fda30400640 1 -- 172.21.15.123:0/1590160809 shutdown_connections 2024-04-24T05:46:06.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.159+0000 7fda30400640 1 --2- 172.21.15.123:0/1590160809 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda0806c920 0x7fda0806ede0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.159+0000 7fda30400640 1 --2- 172.21.15.123:0/1590160809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda280736d0 0x7fda2815aa10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.159+0000 7fda30400640 1 --2- 172.21.15.123:0/1590160809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda2806f470 0x7fda2815a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.159+0000 7fda30400640 1 --2- 172.21.15.123:0/1590160809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda2806f090 0x7fda280cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.159+0000 7fda30400640 1 -- 172.21.15.123:0/1590160809 >> 172.21.15.123:0/1590160809 conn(0x7fda280b1e70 msgr2=0x7fda280b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:06.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.159+0000 7fda30400640 1 -- 172.21.15.123:0/1590160809 shutdown_connections 2024-04-24T05:46:06.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.159+0000 7fda30400640 1 -- 172.21.15.123:0/1590160809 wait complete. 2024-04-24T05:46:06.181 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:06.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.371+0000 7f03c5a00640 1 Processor -- start 2024-04-24T05:46:06.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.371+0000 7f03c5a00640 1 -- start start 2024-04-24T05:46:06.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.372+0000 7f03c5a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03c015a2a0 0x7f03c00ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:06.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.373+0000 7f03c5a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f03c00ad690 0x7f03c00af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:06.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.373+0000 7f03c5a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03c00afbf0 con 0x7f03c00af780 2024-04-24T05:46:06.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.373+0000 7f03c5a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f03c00adaf0 con 0x7f03c00ad690 2024-04-24T05:46:06.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.373+0000 7f03c5a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f03c015a680 con 0x7f03c015a2a0 2024-04-24T05:46:06.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.373+0000 7f03bea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03c015a2a0 0x7f03c00ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:06.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.373+0000 7f03bea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03c015a2a0 0x7f03c00ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:33830/0 (socket says 172.21.15.123:33830) 2024-04-24T05:46:06.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.373+0000 7f03bea00640 1 -- 172.21.15.123:0/816293143 learned_addr learned my addr 172.21.15.123:0/816293143 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:06.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.373+0000 7f03be000640 1 --2- 172.21.15.123:0/816293143 >> v2:172.21.15.155:3300/0 conn(0x7f03c00ad690 0x7f03c00af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:06.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.374+0000 7f03bea00640 1 -- 172.21.15.123:0/816293143 >> v2:172.21.15.155:3300/0 conn(0x7f03c00ad690 msgr2=0x7f03c00af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:06.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.374+0000 7f03bea00640 1 --2- 172.21.15.123:0/816293143 >> v2:172.21.15.155:3300/0 conn(0x7f03c00ad690 0x7f03c00af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.374+0000 7f03bea00640 1 -- 172.21.15.123:0/816293143 >> v1:172.21.15.155:6789/0 conn(0x7f03c00af780 legacy=0x7f03c0141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:06.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.374+0000 7f03bea00640 1 -- 172.21.15.123:0/816293143 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c015cdf0 con 0x7f03c015a2a0 2024-04-24T05:46:06.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.375+0000 7f03bea00640 1 --2- 172.21.15.123:0/816293143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03c015a2a0 0x7f03c00ad150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009870 tx=0x7f03b002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=987b3f0124e062e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:06.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.376+0000 7f03bd600640 1 -- 172.21.15.123:0/816293143 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0037070 con 0x7f03c015a2a0 2024-04-24T05:46:06.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.376+0000 7f03bd600640 1 -- 172.21.15.123:0/816293143 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b0032440 con 0x7f03c015a2a0 2024-04-24T05:46:06.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.376+0000 7f03bd600640 1 -- 172.21.15.123:0/816293143 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0032810 con 0x7f03c015a2a0 2024-04-24T05:46:06.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.376+0000 7f03c5a00640 1 -- 172.21.15.123:0/816293143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03c015a2a0 msgr2=0x7f03c00ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:06.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.376+0000 7f03c5a00640 1 --2- 172.21.15.123:0/816293143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03c015a2a0 0x7f03c00ad150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009870 tx=0x7f03b002a1c0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.376+0000 7f03c5a00640 1 -- 172.21.15.123:0/816293143 shutdown_connections 2024-04-24T05:46:06.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.376+0000 7f03c5a00640 1 --2- 172.21.15.123:0/816293143 >> v2:172.21.15.155:3300/0 conn(0x7f03c00ad690 0x7f03c00af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.376+0000 7f03c5a00640 1 --2- 172.21.15.123:0/816293143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03c015a2a0 0x7f03c00ad150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.376+0000 7f03c5a00640 1 -- 172.21.15.123:0/816293143 >> 172.21.15.123:0/816293143 conn(0x7f03c00b1e70 msgr2=0x7f03c00b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:06.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.376+0000 7f03c5a00640 1 -- 172.21.15.123:0/816293143 shutdown_connections 2024-04-24T05:46:06.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.376+0000 7f03c5a00640 1 -- 172.21.15.123:0/816293143 wait complete. 2024-04-24T05:46:06.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.378+0000 7f03c5a00640 1 Processor -- start 2024-04-24T05:46:06.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.378+0000 7f03c5a00640 1 -- start start 2024-04-24T05:46:06.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.379+0000 7f03c5a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c00ad690 0x7f03c00cee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:06.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.379+0000 7f03c5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03c00af780 0x7f03c015bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:06.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.379+0000 7f03c5a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03c015a2a0 0x7f03c01566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:06.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.379+0000 7f03c5a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03c015d3b0 con 0x7f03c00ad690 2024-04-24T05:46:06.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.379+0000 7f03c5a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f03c015da00 con 0x7f03c00af780 2024-04-24T05:46:06.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.379+0000 7f03c5a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f03c015d690 con 0x7f03c015a2a0 2024-04-24T05:46:06.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.380+0000 7f03bea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c00ad690 0x7f03c00cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:06.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.380+0000 7f03bea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c00ad690 0x7f03c00cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55542/0 (socket says 172.21.15.123:55542) 2024-04-24T05:46:06.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.380+0000 7f03bea00640 1 -- 172.21.15.123:0/3888879629 learned_addr learned my addr 172.21.15.123:0/3888879629 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:06.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.380+0000 7f03bf400640 1 --2- 172.21.15.123:0/3888879629 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03c015a2a0 0x7f03c01566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:06.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.380+0000 7f03bea00640 1 -- 172.21.15.123:0/3888879629 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03c015a2a0 msgr2=0x7f03c01566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:06.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.380+0000 7f03bea00640 1 --2- 172.21.15.123:0/3888879629 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03c015a2a0 0x7f03c01566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.380+0000 7f03bea00640 1 -- 172.21.15.123:0/3888879629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03c00af780 msgr2=0x7f03c015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:06.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.380+0000 7f03bea00640 1 --2- 172.21.15.123:0/3888879629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03c00af780 0x7f03c015bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.380+0000 7f03bea00640 1 -- 172.21.15.123:0/3888879629 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c00cf380 con 0x7f03c00ad690 2024-04-24T05:46:06.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.380+0000 7f03bea00640 1 --2- 172.21.15.123:0/3888879629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c00ad690 0x7f03c00cee40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03b002a200 tx=0x7f03b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:06.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.381+0000 7f03b6a00640 1 -- 172.21.15.123:0/3888879629 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0041030 con 0x7f03c00ad690 2024-04-24T05:46:06.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.381+0000 7f03b6a00640 1 -- 172.21.15.123:0/3888879629 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b0030440 con 0x7f03c00ad690 2024-04-24T05:46:06.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.381+0000 7f03b6a00640 1 -- 172.21.15.123:0/3888879629 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0037040 con 0x7f03c00ad690 2024-04-24T05:46:06.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.381+0000 7f03c5a00640 1 -- 172.21.15.123:0/3888879629 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c00c9ef0 con 0x7f03c00ad690 2024-04-24T05:46:06.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.381+0000 7f03c5a00640 1 -- 172.21.15.123:0/3888879629 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c00ca4b0 con 0x7f03c00ad690 2024-04-24T05:46:06.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.382+0000 7f03c5a00640 1 -- 172.21.15.123:0/3888879629 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03c0068a20 con 0x7f03c00ad690 2024-04-24T05:46:06.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.383+0000 7f03b6a00640 1 -- 172.21.15.123:0/3888879629 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03b0044070 con 0x7f03c00ad690 2024-04-24T05:46:06.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.384+0000 7f03b6a00640 1 --2- 172.21.15.123:0/3888879629 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f03a006c8f0 0x7f03a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:06.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.384+0000 7f03b6a00640 1 -- 172.21.15.123:0/3888879629 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f03b00ac570 con 0x7f03c00ad690 2024-04-24T05:46:06.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.384+0000 7f03be000640 1 --2- 172.21.15.123:0/3888879629 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f03a006c8f0 0x7f03a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:06.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.385+0000 7f03be000640 1 --2- 172.21.15.123:0/3888879629 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f03a006c8f0 0x7f03a006edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f03ac006a90 tx=0x7f03ac006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:06.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.388+0000 7f03b6a00640 1 -- 172.21.15.123:0/3888879629 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f03b007a650 con 0x7f03c00ad690 2024-04-24T05:46:06.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.590+0000 7f03c5a00640 1 -- 172.21.15.123:0/3888879629 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03c01564d0 con 0x7f03c00ad690 2024-04-24T05:46:06.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.593+0000 7f03b6a00640 1 -- 172.21.15.123:0/3888879629 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f03b0079f50 con 0x7f03c00ad690 2024-04-24T05:46:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:06.595 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:04.715857+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:06.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.595+0000 7f03c5a00640 1 -- 172.21.15.123:0/3888879629 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f03a006c8f0 msgr2=0x7f03a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:06.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.595+0000 7f03c5a00640 1 --2- 172.21.15.123:0/3888879629 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f03a006c8f0 0x7f03a006edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f03ac006a90 tx=0x7f03ac006a20 comp rx=0 tx=0).stop 2024-04-24T05:46:06.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.595+0000 7f03c5a00640 1 -- 172.21.15.123:0/3888879629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c00ad690 msgr2=0x7f03c00cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:06.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.595+0000 7f03c5a00640 1 --2- 172.21.15.123:0/3888879629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c00ad690 0x7f03c00cee40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03b002a200 tx=0x7f03b00046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.595+0000 7f03c5a00640 1 -- 172.21.15.123:0/3888879629 shutdown_connections 2024-04-24T05:46:06.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.595+0000 7f03c5a00640 1 --2- 172.21.15.123:0/3888879629 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f03a006c8f0 0x7f03a006edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.595+0000 7f03c5a00640 1 --2- 172.21.15.123:0/3888879629 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03c015a2a0 0x7f03c01566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.595+0000 7f03c5a00640 1 --2- 172.21.15.123:0/3888879629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03c00af780 0x7f03c015bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.595+0000 7f03c5a00640 1 --2- 172.21.15.123:0/3888879629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c00ad690 0x7f03c00cee40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.595+0000 7f03c5a00640 1 -- 172.21.15.123:0/3888879629 >> 172.21.15.123:0/3888879629 conn(0x7f03c00b1e70 msgr2=0x7f03c00b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:06.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.596+0000 7f03c5a00640 1 -- 172.21.15.123:0/3888879629 shutdown_connections 2024-04-24T05:46:06.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.596+0000 7f03c5a00640 1 -- 172.21.15.123:0/3888879629 wait complete. 2024-04-24T05:46:06.631 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T05:46:06.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.829+0000 7fd685c00640 1 Processor -- start 2024-04-24T05:46:06.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.829+0000 7fd685c00640 1 -- start start 2024-04-24T05:46:06.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.830+0000 7fd685c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd680071dc0 0x7fd6800721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:06.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.830+0000 7fd685c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6801675a0 con 0x7fd68006f490 2024-04-24T05:46:06.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.831+0000 7fd685c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd680163070 con 0x7fd68006ba00 2024-04-24T05:46:06.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.831+0000 7fd685c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd68006dff0 con 0x7fd680071dc0 2024-04-24T05:46:06.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.831+0000 7fd67f400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd68006f490 0x7fd680166e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60684/0 (socket says 172.21.15.123:60684) 2024-04-24T05:46:06.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.831+0000 7fd67f400640 1 -- 172.21.15.123:0/2249679656 learned_addr learned my addr 172.21.15.123:0/2249679656 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:06.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.831+0000 7fd67ea00640 1 --2- 172.21.15.123:0/2249679656 >> v2:172.21.15.155:3300/0 conn(0x7fd680071dc0 0x7fd6800721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:06.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.831+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1950968603 0 0) 0x7fd6801675a0 con 0x7fd68006f490 2024-04-24T05:46:06.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.831+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd664003520 con 0x7fd68006f490 2024-04-24T05:46:06.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2123589319 0 0) 0x7fd680163070 con 0x7fd68006ba00 2024-04-24T05:46:06.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6801675a0 con 0x7fd68006ba00 2024-04-24T05:46:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 590196743 0 0) 0x7fd664003520 con 0x7fd68006f490 2024-04-24T05:46:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 >> v2:172.21.15.155:3300/0 conn(0x7fd680071dc0 msgr2=0x7fd6800721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 --2- 172.21.15.123:0/2249679656 >> v2:172.21.15.155:3300/0 conn(0x7fd680071dc0 0x7fd6800721a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 >> v1:172.21.15.155:6789/0 conn(0x7fd68006ba00 legacy=0x7fd680166780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68006cce0 con 0x7fd68006f490 2024-04-24T05:46:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7fd6780019f0 con 0x7fd68006f490 2024-04-24T05:46:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 >> v1:172.21.15.123:6789/0 conn(0x7fd68006f490 legacy=0x7fd680166e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 --2- 172.21.15.123:0/2249679656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd664003b40 0x7fd664003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 --2- 172.21.15.123:0/2249679656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd664004800 0x7fd664004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 --2- 172.21.15.123:0/2249679656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6640051c0 0x7fd66400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd68006dff0 con 0x7fd664003b40 2024-04-24T05:46:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd66400c040 con 0x7fd664004800 2024-04-24T05:46:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd66400c240 con 0x7fd6640051c0 2024-04-24T05:46:06.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.832+0000 7fd67ea00640 1 --2- 172.21.15.123:0/2249679656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd664003b40 0x7fd664003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:06.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.833+0000 7fd67ea00640 1 -- 172.21.15.123:0/2249679656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6640051c0 msgr2=0x7fd66400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:06.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.833+0000 7fd67ea00640 1 --2- 172.21.15.123:0/2249679656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6640051c0 0x7fd66400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.833+0000 7fd676000640 1 --2- 172.21.15.123:0/2249679656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd664004800 0x7fd664004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:06.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.833+0000 7fd67ea00640 1 -- 172.21.15.123:0/2249679656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd664004800 msgr2=0x7fd664004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:06.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.833+0000 7fd67ea00640 1 --2- 172.21.15.123:0/2249679656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd664004800 0x7fd664004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.833+0000 7fd67ea00640 1 -- 172.21.15.123:0/2249679656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd664003520 con 0x7fd664003b40 2024-04-24T05:46:06.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.833+0000 7fd67f400640 1 -- 172.21.15.123:0/2249679656 reap_dead start 2024-04-24T05:46:06.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.833+0000 7fd676000640 1 --2- 172.21.15.123:0/2249679656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd664004800 0x7fd664004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:06.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.834+0000 7fd67ea00640 1 --2- 172.21.15.123:0/2249679656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd664003b40 0x7fd664003f40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd670009ac0 tx=0x7fd664005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38320839d8accbb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:06.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.834+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670036070 con 0x7fd664003b40 2024-04-24T05:46:06.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.834+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd670031400 con 0x7fd664003b40 2024-04-24T05:46:06.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.834+0000 7fd67e000640 1 -- 172.21.15.123:0/2249679656 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670009690 con 0x7fd664003b40 2024-04-24T05:46:06.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.835+0000 7fd685c00640 1 -- 172.21.15.123:0/2249679656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd664003b40 msgr2=0x7fd664003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:06.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.835+0000 7fd685c00640 1 --2- 172.21.15.123:0/2249679656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd664003b40 0x7fd664003f40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd670009ac0 tx=0x7fd664005830 comp rx=0 tx=0).stop 2024-04-24T05:46:06.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.835+0000 7fd685c00640 1 -- 172.21.15.123:0/2249679656 shutdown_connections 2024-04-24T05:46:06.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.835+0000 7fd685c00640 1 --2- 172.21.15.123:0/2249679656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd664003b40 0x7fd664003f40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.835+0000 7fd685c00640 1 -- 172.21.15.123:0/2249679656 >> 172.21.15.123:0/2249679656 conn(0x7fd6800b1e70 msgr2=0x7fd6800b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:06.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.835+0000 7fd685c00640 1 -- 172.21.15.123:0/2249679656 shutdown_connections 2024-04-24T05:46:06.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.835+0000 7fd685c00640 1 -- 172.21.15.123:0/2249679656 wait complete. 2024-04-24T05:46:06.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.837+0000 7fd685c00640 1 Processor -- start 2024-04-24T05:46:06.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.837+0000 7fd685c00640 1 -- start start 2024-04-24T05:46:06.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.838+0000 7fd685c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd664003b40 0x7fd6801f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:06.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.838+0000 7fd685c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6801f8a20 0x7fd6801ec550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:06.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.838+0000 7fd685c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6801eca90 0x7fd6801ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:06.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.838+0000 7fd685c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd68006d2a0 con 0x7fd6801eca90 2024-04-24T05:46:06.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.838+0000 7fd685c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd68006dc80 con 0x7fd664003b40 2024-04-24T05:46:06.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.838+0000 7fd685c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd68006c8f0 con 0x7fd6801f8a20 2024-04-24T05:46:06.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.838+0000 7fd67f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6801eca90 0x7fd6801ecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:06.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.838+0000 7fd67f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6801eca90 0x7fd6801ecf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55556/0 (socket says 172.21.15.123:55556) 2024-04-24T05:46:06.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.838+0000 7fd67f400640 1 -- 172.21.15.123:0/39200738 learned_addr learned my addr 172.21.15.123:0/39200738 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:06.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.838+0000 7fd67ea00640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd664003b40 0x7fd6801f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:06.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.838+0000 7fd676000640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6801f8a20 0x7fd6801ec550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:06.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.839+0000 7fd67f400640 1 -- 172.21.15.123:0/39200738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6801f8a20 msgr2=0x7fd6801ec550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:06.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.839+0000 7fd67f400640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6801f8a20 0x7fd6801ec550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.839+0000 7fd67f400640 1 -- 172.21.15.123:0/39200738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd664003b40 msgr2=0x7fd6801f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:06.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.839+0000 7fd67f400640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd664003b40 0x7fd6801f84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:06.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.839+0000 7fd67f400640 1 -- 172.21.15.123:0/39200738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6800cf6c0 con 0x7fd6801eca90 2024-04-24T05:46:06.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.839+0000 7fd676000640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6801f8a20 0x7fd6801ec550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:06.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.839+0000 7fd67f400640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6801eca90 0x7fd6801ecf10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd680073cb0 tx=0x7fd67800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:06.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.839+0000 7fd677400640 1 -- 172.21.15.123:0/39200738 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678004d60 con 0x7fd6801eca90 2024-04-24T05:46:06.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.839+0000 7fd677400640 1 -- 172.21.15.123:0/39200738 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd67800a610 con 0x7fd6801eca90 2024-04-24T05:46:06.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.839+0000 7fd685c00640 1 -- 172.21.15.123:0/39200738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6801f8ea0 con 0x7fd6801eca90 2024-04-24T05:46:06.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.839+0000 7fd677400640 1 -- 172.21.15.123:0/39200738 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67800aa00 con 0x7fd6801eca90 2024-04-24T05:46:06.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.839+0000 7fd685c00640 1 -- 172.21.15.123:0/39200738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6800cce40 con 0x7fd6801eca90 2024-04-24T05:46:06.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.840+0000 7fd685c00640 1 -- 172.21.15.123:0/39200738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd680068a20 con 0x7fd6801eca90 2024-04-24T05:46:06.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.841+0000 7fd677400640 1 -- 172.21.15.123:0/39200738 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd67800aba0 con 0x7fd6801eca90 2024-04-24T05:46:06.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.842+0000 7fd677400640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd65c06c940 0x7fd65c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:06.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.842+0000 7fd677400640 1 -- 172.21.15.123:0/39200738 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd67808ba10 con 0x7fd6801eca90 2024-04-24T05:46:06.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.843+0000 7fd67ea00640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd65c06c940 0x7fd65c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:06.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.843+0000 7fd67ea00640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd65c06c940 0x7fd65c06ee00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd670033000 tx=0x7fd670008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:06.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:06.847+0000 7fd677400640 1 -- 172.21.15.123:0/39200738 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6800cf6c0 con 0x7fd6801eca90 2024-04-24T05:46:07.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.088+0000 7fd685c00640 1 -- 172.21.15.123:0/39200738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6800d0d60 con 0x7fd6801eca90 2024-04-24T05:46:07.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.089+0000 7fd677400640 1 -- 172.21.15.123:0/39200738 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5189 (secure 0 0 0) 0x7fd678059b70 con 0x7fd6801eca90 2024-04-24T05:46:07.091 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:07.091 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2743935658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2743935658},{"type":"v1","addr":"172.21.15.155:6834","nonce":2743935658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1148561953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1148561953},{"type":"v1","addr":"172.21.15.123:6836","nonce":1148561953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2992052647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2992052647},{"type":"v1","addr":"172.21.15.155:6835","nonce":2992052647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T05:45:59.639548+0000","modified":"2024-04-24T05:46:04.716074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/3370584410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3370584410},{"type":"v1","addr":"172.21.15.123:6837","nonce":3370584410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T05:46:07.091 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 7 2024-04-24T05:46:07.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.092+0000 7fd685c00640 1 -- 172.21.15.123:0/39200738 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd65c06c940 msgr2=0x7fd65c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:07.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.092+0000 7fd685c00640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd65c06c940 0x7fd65c06ee00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd670033000 tx=0x7fd670008030 comp rx=0 tx=0).stop 2024-04-24T05:46:07.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.092+0000 7fd685c00640 1 -- 172.21.15.123:0/39200738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6801eca90 msgr2=0x7fd6801ecf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:07.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.092+0000 7fd685c00640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6801eca90 0x7fd6801ecf10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd680073cb0 tx=0x7fd67800cb90 comp rx=0 tx=0).stop 2024-04-24T05:46:07.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.092+0000 7fd685c00640 1 -- 172.21.15.123:0/39200738 shutdown_connections 2024-04-24T05:46:07.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.092+0000 7fd685c00640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd65c06c940 0x7fd65c06ee00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.092+0000 7fd685c00640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6801eca90 0x7fd6801ecf10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.092+0000 7fd685c00640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6801f8a20 0x7fd6801ec550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.092+0000 7fd685c00640 1 --2- 172.21.15.123:0/39200738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd664003b40 0x7fd6801f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.092+0000 7fd685c00640 1 -- 172.21.15.123:0/39200738 >> 172.21.15.123:0/39200738 conn(0x7fd6800b1e70 msgr2=0x7fd68006e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:07.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.092+0000 7fd685c00640 1 -- 172.21.15.123:0/39200738 shutdown_connections 2024-04-24T05:46:07.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.092+0000 7fd685c00640 1 -- 172.21.15.123:0/39200738 wait complete. 2024-04-24T05:46:07.112 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:07.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.313+0000 7f7f65800640 1 Processor -- start 2024-04-24T05:46:07.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.313+0000 7f7f65800640 1 -- start start 2024-04-24T05:46:07.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.315+0000 7f7f65800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7f600afef0 0x7f7f6006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:07.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.315+0000 7f7f65800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f6006f2e0 0x7f7f600b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:07.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.315+0000 7f7f65800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f6006f850 con 0x7f7f6006d7e0 2024-04-24T05:46:07.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.315+0000 7f7f65800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7f6006dc40 con 0x7f7f600afef0 2024-04-24T05:46:07.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.315+0000 7f7f65800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7f600b3af0 con 0x7f7f6006f2e0 2024-04-24T05:46:07.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.315+0000 7f7f5f400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7f6006d7e0 0x7f7f6006ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60698/0 (socket says 172.21.15.123:60698) 2024-04-24T05:46:07.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.315+0000 7f7f5f400640 1 -- 172.21.15.123:0/3467950124 learned_addr learned my addr 172.21.15.123:0/3467950124 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:07.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.315+0000 7f7f5fe00640 1 --2- 172.21.15.123:0/3467950124 >> v2:172.21.15.155:3300/0 conn(0x7f7f600afef0 0x7f7f6006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:07.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.315+0000 7f7f5ea00640 1 -- 172.21.15.123:0/3467950124 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180802224 0 0) 0x7f7f6006f850 con 0x7f7f6006d7e0 2024-04-24T05:46:07.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.315+0000 7f7f5ea00640 1 -- 172.21.15.123:0/3467950124 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f44003520 con 0x7f7f6006d7e0 2024-04-24T05:46:07.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.315+0000 7f7f64e00640 1 --2- 172.21.15.123:0/3467950124 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f6006f2e0 0x7f7f600b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:07.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5ea00640 1 -- 172.21.15.123:0/3467950124 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3207792223 0 0) 0x7f7f44003520 con 0x7f7f6006d7e0 2024-04-24T05:46:07.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5ea00640 1 -- 172.21.15.123:0/3467950124 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f6006f2e0 msgr2=0x7f7f600b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:07.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5ea00640 1 --2- 172.21.15.123:0/3467950124 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f6006f2e0 0x7f7f600b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5ea00640 1 -- 172.21.15.123:0/3467950124 >> v2:172.21.15.155:3300/0 conn(0x7f7f600afef0 msgr2=0x7f7f6006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:07.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5ea00640 1 --2- 172.21.15.123:0/3467950124 >> v2:172.21.15.155:3300/0 conn(0x7f7f600afef0 0x7f7f6006d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5ea00640 1 -- 172.21.15.123:0/3467950124 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f600b3dd0 con 0x7f7f6006d7e0 2024-04-24T05:46:07.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5ea00640 1 -- 172.21.15.123:0/3467950124 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f7f48001690 con 0x7f7f6006d7e0 2024-04-24T05:46:07.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5ea00640 1 -- 172.21.15.123:0/3467950124 >> v1:172.21.15.123:6789/0 conn(0x7f7f6006d7e0 legacy=0x7f7f6006ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:07.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5ea00640 1 --2- 172.21.15.123:0/3467950124 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f6006f2e0 0x7f7f440039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:07.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5ea00640 1 --2- 172.21.15.123:0/3467950124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f44003f80 0x7f7f44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:07.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5ea00640 1 --2- 172.21.15.123:0/3467950124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f44004d40 0x7f7f440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:07.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5ea00640 1 -- 172.21.15.123:0/3467950124 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f600b3af0 con 0x7f7f44004d40 2024-04-24T05:46:07.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5ea00640 1 -- 172.21.15.123:0/3467950124 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4400b9c0 con 0x7f7f44003f80 2024-04-24T05:46:07.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5ea00640 1 -- 172.21.15.123:0/3467950124 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4400bb40 con 0x7f7f6006f2e0 2024-04-24T05:46:07.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f64e00640 1 --2- 172.21.15.123:0/3467950124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f44004d40 0x7f7f440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:07.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.316+0000 7f7f5fe00640 1 --2- 172.21.15.123:0/3467950124 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f6006f2e0 0x7f7f440039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:07.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.317+0000 7f7f64e00640 1 -- 172.21.15.123:0/3467950124 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f6006f2e0 msgr2=0x7f7f440039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:07.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.317+0000 7f7f64e00640 1 --2- 172.21.15.123:0/3467950124 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f6006f2e0 0x7f7f440039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.317+0000 7f7f64e00640 1 -- 172.21.15.123:0/3467950124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f44003f80 msgr2=0x7f7f44004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:07.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.317+0000 7f7f64e00640 1 --2- 172.21.15.123:0/3467950124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f44003f80 0x7f7f44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.317+0000 7f7f64e00640 1 -- 172.21.15.123:0/3467950124 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f44003520 con 0x7f7f44004d40 2024-04-24T05:46:07.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.318+0000 7f7f64e00640 1 --2- 172.21.15.123:0/3467950124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f44004d40 0x7f7f440051c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009af0 tx=0x7f7f44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b802cfc6a1a771 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:07.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.318+0000 7f7f5ea00640 1 -- 172.21.15.123:0/3467950124 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58036070 con 0x7f7f44004d40 2024-04-24T05:46:07.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.318+0000 7f7f5ea00640 1 -- 172.21.15.123:0/3467950124 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f58031400 con 0x7f7f44004d40 2024-04-24T05:46:07.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.318+0000 7f7f5ea00640 1 -- 172.21.15.123:0/3467950124 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58009690 con 0x7f7f44004d40 2024-04-24T05:46:07.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.318+0000 7f7f65800640 1 -- 172.21.15.123:0/3467950124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f44004d40 msgr2=0x7f7f440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:07.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.318+0000 7f7f65800640 1 --2- 172.21.15.123:0/3467950124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f44004d40 0x7f7f440051c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009af0 tx=0x7f7f44005830 comp rx=0 tx=0).stop 2024-04-24T05:46:07.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.318+0000 7f7f64e00640 1 -- 172.21.15.123:0/3467950124 reap_dead start 2024-04-24T05:46:07.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.319+0000 7f7f65800640 1 -- 172.21.15.123:0/3467950124 shutdown_connections 2024-04-24T05:46:07.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.319+0000 7f7f65800640 1 -- 172.21.15.123:0/3467950124 >> 172.21.15.123:0/3467950124 conn(0x7f7f60072410 msgr2=0x7f7f60072820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:07.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.319+0000 7f7f65800640 1 -- 172.21.15.123:0/3467950124 shutdown_connections 2024-04-24T05:46:07.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.319+0000 7f7f65800640 1 -- 172.21.15.123:0/3467950124 wait complete. 2024-04-24T05:46:07.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.320+0000 7f7f65800640 1 Processor -- start 2024-04-24T05:46:07.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.321+0000 7f7f65800640 1 -- start start 2024-04-24T05:46:07.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.321+0000 7f7f65800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f44004d40 0x7f7f6014a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:07.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.321+0000 7f7f65800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f60152d80 0x7f7f60153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:07.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.321+0000 7f7f65800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f60156080 0x7f7f60156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:07.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.322+0000 7f7f65800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f600b02d0 con 0x7f7f44004d40 2024-04-24T05:46:07.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.322+0000 7f7f65800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6006fb30 con 0x7f7f60156080 2024-04-24T05:46:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.322+0000 7f7f65800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7f600b0450 con 0x7f7f60152d80 2024-04-24T05:46:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.322+0000 7f7f5fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f44004d40 0x7f7f6014a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.322+0000 7f7f5fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f44004d40 0x7f7f6014a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55580/0 (socket says 172.21.15.123:55580) 2024-04-24T05:46:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.322+0000 7f7f5fe00640 1 -- 172.21.15.123:0/623016765 learned_addr learned my addr 172.21.15.123:0/623016765 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.322+0000 7f7f5f400640 1 --2- 172.21.15.123:0/623016765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f60152d80 0x7f7f60153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.322+0000 7f7f64e00640 1 --2- 172.21.15.123:0/623016765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f60156080 0x7f7f60156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.322+0000 7f7f5fe00640 1 -- 172.21.15.123:0/623016765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f60152d80 msgr2=0x7f7f60153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.322+0000 7f7f5fe00640 1 --2- 172.21.15.123:0/623016765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f60152d80 0x7f7f60153200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.322+0000 7f7f5fe00640 1 -- 172.21.15.123:0/623016765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f60156080 msgr2=0x7f7f60156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.322+0000 7f7f5fe00640 1 --2- 172.21.15.123:0/623016765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f60156080 0x7f7f60156500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.322+0000 7f7f5fe00640 1 -- 172.21.15.123:0/623016765 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f60156db0 con 0x7f7f44004d40 2024-04-24T05:46:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.323+0000 7f7f5fe00640 1 --2- 172.21.15.123:0/623016765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f44004d40 0x7f7f6014a5d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000f400 tx=0x7f7f5000f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.323+0000 7f7f57e00640 1 -- 172.21.15.123:0/623016765 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50016620 con 0x7f7f44004d40 2024-04-24T05:46:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.323+0000 7f7f57e00640 1 -- 172.21.15.123:0/623016765 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f5001b070 con 0x7f7f44004d40 2024-04-24T05:46:07.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.323+0000 7f7f65800640 1 -- 172.21.15.123:0/623016765 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f60157090 con 0x7f7f44004d40 2024-04-24T05:46:07.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.323+0000 7f7f57e00640 1 -- 172.21.15.123:0/623016765 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50016a20 con 0x7f7f44004d40 2024-04-24T05:46:07.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.323+0000 7f7f65800640 1 -- 172.21.15.123:0/623016765 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6014c170 con 0x7f7f44004d40 2024-04-24T05:46:07.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.324+0000 7f7f65800640 1 -- 172.21.15.123:0/623016765 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f60075b70 con 0x7f7f44004d40 2024-04-24T05:46:07.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.325+0000 7f7f57e00640 1 -- 172.21.15.123:0/623016765 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f50005d10 con 0x7f7f44004d40 2024-04-24T05:46:07.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.327+0000 7f7f57e00640 1 --2- 172.21.15.123:0/623016765 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7f3c06c8f0 0x7f7f3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:07.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.327+0000 7f7f57e00640 1 -- 172.21.15.123:0/623016765 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7f5008cbb0 con 0x7f7f44004d40 2024-04-24T05:46:07.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.327+0000 7f7f5f400640 1 --2- 172.21.15.123:0/623016765 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7f3c06c8f0 0x7f7f3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:07.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.327+0000 7f7f5f400640 1 --2- 172.21.15.123:0/623016765 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7f3c06c8f0 0x7f7f3c06edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7f60158070 tx=0x7f7f48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:07.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.331+0000 7f7f57e00640 1 -- 172.21.15.123:0/623016765 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f50025020 con 0x7f7f44004d40 2024-04-24T05:46:07.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.533+0000 7f7f65800640 1 -- 172.21.15.123:0/623016765 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f60073000 con 0x7f7f44004d40 2024-04-24T05:46:07.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.535+0000 7f7f57e00640 1 -- 172.21.15.123:0/623016765 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f7f5005ad30 con 0x7f7f44004d40 2024-04-24T05:46:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:07.537 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:04.715857+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:07.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.538+0000 7f7f65800640 1 -- 172.21.15.123:0/623016765 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7f3c06c8f0 msgr2=0x7f7f3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.538+0000 7f7f65800640 1 --2- 172.21.15.123:0/623016765 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7f3c06c8f0 0x7f7f3c06edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7f60158070 tx=0x7f7f48009040 comp rx=0 tx=0).stop 2024-04-24T05:46:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.538+0000 7f7f65800640 1 -- 172.21.15.123:0/623016765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f44004d40 msgr2=0x7f7f6014a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.538+0000 7f7f65800640 1 --2- 172.21.15.123:0/623016765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f44004d40 0x7f7f6014a5d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000f400 tx=0x7f7f5000f8c0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.538+0000 7f7f65800640 1 -- 172.21.15.123:0/623016765 shutdown_connections 2024-04-24T05:46:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.538+0000 7f7f65800640 1 --2- 172.21.15.123:0/623016765 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7f3c06c8f0 0x7f7f3c06edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.538+0000 7f7f65800640 1 --2- 172.21.15.123:0/623016765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f60156080 0x7f7f60156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.538+0000 7f7f65800640 1 --2- 172.21.15.123:0/623016765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f60152d80 0x7f7f60153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.538+0000 7f7f65800640 1 --2- 172.21.15.123:0/623016765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f44004d40 0x7f7f6014a5d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.538+0000 7f7f65800640 1 -- 172.21.15.123:0/623016765 >> 172.21.15.123:0/623016765 conn(0x7f7f60072410 msgr2=0x7f7f6015ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.538+0000 7f7f65800640 1 -- 172.21.15.123:0/623016765 shutdown_connections 2024-04-24T05:46:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:07.538+0000 7f7f65800640 1 -- 172.21.15.123:0/623016765 wait complete. 2024-04-24T05:46:07.961 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T05:45:59.639548+0000', 'modified': '2024-04-24T05:46:04.716074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4273}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4273': {'gid': 4273, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.123:6837/3370584410', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 3370584410}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 3370584410}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4273, 'qdb_cluster': [4273]} 2024-04-24T05:46:07.961 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T05:46:07.961 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T05:46:08.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.166+0000 7f0e37000640 1 Processor -- start 2024-04-24T05:46:08.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.166+0000 7f0e37000640 1 -- start start 2024-04-24T05:46:08.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.168+0000 7f0e37000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e300ad150 0x7f0e3006cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.168+0000 7f0e37000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e3015d5d0 0x7f0e3006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.168+0000 7f0e37000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e3015da60 con 0x7f0e3006ba00 2024-04-24T05:46:08.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.168+0000 7f0e37000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0e3006be90 con 0x7f0e300ad150 2024-04-24T05:46:08.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.168+0000 7f0e37000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3006c170 con 0x7f0e3015d5d0 2024-04-24T05:46:08.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.168+0000 7f0e2fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e300ad150 0x7f0e3006cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.168+0000 7f0e2fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e300ad150 0x7f0e3006cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55592/0 (socket says 172.21.15.123:55592) 2024-04-24T05:46:08.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.168+0000 7f0e2fe00640 1 -- 172.21.15.123:0/3200400003 learned_addr learned my addr 172.21.15.123:0/3200400003 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:08.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.168+0000 7f0e2fe00640 1 -- 172.21.15.123:0/3200400003 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e3015d5d0 msgr2=0x7f0e3006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:08.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.168+0000 7f0e35000640 1 --2- 172.21.15.123:0/3200400003 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e3015d5d0 0x7f0e3006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.168+0000 7f0e2fe00640 1 --2- 172.21.15.123:0/3200400003 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e3015d5d0 0x7f0e3006dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.169+0000 7f0e2fe00640 1 -- 172.21.15.123:0/3200400003 >> v1:172.21.15.155:6789/0 conn(0x7f0e3006ba00 legacy=0x7f0e300ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:08.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.169+0000 7f0e2fe00640 1 -- 172.21.15.123:0/3200400003 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3006c450 con 0x7f0e300ad150 2024-04-24T05:46:08.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.170+0000 7f0e2fe00640 1 --2- 172.21.15.123:0/3200400003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e300ad150 0x7f0e3006cf30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009870 tx=0x7f0e2002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ebfdb5390ce7d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:08.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.170+0000 7f0e2f400640 1 -- 172.21.15.123:0/3200400003 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20037070 con 0x7f0e300ad150 2024-04-24T05:46:08.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.170+0000 7f0e2f400640 1 -- 172.21.15.123:0/3200400003 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e20032440 con 0x7f0e300ad150 2024-04-24T05:46:08.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.170+0000 7f0e2f400640 1 -- 172.21.15.123:0/3200400003 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20032810 con 0x7f0e300ad150 2024-04-24T05:46:08.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.171+0000 7f0e37000640 1 -- 172.21.15.123:0/3200400003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e300ad150 msgr2=0x7f0e3006cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:08.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.171+0000 7f0e37000640 1 --2- 172.21.15.123:0/3200400003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e300ad150 0x7f0e3006cf30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009870 tx=0x7f0e2002a210 comp rx=0 tx=0).stop 2024-04-24T05:46:08.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.171+0000 7f0e37000640 1 -- 172.21.15.123:0/3200400003 shutdown_connections 2024-04-24T05:46:08.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.171+0000 7f0e37000640 1 --2- 172.21.15.123:0/3200400003 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e3015d5d0 0x7f0e3006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.171+0000 7f0e37000640 1 --2- 172.21.15.123:0/3200400003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e300ad150 0x7f0e3006cf30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.171+0000 7f0e37000640 1 -- 172.21.15.123:0/3200400003 >> 172.21.15.123:0/3200400003 conn(0x7f0e300b1e70 msgr2=0x7f0e300b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:08.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.171+0000 7f0e37000640 1 -- 172.21.15.123:0/3200400003 shutdown_connections 2024-04-24T05:46:08.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.171+0000 7f0e37000640 1 -- 172.21.15.123:0/3200400003 wait complete. 2024-04-24T05:46:08.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.173+0000 7f0e37000640 1 Processor -- start 2024-04-24T05:46:08.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.173+0000 7f0e37000640 1 -- start start 2024-04-24T05:46:08.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.174+0000 7f0e37000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e300ad150 0x7f0e300cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.174+0000 7f0e37000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3015d5d0 0x7f0e301f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.174+0000 7f0e37000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e300979c0 0x7f0e301f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.174+0000 7f0e37000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e300af360 con 0x7f0e3015d5d0 2024-04-24T05:46:08.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.174+0000 7f0e37000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3006ca10 con 0x7f0e300ad150 2024-04-24T05:46:08.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.174+0000 7f0e37000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3014ea70 con 0x7f0e300979c0 2024-04-24T05:46:08.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.174+0000 7f0e27e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3015d5d0 0x7f0e301f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.174+0000 7f0e27e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3015d5d0 0x7f0e301f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55598/0 (socket says 172.21.15.123:55598) 2024-04-24T05:46:08.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.174+0000 7f0e27e00640 1 -- 172.21.15.123:0/805215602 learned_addr learned my addr 172.21.15.123:0/805215602 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:08.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.174+0000 7f0e27e00640 1 -- 172.21.15.123:0/805215602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e300979c0 msgr2=0x7f0e301f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:08.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.174+0000 7f0e27e00640 1 --2- 172.21.15.123:0/805215602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e300979c0 0x7f0e301f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.175+0000 7f0e27e00640 1 -- 172.21.15.123:0/805215602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e300ad150 msgr2=0x7f0e300cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:08.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.175+0000 7f0e27e00640 1 --2- 172.21.15.123:0/805215602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e300ad150 0x7f0e300cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.175+0000 7f0e27e00640 1 -- 172.21.15.123:0/805215602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e300ce0f0 con 0x7f0e3015d5d0 2024-04-24T05:46:08.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.175+0000 7f0e27e00640 1 --2- 172.21.15.123:0/805215602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3015d5d0 0x7f0e301f4670 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800c460 tx=0x7f0e1800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:08.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.175+0000 7f0e2cc00640 1 -- 172.21.15.123:0/805215602 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18016020 con 0x7f0e3015d5d0 2024-04-24T05:46:08.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.175+0000 7f0e2cc00640 1 -- 172.21.15.123:0/805215602 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e18012870 con 0x7f0e3015d5d0 2024-04-24T05:46:08.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.175+0000 7f0e37000640 1 -- 172.21.15.123:0/805215602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e300ce3d0 con 0x7f0e3015d5d0 2024-04-24T05:46:08.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.175+0000 7f0e2cc00640 1 -- 172.21.15.123:0/805215602 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18012c60 con 0x7f0e3015d5d0 2024-04-24T05:46:08.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.175+0000 7f0e37000640 1 -- 172.21.15.123:0/805215602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3014e700 con 0x7f0e3015d5d0 2024-04-24T05:46:08.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.177+0000 7f0e2cc00640 1 -- 172.21.15.123:0/805215602 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e18019020 con 0x7f0e3015d5d0 2024-04-24T05:46:08.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.178+0000 7f0e2cc00640 1 --2- 172.21.15.123:0/805215602 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0e0c06c920 0x7f0e0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.178+0000 7f0e2cc00640 1 -- 172.21.15.123:0/805215602 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0e1808bb90 con 0x7f0e3015d5d0 2024-04-24T05:46:08.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.179+0000 7f0e2fe00640 1 --2- 172.21.15.123:0/805215602 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0e0c06c920 0x7f0e0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.179+0000 7f0e39e06740 1 Processor -- start 2024-04-24T05:46:08.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.179+0000 7f0e2fe00640 1 --2- 172.21.15.123:0/805215602 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0e0c06c920 0x7f0e0c06ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e20034000 tx=0x7f0e2002c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:08.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.180+0000 7f0e39e06740 1 -- start start 2024-04-24T05:46:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.180+0000 7f0e39e06740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 0x561e907b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.180+0000 7f0e39e06740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907e78d0 0x561e907e6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.180+0000 7f0e39e06740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907e7190 0x561e907bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.180+0000 7f0e39e06740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561e90721b10 con 0x561e907e7190 2024-04-24T05:46:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.180+0000 7f0e39e06740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561e906c82b0 con 0x561e908047b0 2024-04-24T05:46:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.180+0000 7f0e39e06740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561e9071dad0 con 0x561e907e78d0 2024-04-24T05:46:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.181+0000 7f0e35000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907e7190 0x561e907bbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.181+0000 7f0e35000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907e7190 0x561e907bbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55600/0 (socket says 172.21.15.123:55600) 2024-04-24T05:46:08.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.181+0000 7f0e35000640 1 -- 172.21.15.123:0/2933265544 learned_addr learned my addr 172.21.15.123:0/2933265544 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.181+0000 7f0e2fe00640 1 --2- 172.21.15.123:0/2933265544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 0x561e907b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.181+0000 7f0e27e00640 1 --2- 172.21.15.123:0/2933265544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907e78d0 0x561e907e6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.181+0000 7f0e35000640 1 -- 172.21.15.123:0/2933265544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907e78d0 msgr2=0x561e907e6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.181+0000 7f0e35000640 1 --2- 172.21.15.123:0/2933265544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907e78d0 0x561e907e6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.181+0000 7f0e35000640 1 -- 172.21.15.123:0/2933265544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 msgr2=0x561e907b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.181+0000 7f0e35000640 1 --2- 172.21.15.123:0/2933265544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 0x561e907b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.181+0000 7f0e35000640 1 -- 172.21.15.123:0/2933265544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e904342d0 con 0x561e907e7190 2024-04-24T05:46:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.181+0000 7f0e35000640 1 --2- 172.21.15.123:0/2933265544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907e7190 0x561e907bbfa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0e28008e00 tx=0x7f0e2800d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e864a164abbb170 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.182+0000 7f0e24c00640 1 -- 172.21.15.123:0/2933265544 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28015070 con 0x561e907e7190 2024-04-24T05:46:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.182+0000 7f0e24c00640 1 -- 172.21.15.123:0/2933265544 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e28003990 con 0x561e907e7190 2024-04-24T05:46:08.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.182+0000 7f0e24c00640 1 -- 172.21.15.123:0/2933265544 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28009040 con 0x561e907e7190 2024-04-24T05:46:08.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.182+0000 7f0e39e06740 1 -- 172.21.15.123:0/2933265544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907e7190 msgr2=0x561e907bbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:08.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.182+0000 7f0e39e06740 1 --2- 172.21.15.123:0/2933265544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907e7190 0x561e907bbfa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0e28008e00 tx=0x7f0e2800d650 comp rx=0 tx=0).stop 2024-04-24T05:46:08.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.182+0000 7f0e39e06740 1 -- 172.21.15.123:0/2933265544 shutdown_connections 2024-04-24T05:46:08.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.182+0000 7f0e39e06740 1 --2- 172.21.15.123:0/2933265544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907e7190 0x561e907bbfa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.182+0000 7f0e39e06740 1 --2- 172.21.15.123:0/2933265544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907e78d0 0x561e907e6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.182+0000 7f0e39e06740 1 --2- 172.21.15.123:0/2933265544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 0x561e907b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.182+0000 7f0e39e06740 1 -- 172.21.15.123:0/2933265544 >> 172.21.15.123:0/2933265544 conn(0x561e907b93b0 msgr2=0x561e90801c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:08.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.182+0000 7f0e39e06740 1 -- 172.21.15.123:0/2933265544 shutdown_connections 2024-04-24T05:46:08.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.183+0000 7f0e39e06740 1 -- 172.21.15.123:0/2933265544 wait complete. 2024-04-24T05:46:08.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.183+0000 7f0e39e06740 1 -- start start 2024-04-24T05:46:08.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.183+0000 7f0e39e06740 1 Processor -- start 2024-04-24T05:46:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.184+0000 7f0e39e06740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907e78d0 0x561e907d44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.184+0000 7f0e39e06740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 0x561e907e9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.184+0000 7f0e39e06740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907d4eb0 0x561e907bca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.184+0000 7f0e39e06740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561e907bb560 con 0x561e907d4eb0 2024-04-24T05:46:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.184+0000 7f0e39e06740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561e906db590 con 0x561e908047b0 2024-04-24T05:46:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.184+0000 7f0e39e06740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561e907ba950 con 0x561e907e78d0 2024-04-24T05:46:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.184+0000 7f0e39e06740 10 client.0 _ll_register_callbacks cb 0x561e906be3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T05:46:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.185+0000 7f0e35000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907d4eb0 0x561e907bca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.185+0000 7f0e35000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907d4eb0 0x561e907bca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55614/0 (socket says 172.21.15.123:55614) 2024-04-24T05:46:08.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.185+0000 7f0e35000640 1 -- 172.21.15.123:0/3082257779 learned_addr learned my addr 172.21.15.123:0/3082257779 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.185+0000 7f0e2fe00640 1 --2- 172.21.15.123:0/3082257779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907e78d0 0x561e907d44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.185+0000 7f0e27e00640 1 --2- 172.21.15.123:0/3082257779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 0x561e907e9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.185+0000 7f0e35000640 1 -- 172.21.15.123:0/3082257779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907e78d0 msgr2=0x561e907d44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.185+0000 7f0e35000640 1 --2- 172.21.15.123:0/3082257779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907e78d0 0x561e907d44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.185+0000 7f0e35000640 1 -- 172.21.15.123:0/3082257779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 msgr2=0x561e907e9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.185+0000 7f0e35000640 1 --2- 172.21.15.123:0/3082257779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 0x561e907e9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.185+0000 7f0e35000640 1 -- 172.21.15.123:0/3082257779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e904fe6b0 con 0x561e907d4eb0 2024-04-24T05:46:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.185+0000 7f0e35000640 1 --2- 172.21.15.123:0/3082257779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907d4eb0 0x561e907bca40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800db50 tx=0x7f0e28010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184ff1ba6fceaa00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.185+0000 7f0e1d600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-24T05:46:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.186+0000 7f0e1d600640 1 -- 172.21.15.123:0/3082257779 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28029060 con 0x561e907d4eb0 2024-04-24T05:46:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.186+0000 7f0e1d600640 1 -- 172.21.15.123:0/3082257779 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0e28004430 con 0x561e907d4eb0 2024-04-24T05:46:08.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.186+0000 7f0e39e06740 1 -- 172.21.15.123:0/3082257779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e28003990 con 0x561e907d4eb0 2024-04-24T05:46:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.186+0000 7f0e1d600640 1 -- 172.21.15.123:0/3082257779 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f0e28004630 con 0x561e907d4eb0 2024-04-24T05:46:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.186+0000 7f0e39e06740 10 client.4389 fetch_fsmap learned FSMap version 7 2024-04-24T05:46:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.186+0000 7f0e39e06740 1 -- 172.21.15.123:0/3082257779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x561e907e8770 con 0x561e907d4eb0 2024-04-24T05:46:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.186+0000 7f0e1d600640 1 -- 172.21.15.123:0/3082257779 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e28003990 con 0x561e907d4eb0 2024-04-24T05:46:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.186+0000 7f0e1d600640 1 -- 172.21.15.123:0/3082257779 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f0e2800f040 con 0x561e907d4eb0 2024-04-24T05:46:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.186+0000 7f0e39e06740 10 client.4389 fetch_fsmap finished waiting for FSMap version 7 2024-04-24T05:46:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.186+0000 7f0e39e06740 10 client.4389 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T05:46:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.186+0000 7f0e39e06740 20 client.4389 populate_metadata read hostname 'smithi123' 2024-04-24T05:46:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.186+0000 7f0e39e06740 1 --2- 172.21.15.123:0/3082257779 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x561e907e8e10 0x561e907e52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.186+0000 7f0e39e06740 4 client.4389 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T05:46:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.186+0000 7f0e39e06740 1 -- 172.21.15.123:0/3082257779 --> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e9072f530 con 0x561e907e8e10 2024-04-24T05:46:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.187+0000 7f0e2fe00640 1 --2- 172.21.15.123:0/3082257779 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x561e907e8e10 0x561e907e52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.187+0000 7f0e1d600640 10 client.4389 ms_handle_connect on v2:172.21.15.123:6834/3370584410 2024-04-24T05:46:08.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.187+0000 7f0e2fe00640 1 --2- 172.21.15.123:0/3082257779 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x561e907e8e10 0x561e907e52e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49f0cf02198b2f05 server_cookie=36897f7a9ffb4426 in_seq=0 out_seq=0 2024-04-24T05:46:08.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.188+0000 7f0e1d600640 1 -- 172.21.15.123:0/3082257779 <== mds.0 v2:172.21.15.123:6834/3370584410 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e9072f530 con 0x561e907e8e10 2024-04-24T05:46:08.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.188+0000 7f0e1d600640 10 client.4389 handle_command_reply: tid=0 2024-04-24T05:46:08.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.188+0000 7f0e39e06740 1 client.4389 shutdown 2024-04-24T05:46:08.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.189+0000 7f0e39e06740 1 -- 172.21.15.123:0/3082257779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907d4eb0 msgr2=0x561e907bca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:08.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.189+0000 7f0e39e06740 1 --2- 172.21.15.123:0/3082257779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907d4eb0 0x561e907bca40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800db50 tx=0x7f0e28010000 comp rx=0 tx=0).stop 2024-04-24T05:46:08.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.189+0000 7f0e39e06740 1 -- 172.21.15.123:0/3082257779 shutdown_connections 2024-04-24T05:46:08.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.189+0000 7f0e39e06740 1 --2- 172.21.15.123:0/3082257779 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x561e907e8e10 0x561e907e52e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.189+0000 7f0e39e06740 1 --2- 172.21.15.123:0/3082257779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907d4eb0 0x561e907bca40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.189+0000 7f0e39e06740 1 --2- 172.21.15.123:0/3082257779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 0x561e907e9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.189+0000 7f0e39e06740 1 --2- 172.21.15.123:0/3082257779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907e78d0 0x561e907d44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.189+0000 7f0e1d600640 0 client.4389 ms_handle_reset on v2:172.21.15.123:6834/3370584410 2024-04-24T05:46:08.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.189+0000 7f0e39e06740 1 -- 172.21.15.123:0/3082257779 >> 172.21.15.123:0/3082257779 conn(0x561e907e7190 msgr2=0x561e90801c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:08.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.189+0000 7f0e39e06740 1 -- 172.21.15.123:0/3082257779 shutdown_connections 2024-04-24T05:46:08.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.189+0000 7f0e39e06740 1 -- 172.21.15.123:0/3082257779 wait complete. 2024-04-24T05:46:08.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.189+0000 7f0e39e06740 20 client.4389 trim_cache size 0 max 16384 2024-04-24T05:46:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.210+0000 7f0e39e06740 1 Processor -- start 2024-04-24T05:46:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.210+0000 7f0e39e06740 1 -- start start 2024-04-24T05:46:08.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.211+0000 7f0e39e06740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907d4eb0 0x561e907e2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.211+0000 7f0e39e06740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907e78d0 0x561e90802f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.211+0000 7f0e39e06740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 0x561e907fe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.211+0000 7f0e39e06740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e30201420 con 0x561e907e78d0 2024-04-24T05:46:08.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.211+0000 7f0e39e06740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0e300cde60 con 0x561e908047b0 2024-04-24T05:46:08.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.211+0000 7f0e39e06740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0e30200e00 con 0x561e907d4eb0 2024-04-24T05:46:08.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.211+0000 7f0e27e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907e78d0 0x561e90802f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.211+0000 7f0e27e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907e78d0 0x561e90802f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55626/0 (socket says 172.21.15.123:55626) 2024-04-24T05:46:08.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.211+0000 7f0e27e00640 1 -- 172.21.15.123:0/4128809090 learned_addr learned my addr 172.21.15.123:0/4128809090 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:08.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.212+0000 7f0e2fe00640 1 --2- 172.21.15.123:0/4128809090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907d4eb0 0x561e907e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.212+0000 7f0e35000640 1 --2- 172.21.15.123:0/4128809090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 0x561e907fe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.212+0000 7f0e27e00640 1 -- 172.21.15.123:0/4128809090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907d4eb0 msgr2=0x561e907e2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:08.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.212+0000 7f0e27e00640 1 --2- 172.21.15.123:0/4128809090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907d4eb0 0x561e907e2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.212+0000 7f0e27e00640 1 -- 172.21.15.123:0/4128809090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 msgr2=0x561e907fe600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:08.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.212+0000 7f0e27e00640 1 --2- 172.21.15.123:0/4128809090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 0x561e907fe600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.212+0000 7f0e27e00640 1 -- 172.21.15.123:0/4128809090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e907bcb00 con 0x561e907e78d0 2024-04-24T05:46:08.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.212+0000 7f0e27e00640 1 --2- 172.21.15.123:0/4128809090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907e78d0 0x561e90802f70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800ce20 tx=0x7f0e180596b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a99fe0b7dbc9cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:08.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.212+0000 7f0e1fe00640 1 -- 172.21.15.123:0/4128809090 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18088070 con 0x561e907e78d0 2024-04-24T05:46:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.212+0000 7f0e1fe00640 1 -- 172.21.15.123:0/4128809090 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e18057610 con 0x561e907e78d0 2024-04-24T05:46:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.213+0000 7f0e1fe00640 1 -- 172.21.15.123:0/4128809090 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18071070 con 0x561e907e78d0 2024-04-24T05:46:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.213+0000 7f0e39e06740 1 -- 172.21.15.123:0/4128809090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907e78d0 msgr2=0x561e90802f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.213+0000 7f0e39e06740 1 --2- 172.21.15.123:0/4128809090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907e78d0 0x561e90802f70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800ce20 tx=0x7f0e180596b0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.213+0000 7f0e39e06740 1 -- 172.21.15.123:0/4128809090 shutdown_connections 2024-04-24T05:46:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.213+0000 7f0e39e06740 1 --2- 172.21.15.123:0/4128809090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e908047b0 0x561e907fe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.214+0000 7f0e39e06740 1 --2- 172.21.15.123:0/4128809090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e907e78d0 0x561e90802f70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.214+0000 7f0e39e06740 1 --2- 172.21.15.123:0/4128809090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907d4eb0 0x561e907e2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.214+0000 7f0e39e06740 1 -- 172.21.15.123:0/4128809090 >> 172.21.15.123:0/4128809090 conn(0x561e907e7190 msgr2=0x561e907d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.214+0000 7f0e39e06740 1 -- 172.21.15.123:0/4128809090 shutdown_connections 2024-04-24T05:46:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.214+0000 7f0e39e06740 1 -- 172.21.15.123:0/4128809090 wait complete. 2024-04-24T05:46:08.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.214+0000 7f0e39e06740 1 -- start start 2024-04-24T05:46:08.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.215+0000 7f0e39e06740 1 Processor -- start 2024-04-24T05:46:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e39e06740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907d4eb0 0x561e90872850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e39e06740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e907e78d0 0x561e907b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e39e06740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e908047b0 0x561e90802f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e39e06740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561e907e8c90 con 0x561e908047b0 2024-04-24T05:46:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e39e06740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561e907ea7a0 con 0x561e907e78d0 2024-04-24T05:46:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e39e06740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561e907e92e0 con 0x561e907d4eb0 2024-04-24T05:46:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e39e06740 10 client.0 _ll_register_callbacks cb 0x7f0e28003f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T05:46:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e35000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e908047b0 0x561e90802f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e35000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e908047b0 0x561e90802f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55630/0 (socket says 172.21.15.123:55630) 2024-04-24T05:46:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e35000640 1 -- 172.21.15.123:0/3621493589 learned_addr learned my addr 172.21.15.123:0/3621493589 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:08.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e2fe00640 1 --2- 172.21.15.123:0/3621493589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907d4eb0 0x561e90872850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e35000640 1 -- 172.21.15.123:0/3621493589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907d4eb0 msgr2=0x561e90872850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:08.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e35000640 1 --2- 172.21.15.123:0/3621493589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907d4eb0 0x561e90872850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e35000640 1 -- 172.21.15.123:0/3621493589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e907e78d0 msgr2=0x561e907b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:08.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e35000640 1 --2- 172.21.15.123:0/3621493589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e907e78d0 0x561e907b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.216+0000 7f0e35000640 1 -- 172.21.15.123:0/3621493589 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e907d4740 con 0x561e908047b0 2024-04-24T05:46:08.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.217+0000 7f0e35000640 1 --2- 172.21.15.123:0/3621493589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e908047b0 0x561e90802f70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800d690 tx=0x7f0e28015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af581b3445667bba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:08.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.217+0000 7f0e1d600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-24T05:46:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.217+0000 7f0e1d600640 1 -- 172.21.15.123:0/3621493589 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2802a990 con 0x561e908047b0 2024-04-24T05:46:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.217+0000 7f0e1d600640 1 -- 172.21.15.123:0/3621493589 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0e28033de0 con 0x561e908047b0 2024-04-24T05:46:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.217+0000 7f0e39e06740 1 -- 172.21.15.123:0/3621493589 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e18057610 con 0x561e908047b0 2024-04-24T05:46:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.217+0000 7f0e1d600640 1 -- 172.21.15.123:0/3621493589 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f0e28029020 con 0x561e908047b0 2024-04-24T05:46:08.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.217+0000 7f0e39e06740 10 client.4395 fetch_fsmap learned FSMap version 7 2024-04-24T05:46:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.217+0000 7f0e39e06740 1 -- 172.21.15.123:0/3621493589 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x561e9087c090 con 0x561e908047b0 2024-04-24T05:46:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.218+0000 7f0e1d600640 1 -- 172.21.15.123:0/3621493589 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e18057610 con 0x561e908047b0 2024-04-24T05:46:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.218+0000 7f0e1d600640 1 -- 172.21.15.123:0/3621493589 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f0e28031070 con 0x561e908047b0 2024-04-24T05:46:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.218+0000 7f0e39e06740 10 client.4395 fetch_fsmap finished waiting for FSMap version 7 2024-04-24T05:46:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.218+0000 7f0e39e06740 10 client.4395 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T05:46:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.218+0000 7f0e39e06740 20 client.4395 populate_metadata read hostname 'smithi123' 2024-04-24T05:46:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.218+0000 7f0e39e06740 1 --2- 172.21.15.123:0/3621493589 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x561e9087fed0 0x561e907d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.218+0000 7f0e39e06740 4 client.4395 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T05:46:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.218+0000 7f0e39e06740 1 -- 172.21.15.123:0/3621493589 --> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] -- command(tid 0: {"prefix": "status"}) -- 0x561e907293a0 con 0x561e9087fed0 2024-04-24T05:46:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.218+0000 7f0e2fe00640 1 --2- 172.21.15.123:0/3621493589 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x561e9087fed0 0x561e907d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.219+0000 7f0e2fe00640 1 --2- 172.21.15.123:0/3621493589 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x561e9087fed0 0x561e907d2ac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bf0dfc6f62434e5 server_cookie=76b1eb825af39d38 in_seq=0 out_seq=0 2024-04-24T05:46:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.219+0000 7f0e1d600640 10 client.4395 ms_handle_connect on v2:172.21.15.123:6834/3370584410 2024-04-24T05:46:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.219+0000 7f0e1d600640 1 -- 172.21.15.123:0/3621493589 <== mds.0 v2:172.21.15.123:6834/3370584410 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x561e907293a0 con 0x561e9087fed0 2024-04-24T05:46:08.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.219+0000 7f0e1d600640 10 client.4395 handle_command_reply: tid=0 2024-04-24T05:46:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.219+0000 7f0e39e06740 1 client.4395 shutdown 2024-04-24T05:46:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.219+0000 7f0e39e06740 1 -- 172.21.15.123:0/3621493589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e908047b0 msgr2=0x561e90802f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.219+0000 7f0e39e06740 1 --2- 172.21.15.123:0/3621493589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e908047b0 0x561e90802f70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800d690 tx=0x7f0e28015000 comp rx=0 tx=0).stop 2024-04-24T05:46:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.219+0000 7f0e39e06740 1 -- 172.21.15.123:0/3621493589 shutdown_connections 2024-04-24T05:46:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.219+0000 7f0e39e06740 1 --2- 172.21.15.123:0/3621493589 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x561e9087fed0 0x561e907d2ac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.219+0000 7f0e39e06740 1 --2- 172.21.15.123:0/3621493589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e908047b0 0x561e90802f70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.220+0000 7f0e39e06740 1 --2- 172.21.15.123:0/3621493589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e907e78d0 0x561e907b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.220+0000 7f0e39e06740 1 --2- 172.21.15.123:0/3621493589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561e907d4eb0 0x561e90872850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.220+0000 7f0e1d600640 0 client.4395 ms_handle_reset on v2:172.21.15.123:6834/3370584410 2024-04-24T05:46:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.220+0000 7f0e39e06740 1 -- 172.21.15.123:0/3621493589 >> 172.21.15.123:0/3621493589 conn(0x561e907e7190 msgr2=0x561e90801d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.220+0000 7f0e39e06740 1 -- 172.21.15.123:0/3621493589 shutdown_connections 2024-04-24T05:46:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.220+0000 7f0e39e06740 1 -- 172.21.15.123:0/3621493589 wait complete. 2024-04-24T05:46:08.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.220+0000 7f0e39e06740 20 client.4395 trim_cache size 0 max 16384 2024-04-24T05:46:08.224 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-24T05:46:08.224 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "6efb8524-c0d1-4076-b7c9-7485084c7ada", 2024-04-24T05:46:08.224 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2024-04-24T05:46:08.224 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4273, 2024-04-24T05:46:08.224 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-24T05:46:08.224 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-24T05:46:08.224 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-24T05:46:08.224 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 8.0196890409999995, 2024-04-24T05:46:08.224 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 7, 2024-04-24T05:46:08.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 25, 2024-04-24T05:46:08.224 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 24, 2024-04-24T05:46:08.225 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 13.525080657 2024-04-24T05:46:08.225 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-24T05:46:08.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.221+0000 7f0e37000640 1 -- 172.21.15.123:0/805215602 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0e0c06c920 msgr2=0x7f0e0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:08.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.221+0000 7f0e37000640 1 --2- 172.21.15.123:0/805215602 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0e0c06c920 0x7f0e0c06ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e20034000 tx=0x7f0e2002c000 comp rx=0 tx=0).stop 2024-04-24T05:46:08.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.221+0000 7f0e37000640 1 -- 172.21.15.123:0/805215602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3015d5d0 msgr2=0x7f0e301f4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:08.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.221+0000 7f0e37000640 1 --2- 172.21.15.123:0/805215602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3015d5d0 0x7f0e301f4670 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800c460 tx=0x7f0e1800c920 comp rx=0 tx=0).stop 2024-04-24T05:46:08.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.221+0000 7f0e37000640 1 -- 172.21.15.123:0/805215602 shutdown_connections 2024-04-24T05:46:08.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.221+0000 7f0e37000640 1 --2- 172.21.15.123:0/805215602 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0e0c06c920 0x7f0e0c06ede0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.221+0000 7f0e37000640 1 --2- 172.21.15.123:0/805215602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e300979c0 0x7f0e301f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.221+0000 7f0e37000640 1 --2- 172.21.15.123:0/805215602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e3015d5d0 0x7f0e301f4670 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.221+0000 7f0e37000640 1 --2- 172.21.15.123:0/805215602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e300ad150 0x7f0e300cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.221+0000 7f0e37000640 1 -- 172.21.15.123:0/805215602 >> 172.21.15.123:0/805215602 conn(0x7f0e300b1e70 msgr2=0x7f0e30073770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:08.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.221+0000 7f0e37000640 1 -- 172.21.15.123:0/805215602 shutdown_connections 2024-04-24T05:46:08.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.221+0000 7f0e37000640 1 -- 172.21.15.123:0/805215602 wait complete. 2024-04-24T05:46:08.241 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T05:46:08.241 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-24T05:46:08.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T05:46:08.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.434+0000 7f7769000640 1 Processor -- start 2024-04-24T05:46:08.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.434+0000 7f7769000640 1 -- start start 2024-04-24T05:46:08.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.435+0000 7f7769000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f776415c5d0 0x7f7764160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.435+0000 7f7769000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f77640adee0 0x7f77640acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.435+0000 7f7769000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640ae2e0 0x7f77640ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.435+0000 7f7769000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7764160470 con 0x7f77640adee0 2024-04-24T05:46:08.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.435+0000 7f7769000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7764160120 con 0x7f776415c5d0 2024-04-24T05:46:08.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.435+0000 7f7769000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f776406dd80 con 0x7f77640ae2e0 2024-04-24T05:46:08.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.436+0000 7f775be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f77640adee0 0x7f77640acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.436+0000 7f775be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f77640adee0 0x7f77640acf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55644/0 (socket says 172.21.15.123:55644) 2024-04-24T05:46:08.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.436+0000 7f775be00640 1 -- 172.21.15.123:0/1948942878 learned_addr learned my addr 172.21.15.123:0/1948942878 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:08.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.436+0000 7f7763e00640 1 --2- 172.21.15.123:0/1948942878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640ae2e0 0x7f77640ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.436+0000 7f7763400640 1 --2- 172.21.15.123:0/1948942878 >> v2:172.21.15.155:3300/0 conn(0x7f776415c5d0 0x7f7764160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.436+0000 7f775be00640 1 -- 172.21.15.123:0/1948942878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640ae2e0 msgr2=0x7f77640ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:08.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.436+0000 7f775be00640 1 --2- 172.21.15.123:0/1948942878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640ae2e0 0x7f77640ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.436+0000 7f775be00640 1 -- 172.21.15.123:0/1948942878 >> v2:172.21.15.155:3300/0 conn(0x7f776415c5d0 msgr2=0x7f7764160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:08.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.436+0000 7f775be00640 1 --2- 172.21.15.123:0/1948942878 >> v2:172.21.15.155:3300/0 conn(0x7f776415c5d0 0x7f7764160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.436+0000 7f775be00640 1 -- 172.21.15.123:0/1948942878 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776406e060 con 0x7f77640adee0 2024-04-24T05:46:08.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.437+0000 7f775be00640 1 --2- 172.21.15.123:0/1948942878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77640adee0 0x7f77640acf40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77640741d0 tx=0x7f774c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b3ecede24208cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:08.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.438+0000 7f7762a00640 1 -- 172.21.15.123:0/1948942878 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c037070 con 0x7f77640adee0 2024-04-24T05:46:08.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.438+0000 7f7762a00640 1 -- 172.21.15.123:0/1948942878 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774c032440 con 0x7f77640adee0 2024-04-24T05:46:08.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.438+0000 7f7762a00640 1 -- 172.21.15.123:0/1948942878 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c032810 con 0x7f77640adee0 2024-04-24T05:46:08.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.438+0000 7f7769000640 1 -- 172.21.15.123:0/1948942878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77640adee0 msgr2=0x7f77640acf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:08.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.438+0000 7f7769000640 1 --2- 172.21.15.123:0/1948942878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77640adee0 0x7f77640acf40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77640741d0 tx=0x7f774c02a1e0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.438+0000 7f7769000640 1 -- 172.21.15.123:0/1948942878 shutdown_connections 2024-04-24T05:46:08.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.438+0000 7f7769000640 1 --2- 172.21.15.123:0/1948942878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640ae2e0 0x7f77640ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.438+0000 7f7769000640 1 --2- 172.21.15.123:0/1948942878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77640adee0 0x7f77640acf40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.438+0000 7f7769000640 1 --2- 172.21.15.123:0/1948942878 >> v2:172.21.15.155:3300/0 conn(0x7f776415c5d0 0x7f7764160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.438+0000 7f7769000640 1 -- 172.21.15.123:0/1948942878 >> 172.21.15.123:0/1948942878 conn(0x7f77640d0070 msgr2=0x7f77640d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:08.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.438+0000 7f7769000640 1 -- 172.21.15.123:0/1948942878 shutdown_connections 2024-04-24T05:46:08.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.438+0000 7f7769000640 1 -- 172.21.15.123:0/1948942878 wait complete. 2024-04-24T05:46:08.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.440+0000 7f7769000640 1 Processor -- start 2024-04-24T05:46:08.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.440+0000 7f7769000640 1 -- start start 2024-04-24T05:46:08.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.441+0000 7f7769000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77640adee0 0x7f77640b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.441+0000 7f7769000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640ae2e0 0x7f776414dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.441+0000 7f7769000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f776415c5d0 0x7f77640ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.441+0000 7f7769000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7764160fc0 con 0x7f776415c5d0 2024-04-24T05:46:08.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.441+0000 7f7769000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7764160e40 con 0x7f77640adee0 2024-04-24T05:46:08.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.441+0000 7f7769000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f77640d0b60 con 0x7f77640ae2e0 2024-04-24T05:46:08.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.441+0000 7f7763e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f776415c5d0 0x7f77640ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.441+0000 7f7763e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f776415c5d0 0x7f77640ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55652/0 (socket says 172.21.15.123:55652) 2024-04-24T05:46:08.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.441+0000 7f7763e00640 1 -- 172.21.15.123:0/2276779930 learned_addr learned my addr 172.21.15.123:0/2276779930 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.442+0000 7f775be00640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640ae2e0 0x7f776414dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.442+0000 7f7763400640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77640adee0 0x7f77640b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.442+0000 7f7763e00640 1 -- 172.21.15.123:0/2276779930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640ae2e0 msgr2=0x7f776414dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.442+0000 7f7763e00640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640ae2e0 0x7f776414dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.442+0000 7f7763e00640 1 -- 172.21.15.123:0/2276779930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77640adee0 msgr2=0x7f77640b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.442+0000 7f7763e00640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77640adee0 0x7f77640b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.442+0000 7f7763e00640 1 -- 172.21.15.123:0/2276779930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7764073100 con 0x7f776415c5d0 2024-04-24T05:46:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.442+0000 7f7763400640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77640adee0 0x7f77640b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.442+0000 7f7763e00640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f776415c5d0 0x7f77640ceda0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f775c007d80 tx=0x7f775c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.442+0000 7f775b400640 1 -- 172.21.15.123:0/2276779930 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c014420 con 0x7f776415c5d0 2024-04-24T05:46:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.443+0000 7f775b400640 1 -- 172.21.15.123:0/2276779930 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f775c019070 con 0x7f776415c5d0 2024-04-24T05:46:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.443+0000 7f7769000640 1 -- 172.21.15.123:0/2276779930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77640733e0 con 0x7f776415c5d0 2024-04-24T05:46:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.443+0000 7f775b400640 1 -- 172.21.15.123:0/2276779930 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c014800 con 0x7f776415c5d0 2024-04-24T05:46:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.443+0000 7f7769000640 1 -- 172.21.15.123:0/2276779930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77640739a0 con 0x7f776415c5d0 2024-04-24T05:46:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.444+0000 7f7769000640 1 -- 172.21.15.123:0/2276779930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7764075b70 con 0x7f776415c5d0 2024-04-24T05:46:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.445+0000 7f775b400640 1 -- 172.21.15.123:0/2276779930 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f775c0149a0 con 0x7f776415c5d0 2024-04-24T05:46:08.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.446+0000 7f775b400640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f774006c8f0 0x7f774006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:08.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.446+0000 7f775b400640 1 -- 172.21.15.123:0/2276779930 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f775c08c830 con 0x7f776415c5d0 2024-04-24T05:46:08.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.446+0000 7f7763400640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f774006c8f0 0x7f774006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:08.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.447+0000 7f7763400640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f774006c8f0 0x7f774006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7754006fd0 tx=0x7f7754008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:08.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.451+0000 7f775b400640 1 -- 172.21.15.123:0/2276779930 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f775c05a7f0 con 0x7f776415c5d0 2024-04-24T05:46:08.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.694+0000 7f7769000640 1 -- 172.21.15.123:0/2276779930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f77640cfde0 con 0x7f776415c5d0 2024-04-24T05:46:08.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.695+0000 7f775b400640 1 -- 172.21.15.123:0/2276779930 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f775c05a0f0 con 0x7f776415c5d0 2024-04-24T05:46:08.697 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T05:46:08.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.698+0000 7f7769000640 1 -- 172.21.15.123:0/2276779930 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f774006c8f0 msgr2=0x7f774006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:08.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.698+0000 7f7769000640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f774006c8f0 0x7f774006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7754006fd0 tx=0x7f7754008040 comp rx=0 tx=0).stop 2024-04-24T05:46:08.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.698+0000 7f7769000640 1 -- 172.21.15.123:0/2276779930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f776415c5d0 msgr2=0x7f77640ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:08.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.698+0000 7f7769000640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f776415c5d0 0x7f77640ceda0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f775c007d80 tx=0x7f775c00e4b0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.698+0000 7f7769000640 1 -- 172.21.15.123:0/2276779930 shutdown_connections 2024-04-24T05:46:08.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.698+0000 7f7769000640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f774006c8f0 0x7f774006edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.698+0000 7f7769000640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f776415c5d0 0x7f77640ceda0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.698+0000 7f7769000640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77640ae2e0 0x7f776414dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.698+0000 7f7769000640 1 --2- 172.21.15.123:0/2276779930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77640adee0 0x7f77640b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:08.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.698+0000 7f7769000640 1 -- 172.21.15.123:0/2276779930 >> 172.21.15.123:0/2276779930 conn(0x7f77640d0070 msgr2=0x7f77640b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:08.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.698+0000 7f7769000640 1 -- 172.21.15.123:0/2276779930 shutdown_connections 2024-04-24T05:46:08.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08.699+0000 7f7769000640 1 -- 172.21.15.123:0/2276779930 wait complete. 2024-04-24T05:46:08.718 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T05:46:08.718 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T05:46:08.718 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-04-24T05:46:08.718 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-24T05:46:08.718 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-24T05:46:08.718 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T05:46:08.718 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T05:46:08.718 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T05:46:08.718 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-24T05:46:08.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:08.719 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-04-24T05:46:08.768 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T05:46:08.768 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T05:46:08.768 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T05:46:08.768 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-24T05:46:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:08.769 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-04-24T05:46:08.769 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-04-24T05:46:08.769 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:08.769 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T05:46:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T05:46:08.769 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42618sec preferred_lft 42618sec 2024-04-24T05:46:08.770 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-04-24T05:46:08.770 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:08.771 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T05:46:08.771 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:08.771 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:46:08.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2024-04-24T05:46:08.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2024-04-24T05:46:08.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2024-04-24T05:46:08.772 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T05:46:08.772 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:46:08.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:09.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:09.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:09.239 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T05:46:09.302 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-04-24T05:46:09.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:09.305 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-04-24T05:46:09.357 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-04-24T05:46:09.357 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T05:46:09.357 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T05:46:09.357 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T05:46:09.357 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T05:46:09.358 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:09.358 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:46:09.358 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T05:46:09.358 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T05:46:09.358 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T05:46:09.358 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:46:09.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:10.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:10.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:10.064 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-24T05:46:10.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:10.111 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-04-24T05:46:10.195 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:10.195 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:46:10.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T05:46:10.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-24T05:46:10.195 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:46:10.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:10.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:10.403 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-24T05:46:10.403 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:10.403 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:46:10.404 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-24T05:46:10.404 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T05:46:10.404 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-24T05:46:10.404 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-24T05:46:10.404 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-24T05:46:10.404 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:46:10.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:10.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:10.662 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:10.662 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:46:10.662 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-04-24T05:46:10.662 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T05:46:10.663 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:46:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:10.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:10.850 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-24T05:46:10.851 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T05:46:10.851 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-24T05:46:10.891 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-24T05:46:10.891 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T05:46:10.892 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-24T05:46:10.977 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-24T05:46:11.084 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T05:46:11.126 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-24T05:46:11.127 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-24T05:46:11.127 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-24T05:46:11.127 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-24T05:46:11.127 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-24T05:46:11.127 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-24T05:46:11.127 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-24T05:46:11.127 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-24T05:46:11.127 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-24T05:46:11.127 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-24T05:46:11.127 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-24T05:46:11.127 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-24T05:46:11.128 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T05:46:11.128 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-24T05:46:11.128 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-24T05:46:11.128 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-24T05:46:11.128 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-24T05:46:11.128 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-24T05:46:11.128 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-24T05:46:11.128 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T05:46:11.128 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T05:46:11.128 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T05:46:11.128 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T05:46:11.128 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-24T05:46:11.128 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-24T05:46:11.129 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T05:46:11.129 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T05:46:11.129 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T05:46:11.129 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T05:46:11.129 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-24T05:46:11.129 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T05:46:11.129 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T05:46:11.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:11.130 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-24T05:46:11.222 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T05:46:11.222 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-24T05:46:11.227 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-24T05:46:11.294 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T05:46:11.341 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-24T05:46:11.341 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-24T05:46:11.341 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-24T05:46:11.342 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-24T05:46:11.342 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-24T05:46:11.342 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-24T05:46:11.342 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-24T05:46:11.342 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-24T05:46:11.342 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-24T05:46:11.342 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-24T05:46:11.343 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-24T05:46:11.343 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-24T05:46:11.343 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T05:46:11.343 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-24T05:46:11.343 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-24T05:46:11.343 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-24T05:46:11.343 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-24T05:46:11.343 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-24T05:46:11.344 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-24T05:46:11.344 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T05:46:11.344 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T05:46:11.344 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T05:46:11.344 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T05:46:11.344 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-24T05:46:11.344 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-24T05:46:11.345 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T05:46:11.345 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T05:46:11.345 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T05:46:11.345 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T05:46:11.345 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-24T05:46:11.346 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T05:46:11.346 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T05:46:11.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:11.348 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-24T05:46:11.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi123.stderr:2024-04-24T05:46:11.407+0000 7ff885dee480 -1 init, newargv = 0x560f6f7f21c0 newargc=15 2024-04-24T05:46:11.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi123.stderr:ceph-fuse[40366]: starting ceph client 2024-04-24T05:46:11.420 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi123.stderr:ceph-fuse[40366]: starting fuse 2024-04-24T05:46:11.430 INFO:teuthology.orchestra.run.smithi123.stdout:41 2024-04-24T05:46:11.430 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-24T05:46:11.430 DEBUG:teuthology.orchestra.run.smithi123:> sudo stdin-killer -- python3 -c ' 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> import re 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> import os 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2024-04-24T05:46:11.431 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:46:11.432 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2024-04-24T05:46:11.432 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T05:46:11.432 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T05:46:11.432 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T05:46:11.432 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2024-04-24T05:46:11.432 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2024-04-24T05:46:11.432 DEBUG:teuthology.orchestra.run.smithi123:> return f 2024-04-24T05:46:11.432 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T05:46:11.432 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:46:11.432 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.admin")) 2024-04-24T05:46:11.432 DEBUG:teuthology.orchestra.run.smithi123:> ' 2024-04-24T05:46:11.617 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.admin.40366.asok 2024-04-24T05:46:11.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:11.632 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.40366.asok 2024-04-24T05:46:11.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:11.632 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.40366.asok status 2024-04-24T05:46:11.854 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-24T05:46:11.854 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2024-04-24T05:46:11.855 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T05:46:11.855 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T05:46:11.855 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "admin", 2024-04-24T05:46:11.855 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2024-04-24T05:46:11.855 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-24T05:46:11.855 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "40366", 2024-04-24T05:46:11.855 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2024-04-24T05:46:11.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:46:11.855 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2024-04-24T05:46:11.855 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2024-04-24T05:46:11.855 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4407, 2024-04-24T05:46:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2024-04-24T05:46:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2024-04-24T05:46:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2024-04-24T05:46:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 4407 2024-04-24T05:46:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:46:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-24T05:46:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-24T05:46:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-04-24T05:46:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 694256158 2024-04-24T05:46:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:46:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:46:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-24T05:46:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-24T05:46:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-04-24T05:46:11.857 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 694256158 2024-04-24T05:46:11.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:46:11.857 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.4407 192.168.0.1:0/694256158", 2024-04-24T05:46:11.857 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.1:0/694256158", 2024-04-24T05:46:11.857 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2024-04-24T05:46:11.857 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 7, 2024-04-24T05:46:11.857 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 25, 2024-04-24T05:46:11.857 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2024-04-24T05:46:11.857 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2024-04-24T05:46:11.857 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2024-04-24T05:46:11.857 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-24T05:46:11.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:11.869 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2024-04-24T05:46:11.869 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-24T05:46:11.919 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2024-04-24T05:46:11.919 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-24T05:46:11.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:11.920 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-24T05:46:12.002 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-24T05:46:12.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T05:46:12.135 INFO:teuthology.orchestra.run.smithi123.stdout:# file: . 2024-04-24T05:46:12.136 INFO:teuthology.orchestra.run.smithi123.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-24T05:46:12.136 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:12.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:12.145 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:46:12.146 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T05:46:12.191 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:12.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.432+0000 7fa7c9400640 1 Processor -- start 2024-04-24T05:46:12.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.432+0000 7fa7c9400640 1 -- start start 2024-04-24T05:46:12.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.434+0000 7fa7c9400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7c406e660 0x7fa7c406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:12.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.434+0000 7fa7c9400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c40d0de0 con 0x7fa7c40cf7b0 2024-04-24T05:46:12.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.434+0000 7fa7c9400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c406dc80 con 0x7fa7c40ad950 2024-04-24T05:46:12.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.434+0000 7fa7c9400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c415ec00 con 0x7fa7c406e660 2024-04-24T05:46:12.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.435+0000 7fa7c8a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa7c40cf7b0 0x7fa7c406c110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43456/0 (socket says 172.21.15.123:43456) 2024-04-24T05:46:12.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.435+0000 7fa7c8a00640 1 -- 172.21.15.123:0/745579936 learned_addr learned my addr 172.21.15.123:0/745579936 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:12.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.435+0000 7fa7c3e00640 1 --2- 172.21.15.123:0/745579936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7c406e660 0x7fa7c406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:12.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.435+0000 7fa7c2a00640 1 -- 172.21.15.123:0/745579936 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3388460695 0 0) 0x7fa7c40d0de0 con 0x7fa7c40cf7b0 2024-04-24T05:46:12.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.435+0000 7fa7c2a00640 1 -- 172.21.15.123:0/745579936 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7a8003520 con 0x7fa7c40cf7b0 2024-04-24T05:46:12.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.435+0000 7fa7c2a00640 1 -- 172.21.15.123:0/745579936 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1586800538 0 0) 0x7fa7a8003520 con 0x7fa7c40cf7b0 2024-04-24T05:46:12.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.435+0000 7fa7c2a00640 1 -- 172.21.15.123:0/745579936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7c406e660 msgr2=0x7fa7c406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:12.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.435+0000 7fa7c2a00640 1 --2- 172.21.15.123:0/745579936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7c406e660 0x7fa7c406ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:12.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.435+0000 7fa7c2a00640 1 -- 172.21.15.123:0/745579936 >> v1:172.21.15.155:6789/0 conn(0x7fa7c40ad950 legacy=0x7fa7c406ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:12.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.435+0000 7fa7c2a00640 1 -- 172.21.15.123:0/745579936 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c40ae0a0 con 0x7fa7c40cf7b0 2024-04-24T05:46:12.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.435+0000 7fa7c2a00640 1 -- 172.21.15.123:0/745579936 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7fa7bc0019f0 con 0x7fa7c40cf7b0 2024-04-24T05:46:12.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.436+0000 7fa7c2a00640 1 -- 172.21.15.123:0/745579936 >> v1:172.21.15.123:6789/0 conn(0x7fa7c40cf7b0 legacy=0x7fa7c406c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:12.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.436+0000 7fa7c2a00640 1 --2- 172.21.15.123:0/745579936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7c406e660 0x7fa7a80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:12.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.436+0000 7fa7c2a00640 1 --2- 172.21.15.123:0/745579936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7a8003f60 0x7fa7a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:12.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.436+0000 7fa7c2a00640 1 --2- 172.21.15.123:0/745579936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a8004d40 0x7fa7a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:12.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.436+0000 7fa7c2a00640 1 -- 172.21.15.123:0/745579936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a800ba70 con 0x7fa7a8004d40 2024-04-24T05:46:12.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.436+0000 7fa7c2a00640 1 -- 172.21.15.123:0/745579936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a800bc20 con 0x7fa7a8003f60 2024-04-24T05:46:12.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.436+0000 7fa7c2a00640 1 -- 172.21.15.123:0/745579936 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a800be20 con 0x7fa7c406e660 2024-04-24T05:46:12.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.436+0000 7fa7c8a00640 1 --2- 172.21.15.123:0/745579936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a8004d40 0x7fa7a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:12.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.436+0000 7fa7c3400640 1 --2- 172.21.15.123:0/745579936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7a8003f60 0x7fa7a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:12.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.436+0000 7fa7c8a00640 1 -- 172.21.15.123:0/745579936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7c406e660 msgr2=0x7fa7a80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:12.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.436+0000 7fa7c8a00640 1 --2- 172.21.15.123:0/745579936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7c406e660 0x7fa7a80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:12.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.436+0000 7fa7c8a00640 1 -- 172.21.15.123:0/745579936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7a8003f60 msgr2=0x7fa7a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.436+0000 7fa7c8a00640 1 --2- 172.21.15.123:0/745579936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7a8003f60 0x7fa7a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.436+0000 7fa7c8a00640 1 -- 172.21.15.123:0/745579936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a8003520 con 0x7fa7a8004d40 2024-04-24T05:46:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.437+0000 7fa7c8a00640 1 --2- 172.21.15.123:0/745579936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a8004d40 0x7fa7a80051c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00a850 tx=0x7fa7a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4d9f5304fba0a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.438+0000 7fa7c2a00640 1 -- 172.21.15.123:0/745579936 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc00a4a0 con 0x7fa7a8004d40 2024-04-24T05:46:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.438+0000 7fa7c2a00640 1 -- 172.21.15.123:0/745579936 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7bc02f070 con 0x7fa7a8004d40 2024-04-24T05:46:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.438+0000 7fa7c2a00640 1 -- 172.21.15.123:0/745579936 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc02bd20 con 0x7fa7a8004d40 2024-04-24T05:46:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.438+0000 7fa7c9400640 1 -- 172.21.15.123:0/745579936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a8004d40 msgr2=0x7fa7a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.438+0000 7fa7c9400640 1 --2- 172.21.15.123:0/745579936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a8004d40 0x7fa7a80051c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00a850 tx=0x7fa7a8005830 comp rx=0 tx=0).stop 2024-04-24T05:46:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.438+0000 7fa7c8a00640 1 -- 172.21.15.123:0/745579936 reap_dead start 2024-04-24T05:46:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.438+0000 7fa7c9400640 1 -- 172.21.15.123:0/745579936 shutdown_connections 2024-04-24T05:46:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.438+0000 7fa7c9400640 1 -- 172.21.15.123:0/745579936 >> 172.21.15.123:0/745579936 conn(0x7fa7c40d0070 msgr2=0x7fa7c40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.439+0000 7fa7c9400640 1 -- 172.21.15.123:0/745579936 shutdown_connections 2024-04-24T05:46:12.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.439+0000 7fa7c9400640 1 -- 172.21.15.123:0/745579936 wait complete. 2024-04-24T05:46:12.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.440+0000 7fa7c9400640 1 Processor -- start 2024-04-24T05:46:12.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.441+0000 7fa7c9400640 1 -- start start 2024-04-24T05:46:12.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a8004d40 0x7fa7c40cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:12.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c9400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7c40cb270 0x7fa7c40c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:12.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c9400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7c40c98f0 0x7fa7c40c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:12.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c9400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c406ea40 con 0x7fa7a8004d40 2024-04-24T05:46:12.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c9400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c40ade50 con 0x7fa7c40c98f0 2024-04-24T05:46:12.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c9400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c406ebc0 con 0x7fa7c40cb270 2024-04-24T05:46:12.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a8004d40 0x7fa7c40cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:12.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a8004d40 0x7fa7c40cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60166/0 (socket says 172.21.15.123:60166) 2024-04-24T05:46:12.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c3e00640 1 -- 172.21.15.123:0/397193740 learned_addr learned my addr 172.21.15.123:0/397193740 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:12.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c3e00640 1 -- 172.21.15.123:0/397193740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7c40cb270 msgr2=0x7fa7c40c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:12.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c3400640 1 --2- 172.21.15.123:0/397193740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7c40cb270 0x7fa7c40c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:12.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c8a00640 1 --2- 172.21.15.123:0/397193740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7c40c98f0 0x7fa7c40c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c3e00640 1 --2- 172.21.15.123:0/397193740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7c40cb270 0x7fa7c40c93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c3e00640 1 -- 172.21.15.123:0/397193740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7c40c98f0 msgr2=0x7fa7c40c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c3e00640 1 --2- 172.21.15.123:0/397193740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7c40c98f0 0x7fa7c40c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.442+0000 7fa7c3e00640 1 -- 172.21.15.123:0/397193740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c415c090 con 0x7fa7a8004d40 2024-04-24T05:46:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.443+0000 7fa7c3e00640 1 --2- 172.21.15.123:0/397193740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a8004d40 0x7fa7c40cbbf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400e850 tx=0x7fa7b400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.443+0000 7fa7bbe00640 1 -- 172.21.15.123:0/397193740 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4017070 con 0x7fa7a8004d40 2024-04-24T05:46:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.443+0000 7fa7bbe00640 1 -- 172.21.15.123:0/397193740 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b400c8e0 con 0x7fa7a8004d40 2024-04-24T05:46:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.443+0000 7fa7bbe00640 1 -- 172.21.15.123:0/397193740 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400ccb0 con 0x7fa7a8004d40 2024-04-24T05:46:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.443+0000 7fa7c9400640 1 -- 172.21.15.123:0/397193740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c415c370 con 0x7fa7a8004d40 2024-04-24T05:46:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.443+0000 7fa7c9400640 1 -- 172.21.15.123:0/397193740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c415c930 con 0x7fa7a8004d40 2024-04-24T05:46:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.444+0000 7fa7c9400640 1 -- 172.21.15.123:0/397193740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c4075b70 con 0x7fa7a8004d40 2024-04-24T05:46:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.445+0000 7fa7bbe00640 1 -- 172.21.15.123:0/397193740 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7b401a070 con 0x7fa7a8004d40 2024-04-24T05:46:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.446+0000 7fa7bbe00640 1 --2- 172.21.15.123:0/397193740 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa7a406c8f0 0x7fa7a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.446+0000 7fa7bbe00640 1 -- 172.21.15.123:0/397193740 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa7b408bcc0 con 0x7fa7a8004d40 2024-04-24T05:46:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.446+0000 7fa7c3400640 1 --2- 172.21.15.123:0/397193740 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa7a406c8f0 0x7fa7a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.447+0000 7fa7c3400640 1 --2- 172.21.15.123:0/397193740 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa7a406c8f0 0x7fa7a406edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0007fd0 tx=0x7fa7b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:12.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.451+0000 7fa7bbe00640 1 -- 172.21.15.123:0/397193740 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa7b4092080 con 0x7fa7a8004d40 2024-04-24T05:46:12.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.657+0000 7fa7c9400640 1 -- 172.21.15.123:0/397193740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7c4074e70 con 0x7fa7a8004d40 2024-04-24T05:46:12.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.660+0000 7fa7bbe00640 1 -- 172.21.15.123:0/397193740 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7fa7b4059d10 con 0x7fa7a8004d40 2024-04-24T05:46:12.661 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:12.661 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:04.715857+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:12.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.663+0000 7fa7c9400640 1 -- 172.21.15.123:0/397193740 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa7a406c8f0 msgr2=0x7fa7a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:12.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.663+0000 7fa7c9400640 1 --2- 172.21.15.123:0/397193740 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa7a406c8f0 0x7fa7a406edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0007fd0 tx=0x7fa7b0009040 comp rx=0 tx=0).stop 2024-04-24T05:46:12.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.663+0000 7fa7c9400640 1 -- 172.21.15.123:0/397193740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a8004d40 msgr2=0x7fa7c40cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:12.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.663+0000 7fa7c9400640 1 --2- 172.21.15.123:0/397193740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a8004d40 0x7fa7c40cbbf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400e850 tx=0x7fa7b400ed10 comp rx=0 tx=0).stop 2024-04-24T05:46:12.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.663+0000 7fa7c9400640 1 -- 172.21.15.123:0/397193740 shutdown_connections 2024-04-24T05:46:12.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.663+0000 7fa7c9400640 1 --2- 172.21.15.123:0/397193740 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa7a406c8f0 0x7fa7a406edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:12.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.663+0000 7fa7c9400640 1 --2- 172.21.15.123:0/397193740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7c40c98f0 0x7fa7c40c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:12.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.664+0000 7fa7c9400640 1 --2- 172.21.15.123:0/397193740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7c40cb270 0x7fa7c40c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:12.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.664+0000 7fa7c9400640 1 --2- 172.21.15.123:0/397193740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a8004d40 0x7fa7c40cbbf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:12.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.664+0000 7fa7c9400640 1 -- 172.21.15.123:0/397193740 >> 172.21.15.123:0/397193740 conn(0x7fa7c40d0070 msgr2=0x7fa7c415aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:12.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.664+0000 7fa7c9400640 1 -- 172.21.15.123:0/397193740 shutdown_connections 2024-04-24T05:46:12.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.664+0000 7fa7c9400640 1 -- 172.21.15.123:0/397193740 wait complete. 2024-04-24T05:46:12.682 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:46:12.683 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:46:12.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.887+0000 7ff0f6e00640 1 Processor -- start 2024-04-24T05:46:12.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.888+0000 7ff0f6e00640 1 -- start start 2024-04-24T05:46:12.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.889+0000 7ff0f6e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff0f00ae990 0x7ff0f00d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:12.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.889+0000 7ff0f6e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff0f006c410 0x7ff0f00ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:12.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.889+0000 7ff0f6e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0f00d0740 con 0x7ff0f00ce610 2024-04-24T05:46:12.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.889+0000 7ff0f6e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff0f006dcb0 con 0x7ff0f00ae990 2024-04-24T05:46:12.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.889+0000 7ff0f6e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff0f006e270 con 0x7ff0f006c410 2024-04-24T05:46:12.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.889+0000 7ff0efe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff0f00ae990 0x7ff0f00d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:12.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.889+0000 7ff0efe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff0f00ae990 0x7ff0f00d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60178/0 (socket says 172.21.15.123:60178) 2024-04-24T05:46:12.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.889+0000 7ff0efe00640 1 -- 172.21.15.123:0/4084894118 learned_addr learned my addr 172.21.15.123:0/4084894118 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:12.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.890+0000 7ff0ef400640 1 --2- 172.21.15.123:0/4084894118 >> v2:172.21.15.155:3300/0 conn(0x7ff0f006c410 0x7ff0f00ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:12.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.890+0000 7ff0efe00640 1 -- 172.21.15.123:0/4084894118 >> v2:172.21.15.155:3300/0 conn(0x7ff0f006c410 msgr2=0x7ff0f00ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:12.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.890+0000 7ff0efe00640 1 --2- 172.21.15.123:0/4084894118 >> v2:172.21.15.155:3300/0 conn(0x7ff0f006c410 0x7ff0f00ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:12.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.890+0000 7ff0efe00640 1 -- 172.21.15.123:0/4084894118 >> v1:172.21.15.155:6789/0 conn(0x7ff0f00ce610 legacy=0x7ff0f015f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:12.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.890+0000 7ff0efe00640 1 -- 172.21.15.123:0/4084894118 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f00d0a20 con 0x7ff0f00ae990 2024-04-24T05:46:12.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.890+0000 7ff0ef400640 1 --2- 172.21.15.123:0/4084894118 >> v2:172.21.15.155:3300/0 conn(0x7ff0f006c410 0x7ff0f00ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:12.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.891+0000 7ff0efe00640 1 --2- 172.21.15.123:0/4084894118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f00ae990 0x7ff0f00d0160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0071c00 tx=0x7ff0d802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c284abea1e564b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:12.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.891+0000 7ff0eea00640 1 -- 172.21.15.123:0/4084894118 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8037070 con 0x7ff0f00ae990 2024-04-24T05:46:12.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.891+0000 7ff0eea00640 1 -- 172.21.15.123:0/4084894118 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d8032440 con 0x7ff0f00ae990 2024-04-24T05:46:12.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.891+0000 7ff0eea00640 1 -- 172.21.15.123:0/4084894118 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8032810 con 0x7ff0f00ae990 2024-04-24T05:46:12.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.892+0000 7ff0f6e00640 1 -- 172.21.15.123:0/4084894118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f00ae990 msgr2=0x7ff0f00d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:12.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.892+0000 7ff0f6e00640 1 --2- 172.21.15.123:0/4084894118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f00ae990 0x7ff0f00d0160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0071c00 tx=0x7ff0d802a230 comp rx=0 tx=0).stop 2024-04-24T05:46:12.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.892+0000 7ff0f6e00640 1 -- 172.21.15.123:0/4084894118 shutdown_connections 2024-04-24T05:46:12.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.892+0000 7ff0f6e00640 1 --2- 172.21.15.123:0/4084894118 >> v2:172.21.15.155:3300/0 conn(0x7ff0f006c410 0x7ff0f00ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:12.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.892+0000 7ff0f6e00640 1 --2- 172.21.15.123:0/4084894118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f00ae990 0x7ff0f00d0160 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:12.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.892+0000 7ff0f6e00640 1 -- 172.21.15.123:0/4084894118 >> 172.21.15.123:0/4084894118 conn(0x7ff0f00b51f0 msgr2=0x7ff0f00b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:12.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.892+0000 7ff0f6e00640 1 -- 172.21.15.123:0/4084894118 shutdown_connections 2024-04-24T05:46:12.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.892+0000 7ff0f6e00640 1 -- 172.21.15.123:0/4084894118 wait complete. 2024-04-24T05:46:12.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.894+0000 7ff0f6e00640 1 Processor -- start 2024-04-24T05:46:12.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.894+0000 7ff0f6e00640 1 -- start start 2024-04-24T05:46:12.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.895+0000 7ff0f6e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f006c410 0x7ff0f00c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:12.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.895+0000 7ff0f6e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f00ae990 0x7ff0f00bab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:12.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.895+0000 7ff0f6e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0f00ce610 0x7ff0f00ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:12.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.895+0000 7ff0f6e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f006c870 con 0x7ff0f00ae990 2024-04-24T05:46:12.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.895+0000 7ff0f6e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f00aed70 con 0x7ff0f00ce610 2024-04-24T05:46:12.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.895+0000 7ff0f6e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f00c9e70 con 0x7ff0f006c410 2024-04-24T05:46:12.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.895+0000 7ff0ef400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f00ae990 0x7ff0f00bab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:12.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.895+0000 7ff0ef400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f00ae990 0x7ff0f00bab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60186/0 (socket says 172.21.15.123:60186) 2024-04-24T05:46:12.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.895+0000 7ff0ef400640 1 -- 172.21.15.123:0/1878508803 learned_addr learned my addr 172.21.15.123:0/1878508803 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:12.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.896+0000 7ff0efe00640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f006c410 0x7ff0f00c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:12.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.896+0000 7ff0f4e00640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0f00ce610 0x7ff0f00ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:12.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.896+0000 7ff0ef400640 1 -- 172.21.15.123:0/1878508803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f006c410 msgr2=0x7ff0f00c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:12.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.896+0000 7ff0ef400640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f006c410 0x7ff0f00c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:12.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.896+0000 7ff0ef400640 1 -- 172.21.15.123:0/1878508803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0f00ce610 msgr2=0x7ff0f00ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:12.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.896+0000 7ff0ef400640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0f00ce610 0x7ff0f00ad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:12.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.896+0000 7ff0ef400640 1 -- 172.21.15.123:0/1878508803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f00c9610 con 0x7ff0f00ae990 2024-04-24T05:46:12.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.896+0000 7ff0efe00640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f006c410 0x7ff0f00c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:12.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.896+0000 7ff0ef400640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f00ae990 0x7ff0f00bab00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000cf60 tx=0x7ff0e000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:12.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.896+0000 7ff0e7e00640 1 -- 172.21.15.123:0/1878508803 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e001a070 con 0x7ff0f00ae990 2024-04-24T05:46:12.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.897+0000 7ff0e7e00640 1 -- 172.21.15.123:0/1878508803 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e000d840 con 0x7ff0f00ae990 2024-04-24T05:46:12.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.897+0000 7ff0f6e00640 1 -- 172.21.15.123:0/1878508803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f00c98b0 con 0x7ff0f00ae990 2024-04-24T05:46:12.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.897+0000 7ff0e7e00640 1 -- 172.21.15.123:0/1878508803 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e000dc10 con 0x7ff0f00ae990 2024-04-24T05:46:12.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.897+0000 7ff0f6e00640 1 -- 172.21.15.123:0/1878508803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f015bcb0 con 0x7ff0f00ae990 2024-04-24T05:46:12.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.898+0000 7ff0f6e00640 1 -- 172.21.15.123:0/1878508803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f0068a20 con 0x7ff0f00ae990 2024-04-24T05:46:12.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.899+0000 7ff0e7e00640 1 -- 172.21.15.123:0/1878508803 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0e001d070 con 0x7ff0f00ae990 2024-04-24T05:46:12.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.900+0000 7ff0e7e00640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff0cc06c8f0 0x7ff0cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:12.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.900+0000 7ff0e7e00640 1 -- 172.21.15.123:0/1878508803 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff0e008c680 con 0x7ff0f00ae990 2024-04-24T05:46:12.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.900+0000 7ff0efe00640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff0cc06c8f0 0x7ff0cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:12.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.901+0000 7ff0efe00640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff0cc06c8f0 0x7ff0cc06edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8034000 tx=0x7ff0d802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:12.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:12.904+0000 7ff0e7e00640 1 -- 172.21.15.123:0/1878508803 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff0e005a080 con 0x7ff0f00ae990 2024-04-24T05:46:13.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.104+0000 7ff0f6e00640 1 -- 172.21.15.123:0/1878508803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff0f015f5f0 con 0x7ff0f00ae990 2024-04-24T05:46:13.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.105+0000 7ff0e7e00640 1 -- 172.21.15.123:0/1878508803 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7ff0e005e380 con 0x7ff0f00ae990 2024-04-24T05:46:13.106 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:46:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.108+0000 7ff0f6e00640 1 -- 172.21.15.123:0/1878508803 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff0cc06c8f0 msgr2=0x7ff0cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.108+0000 7ff0f6e00640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff0cc06c8f0 0x7ff0cc06edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8034000 tx=0x7ff0d802e000 comp rx=0 tx=0).stop 2024-04-24T05:46:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.108+0000 7ff0f6e00640 1 -- 172.21.15.123:0/1878508803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f00ae990 msgr2=0x7ff0f00bab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.108+0000 7ff0f6e00640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f00ae990 0x7ff0f00bab00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000cf60 tx=0x7ff0e000cf90 comp rx=0 tx=0).stop 2024-04-24T05:46:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.108+0000 7ff0f6e00640 1 -- 172.21.15.123:0/1878508803 shutdown_connections 2024-04-24T05:46:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.108+0000 7ff0f6e00640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff0cc06c8f0 0x7ff0cc06edb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.108+0000 7ff0f6e00640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0f00ce610 0x7ff0f00ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.108+0000 7ff0f6e00640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f00ae990 0x7ff0f00bab00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.108+0000 7ff0f6e00640 1 --2- 172.21.15.123:0/1878508803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f006c410 0x7ff0f00c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.108+0000 7ff0f6e00640 1 -- 172.21.15.123:0/1878508803 >> 172.21.15.123:0/1878508803 conn(0x7ff0f00b51f0 msgr2=0x7ff0f00cb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.108+0000 7ff0f6e00640 1 -- 172.21.15.123:0/1878508803 shutdown_connections 2024-04-24T05:46:13.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.108+0000 7ff0f6e00640 1 -- 172.21.15.123:0/1878508803 wait complete. 2024-04-24T05:46:13.128 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:46:13.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.331+0000 7fe87e200640 1 Processor -- start 2024-04-24T05:46:13.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.332+0000 7fe87e200640 1 -- start start 2024-04-24T05:46:13.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.333+0000 7fe87e200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8780ae990 0x7fe8780d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:13.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.333+0000 7fe87e200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe8780ce7e0 0x7fe87806e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:13.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.333+0000 7fe87e200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8780d0740 con 0x7fe87806c410 2024-04-24T05:46:13.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.333+0000 7fe87e200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe87806dcb0 con 0x7fe8780ce7e0 2024-04-24T05:46:13.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.333+0000 7fe87e200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe87806e270 con 0x7fe8780ae990 2024-04-24T05:46:13.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.333+0000 7fe877e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe8780ce7e0 0x7fe87806e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:13.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.333+0000 7fe877e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe8780ce7e0 0x7fe87806e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60188/0 (socket says 172.21.15.123:60188) 2024-04-24T05:46:13.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.333+0000 7fe877e00640 1 -- 172.21.15.123:0/1594253548 learned_addr learned my addr 172.21.15.123:0/1594253548 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:13.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.334+0000 7fe877400640 1 --2- 172.21.15.123:0/1594253548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8780ae990 0x7fe8780d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:13.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.334+0000 7fe877e00640 1 -- 172.21.15.123:0/1594253548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8780ae990 msgr2=0x7fe8780d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:13.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.334+0000 7fe877e00640 1 --2- 172.21.15.123:0/1594253548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8780ae990 0x7fe8780d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.334+0000 7fe877e00640 1 -- 172.21.15.123:0/1594253548 >> v1:172.21.15.155:6789/0 conn(0x7fe87806c410 legacy=0x7fe8780ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:13.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.334+0000 7fe877e00640 1 -- 172.21.15.123:0/1594253548 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8780d0a20 con 0x7fe8780ce7e0 2024-04-24T05:46:13.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.335+0000 7fe877e00640 1 --2- 172.21.15.123:0/1594253548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8780ce7e0 0x7fe87806e700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe87000d150 tx=0x7fe87002c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a93f15403807c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:13.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.335+0000 7fe876000640 1 -- 172.21.15.123:0/1594253548 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870009600 con 0x7fe8780ce7e0 2024-04-24T05:46:13.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.335+0000 7fe876000640 1 -- 172.21.15.123:0/1594253548 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe870037070 con 0x7fe8780ce7e0 2024-04-24T05:46:13.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.335+0000 7fe876000640 1 -- 172.21.15.123:0/1594253548 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8700099c0 con 0x7fe8780ce7e0 2024-04-24T05:46:13.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.336+0000 7fe87e200640 1 -- 172.21.15.123:0/1594253548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8780ce7e0 msgr2=0x7fe87806e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:13.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.336+0000 7fe87e200640 1 --2- 172.21.15.123:0/1594253548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8780ce7e0 0x7fe87806e700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe87000d150 tx=0x7fe87002c930 comp rx=0 tx=0).stop 2024-04-24T05:46:13.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.336+0000 7fe87e200640 1 -- 172.21.15.123:0/1594253548 shutdown_connections 2024-04-24T05:46:13.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.336+0000 7fe87e200640 1 --2- 172.21.15.123:0/1594253548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8780ce7e0 0x7fe87806e700 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.336+0000 7fe87e200640 1 --2- 172.21.15.123:0/1594253548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8780ae990 0x7fe8780d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.336+0000 7fe87e200640 1 -- 172.21.15.123:0/1594253548 >> 172.21.15.123:0/1594253548 conn(0x7fe8780b51f0 msgr2=0x7fe8780b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:13.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.336+0000 7fe87e200640 1 -- 172.21.15.123:0/1594253548 shutdown_connections 2024-04-24T05:46:13.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.336+0000 7fe87e200640 1 -- 172.21.15.123:0/1594253548 wait complete. 2024-04-24T05:46:13.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.338+0000 7fe87e200640 1 Processor -- start 2024-04-24T05:46:13.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.338+0000 7fe87e200640 1 -- start start 2024-04-24T05:46:13.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.339+0000 7fe87e200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe87806c410 0x7fe8781efe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:13.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.339+0000 7fe87e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8780ae990 0x7fe87809dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:13.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.339+0000 7fe87e200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8780ce7e0 0x7fe87815b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:13.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.339+0000 7fe87e200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe87806c870 con 0x7fe8780ae990 2024-04-24T05:46:13.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.339+0000 7fe87e200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8780aed70 con 0x7fe87806c410 2024-04-24T05:46:13.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.339+0000 7fe87e200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe87809e200 con 0x7fe8780ce7e0 2024-04-24T05:46:13.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.339+0000 7fe876a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8780ae990 0x7fe87809dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:13.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.339+0000 7fe876a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8780ae990 0x7fe87809dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60198/0 (socket says 172.21.15.123:60198) 2024-04-24T05:46:13.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.339+0000 7fe876a00640 1 -- 172.21.15.123:0/1138808233 learned_addr learned my addr 172.21.15.123:0/1138808233 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:13.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.339+0000 7fe877400640 1 --2- 172.21.15.123:0/1138808233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe87806c410 0x7fe8781efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:13.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.340+0000 7fe877e00640 1 --2- 172.21.15.123:0/1138808233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8780ce7e0 0x7fe87815b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:13.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.340+0000 7fe876a00640 1 -- 172.21.15.123:0/1138808233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8780ce7e0 msgr2=0x7fe87815b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:13.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.340+0000 7fe876a00640 1 --2- 172.21.15.123:0/1138808233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8780ce7e0 0x7fe87815b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.340+0000 7fe876a00640 1 -- 172.21.15.123:0/1138808233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe87806c410 msgr2=0x7fe8781efe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:13.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.340+0000 7fe876a00640 1 --2- 172.21.15.123:0/1138808233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe87806c410 0x7fe8781efe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.340+0000 7fe876a00640 1 -- 172.21.15.123:0/1138808233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8781f0590 con 0x7fe8780ae990 2024-04-24T05:46:13.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.340+0000 7fe876a00640 1 --2- 172.21.15.123:0/1138808233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8780ae990 0x7fe87809dcc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c450 tx=0x7fe86000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:13.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.340+0000 7fe86f400640 1 -- 172.21.15.123:0/1138808233 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8600052d0 con 0x7fe8780ae990 2024-04-24T05:46:13.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.340+0000 7fe86f400640 1 -- 172.21.15.123:0/1138808233 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe860005470 con 0x7fe8780ae990 2024-04-24T05:46:13.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.340+0000 7fe87e200640 1 -- 172.21.15.123:0/1138808233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8781f0870 con 0x7fe8780ae990 2024-04-24T05:46:13.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.340+0000 7fe86f400640 1 -- 172.21.15.123:0/1138808233 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860012870 con 0x7fe8780ae990 2024-04-24T05:46:13.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.340+0000 7fe87e200640 1 -- 172.21.15.123:0/1138808233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8781f0e30 con 0x7fe8780ae990 2024-04-24T05:46:13.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.342+0000 7fe87e200640 1 -- 172.21.15.123:0/1138808233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe878068a20 con 0x7fe8780ae990 2024-04-24T05:46:13.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.343+0000 7fe86f400640 1 -- 172.21.15.123:0/1138808233 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe860012a10 con 0x7fe8780ae990 2024-04-24T05:46:13.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.344+0000 7fe86f400640 1 --2- 172.21.15.123:0/1138808233 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe85406c8f0 0x7fe85406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:13.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.344+0000 7fe86f400640 1 -- 172.21.15.123:0/1138808233 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe86008baa0 con 0x7fe8780ae990 2024-04-24T05:46:13.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.344+0000 7fe877400640 1 --2- 172.21.15.123:0/1138808233 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe85406c8f0 0x7fe85406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:13.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.344+0000 7fe877400640 1 --2- 172.21.15.123:0/1138808233 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe85406c8f0 0x7fe85406edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe868006fd0 tx=0x7fe868008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:13.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.348+0000 7fe86f400640 1 -- 172.21.15.123:0/1138808233 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe860059b80 con 0x7fe8780ae990 2024-04-24T05:46:13.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.545+0000 7fe87e200640 1 -- 172.21.15.123:0/1138808233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe87815ff00 con 0x7fe8780ae990 2024-04-24T05:46:13.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.546+0000 7fe86f400640 1 -- 172.21.15.123:0/1138808233 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fe860059480 con 0x7fe8780ae990 2024-04-24T05:46:13.547 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:13.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.549+0000 7fe87e200640 1 -- 172.21.15.123:0/1138808233 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe85406c8f0 msgr2=0x7fe85406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:13.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.549+0000 7fe87e200640 1 --2- 172.21.15.123:0/1138808233 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe85406c8f0 0x7fe85406edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe868006fd0 tx=0x7fe868008040 comp rx=0 tx=0).stop 2024-04-24T05:46:13.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.549+0000 7fe87e200640 1 -- 172.21.15.123:0/1138808233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8780ae990 msgr2=0x7fe87809dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:13.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.549+0000 7fe87e200640 1 --2- 172.21.15.123:0/1138808233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8780ae990 0x7fe87809dcc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c450 tx=0x7fe86000c910 comp rx=0 tx=0).stop 2024-04-24T05:46:13.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.549+0000 7fe87e200640 1 -- 172.21.15.123:0/1138808233 shutdown_connections 2024-04-24T05:46:13.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.549+0000 7fe87e200640 1 --2- 172.21.15.123:0/1138808233 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe85406c8f0 0x7fe85406edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.549+0000 7fe87e200640 1 --2- 172.21.15.123:0/1138808233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8780ce7e0 0x7fe87815b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.549+0000 7fe87e200640 1 --2- 172.21.15.123:0/1138808233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8780ae990 0x7fe87809dcc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.549+0000 7fe87e200640 1 --2- 172.21.15.123:0/1138808233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe87806c410 0x7fe8781efe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.549+0000 7fe87e200640 1 -- 172.21.15.123:0/1138808233 >> 172.21.15.123:0/1138808233 conn(0x7fe8780b51f0 msgr2=0x7fe8780ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:13.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.549+0000 7fe87e200640 1 -- 172.21.15.123:0/1138808233 shutdown_connections 2024-04-24T05:46:13.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.549+0000 7fe87e200640 1 -- 172.21.15.123:0/1138808233 wait complete. 2024-04-24T05:46:13.567 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:46:13.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.766+0000 7f5553400640 1 Processor -- start 2024-04-24T05:46:13.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.767+0000 7f5553400640 1 -- start start 2024-04-24T05:46:13.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.768+0000 7f5553400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f554c06c510 0x7f554c071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:13.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.768+0000 7f5553400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f554c06c8f0 0x7f554c06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:13.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.768+0000 7f5553400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f554c0cf170 con 0x7f554c0ccbb0 2024-04-24T05:46:13.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.768+0000 7f5553400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f554c0cf450 con 0x7f554c06c8f0 2024-04-24T05:46:13.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.768+0000 7f5553400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f554c0cf730 con 0x7f554c06c510 2024-04-24T05:46:13.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.768+0000 7f5551400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f554c0ccbb0 0x7f554c0d0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43472/0 (socket says 172.21.15.123:43472) 2024-04-24T05:46:13.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.768+0000 7f5551400640 1 -- 172.21.15.123:0/3898451976 learned_addr learned my addr 172.21.15.123:0/3898451976 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:13.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.768+0000 7f5543e00640 1 --2- 172.21.15.123:0/3898451976 >> v2:172.21.15.123:3300/0 conn(0x7f554c06c8f0 0x7f554c06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:13.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.769+0000 7f5550a00640 1 --2- 172.21.15.123:0/3898451976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f554c06c510 0x7f554c071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:13.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.769+0000 7f5543e00640 1 -- 172.21.15.123:0/3898451976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f554c06c510 msgr2=0x7f554c071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:13.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.769+0000 7f5543e00640 1 --2- 172.21.15.123:0/3898451976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f554c06c510 0x7f554c071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.769+0000 7f5543e00640 1 -- 172.21.15.123:0/3898451976 >> v1:172.21.15.123:6789/0 conn(0x7f554c0ccbb0 legacy=0x7f554c0d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:13.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.769+0000 7f5543e00640 1 -- 172.21.15.123:0/3898451976 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554c0cfa10 con 0x7f554c06c8f0 2024-04-24T05:46:13.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.770+0000 7f5543e00640 1 --2- 172.21.15.123:0/3898451976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554c06c8f0 0x7f554c06cd50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f55340097a0 tx=0x7f553402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4810d6267274eb26 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:13.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.770+0000 7f554be00640 1 -- 172.21.15.123:0/3898451976 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534037070 con 0x7f554c06c8f0 2024-04-24T05:46:13.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.770+0000 7f554be00640 1 -- 172.21.15.123:0/3898451976 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5534032440 con 0x7f554c06c8f0 2024-04-24T05:46:13.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.770+0000 7f554be00640 1 -- 172.21.15.123:0/3898451976 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534032810 con 0x7f554c06c8f0 2024-04-24T05:46:13.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.771+0000 7f5553400640 1 -- 172.21.15.123:0/3898451976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554c06c8f0 msgr2=0x7f554c06cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:13.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.771+0000 7f5553400640 1 --2- 172.21.15.123:0/3898451976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554c06c8f0 0x7f554c06cd50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f55340097a0 tx=0x7f553402a180 comp rx=0 tx=0).stop 2024-04-24T05:46:13.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.771+0000 7f5553400640 1 -- 172.21.15.123:0/3898451976 shutdown_connections 2024-04-24T05:46:13.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.771+0000 7f5553400640 1 --2- 172.21.15.123:0/3898451976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554c06c8f0 0x7f554c06cd50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.771+0000 7f5553400640 1 --2- 172.21.15.123:0/3898451976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f554c06c510 0x7f554c071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.771+0000 7f5553400640 1 -- 172.21.15.123:0/3898451976 >> 172.21.15.123:0/3898451976 conn(0x7f554c0b51f0 msgr2=0x7f554c0b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:13.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.771+0000 7f5553400640 1 -- 172.21.15.123:0/3898451976 shutdown_connections 2024-04-24T05:46:13.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.771+0000 7f5553400640 1 -- 172.21.15.123:0/3898451976 wait complete. 2024-04-24T05:46:13.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.773+0000 7f5553400640 1 Processor -- start 2024-04-24T05:46:13.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.773+0000 7f5553400640 1 -- start start 2024-04-24T05:46:13.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.774+0000 7f5553400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f554c06c510 0x7f554c0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:13.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.774+0000 7f5553400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f554c06c8f0 0x7f554c0c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:13.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.774+0000 7f5553400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554c0ccbb0 0x7f554c0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:13.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.774+0000 7f5553400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f554c0cee80 con 0x7f554c0ccbb0 2024-04-24T05:46:13.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.774+0000 7f5553400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f554c0ced00 con 0x7f554c06c510 2024-04-24T05:46:13.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.774+0000 7f5553400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f554c0cd040 con 0x7f554c06c8f0 2024-04-24T05:46:13.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.774+0000 7f5551400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554c0ccbb0 0x7f554c0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:13.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.774+0000 7f5551400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554c0ccbb0 0x7f554c0cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60220/0 (socket says 172.21.15.123:60220) 2024-04-24T05:46:13.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.774+0000 7f5551400640 1 -- 172.21.15.123:0/1961203243 learned_addr learned my addr 172.21.15.123:0/1961203243 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:13.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.774+0000 7f5550a00640 1 --2- 172.21.15.123:0/1961203243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f554c06c510 0x7f554c0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:13.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.775+0000 7f5543e00640 1 --2- 172.21.15.123:0/1961203243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f554c06c8f0 0x7f554c0c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:13.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.775+0000 7f5551400640 1 -- 172.21.15.123:0/1961203243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f554c06c8f0 msgr2=0x7f554c0c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:13.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.775+0000 7f5551400640 1 --2- 172.21.15.123:0/1961203243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f554c06c8f0 0x7f554c0c4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.775+0000 7f5551400640 1 -- 172.21.15.123:0/1961203243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f554c06c510 msgr2=0x7f554c0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:13.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.775+0000 7f5551400640 1 --2- 172.21.15.123:0/1961203243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f554c06c510 0x7f554c0c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.775+0000 7f5551400640 1 -- 172.21.15.123:0/1961203243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554c15cb80 con 0x7f554c0ccbb0 2024-04-24T05:46:13.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.775+0000 7f5551400640 1 --2- 172.21.15.123:0/1961203243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554c0ccbb0 0x7f554c0cb230 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f554400ca20 tx=0x7f554400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:13.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.775+0000 7f5549600640 1 -- 172.21.15.123:0/1961203243 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5544017070 con 0x7f554c0ccbb0 2024-04-24T05:46:13.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.775+0000 7f5549600640 1 -- 172.21.15.123:0/1961203243 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f554400abd0 con 0x7f554c0ccbb0 2024-04-24T05:46:13.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.775+0000 7f5553400640 1 -- 172.21.15.123:0/1961203243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554c15ce60 con 0x7f554c0ccbb0 2024-04-24T05:46:13.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.775+0000 7f5549600640 1 -- 172.21.15.123:0/1961203243 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5544012400 con 0x7f554c0ccbb0 2024-04-24T05:46:13.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.775+0000 7f5553400640 1 -- 172.21.15.123:0/1961203243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554c0c51f0 con 0x7f554c0ccbb0 2024-04-24T05:46:13.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.777+0000 7f5553400640 1 -- 172.21.15.123:0/1961203243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f554c068a20 con 0x7f554c0ccbb0 2024-04-24T05:46:13.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.778+0000 7f5549600640 1 -- 172.21.15.123:0/1961203243 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f554401a070 con 0x7f554c0ccbb0 2024-04-24T05:46:13.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.778+0000 7f5549600640 1 --2- 172.21.15.123:0/1961203243 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f552806c920 0x7f552806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:13.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.779+0000 7f5549600640 1 -- 172.21.15.123:0/1961203243 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f554408bb40 con 0x7f554c0ccbb0 2024-04-24T05:46:13.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.779+0000 7f5550a00640 1 --2- 172.21.15.123:0/1961203243 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f552806c920 0x7f552806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:13.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.779+0000 7f5550a00640 1 --2- 172.21.15.123:0/1961203243 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f552806c920 0x7f552806ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f553c006fd0 tx=0x7f553c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:13.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.783+0000 7f5549600640 1 -- 172.21.15.123:0/1961203243 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5544059c30 con 0x7f554c0ccbb0 2024-04-24T05:46:13.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.982+0000 7f5553400640 1 -- 172.21.15.123:0/1961203243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f554c0c6310 con 0x7f554c0ccbb0 2024-04-24T05:46:13.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.983+0000 7f5549600640 1 -- 172.21.15.123:0/1961203243 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f5544059530 con 0x7f554c0ccbb0 2024-04-24T05:46:13.983 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:46:13.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.985+0000 7f5553400640 1 -- 172.21.15.123:0/1961203243 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f552806c920 msgr2=0x7f552806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:13.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.985+0000 7f5553400640 1 --2- 172.21.15.123:0/1961203243 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f552806c920 0x7f552806ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f553c006fd0 tx=0x7f553c008040 comp rx=0 tx=0).stop 2024-04-24T05:46:13.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.985+0000 7f5553400640 1 -- 172.21.15.123:0/1961203243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554c0ccbb0 msgr2=0x7f554c0cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:13.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.985+0000 7f5553400640 1 --2- 172.21.15.123:0/1961203243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554c0ccbb0 0x7f554c0cb230 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f554400ca20 tx=0x7f554400cee0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.985+0000 7f5553400640 1 -- 172.21.15.123:0/1961203243 shutdown_connections 2024-04-24T05:46:13.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.985+0000 7f5553400640 1 --2- 172.21.15.123:0/1961203243 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f552806c920 0x7f552806ede0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.985+0000 7f5553400640 1 --2- 172.21.15.123:0/1961203243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f554c0ccbb0 0x7f554c0cb230 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.985+0000 7f5553400640 1 --2- 172.21.15.123:0/1961203243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f554c06c8f0 0x7f554c0c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.985+0000 7f5553400640 1 --2- 172.21.15.123:0/1961203243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f554c06c510 0x7f554c0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.985+0000 7f5553400640 1 -- 172.21.15.123:0/1961203243 >> 172.21.15.123:0/1961203243 conn(0x7f554c0b51f0 msgr2=0x7f554c0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.985+0000 7f5553400640 1 -- 172.21.15.123:0/1961203243 shutdown_connections 2024-04-24T05:46:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:13.986+0000 7f5553400640 1 -- 172.21.15.123:0/1961203243 wait complete. 2024-04-24T05:46:14.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:46:14.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.208+0000 7f258d800640 1 Processor -- start 2024-04-24T05:46:14.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.208+0000 7f258d800640 1 -- start start 2024-04-24T05:46:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.209+0000 7f258d800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f258815c470 0x7f25880aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.209+0000 7f258d800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f25880af040 0x7f2588164f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.209+0000 7f258d800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f258806dbc0 con 0x7f258815b980 2024-04-24T05:46:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.209+0000 7f258d800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2588071b80 con 0x7f25880af040 2024-04-24T05:46:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.210+0000 7f258d800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f258806de30 con 0x7f258815c470 2024-04-24T05:46:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.210+0000 7f258ce00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f25880af040 0x7f2588164f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.210+0000 7f258ce00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f25880af040 0x7f2588164f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60228/0 (socket says 172.21.15.123:60228) 2024-04-24T05:46:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.210+0000 7f258ce00640 1 -- 172.21.15.123:0/428262357 learned_addr learned my addr 172.21.15.123:0/428262357 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.210+0000 7f2587400640 1 --2- 172.21.15.123:0/428262357 >> v2:172.21.15.155:3300/0 conn(0x7f258815c470 0x7f25880aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.210+0000 7f258ce00640 1 -- 172.21.15.123:0/428262357 >> v2:172.21.15.155:3300/0 conn(0x7f258815c470 msgr2=0x7f25880aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.210+0000 7f258ce00640 1 --2- 172.21.15.123:0/428262357 >> v2:172.21.15.155:3300/0 conn(0x7f258815c470 0x7f25880aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.210+0000 7f258ce00640 1 -- 172.21.15.123:0/428262357 >> v1:172.21.15.155:6789/0 conn(0x7f258815b980 legacy=0x7f258815bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:14.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.210+0000 7f258ce00640 1 -- 172.21.15.123:0/428262357 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258806e110 con 0x7f25880af040 2024-04-24T05:46:14.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.211+0000 7f258ce00640 1 --2- 172.21.15.123:0/428262357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25880af040 0x7f2588164f30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2588050710 tx=0x7f258002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0fd257e93687fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:14.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.212+0000 7f2586a00640 1 -- 172.21.15.123:0/428262357 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580039070 con 0x7f25880af040 2024-04-24T05:46:14.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.212+0000 7f2586a00640 1 -- 172.21.15.123:0/428262357 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f258000b2e0 con 0x7f25880af040 2024-04-24T05:46:14.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.212+0000 7f2586a00640 1 -- 172.21.15.123:0/428262357 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580009820 con 0x7f25880af040 2024-04-24T05:46:14.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.212+0000 7f258d800640 1 -- 172.21.15.123:0/428262357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25880af040 msgr2=0x7f2588164f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:14.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.212+0000 7f258d800640 1 --2- 172.21.15.123:0/428262357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25880af040 0x7f2588164f30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2588050710 tx=0x7f258002c860 comp rx=0 tx=0).stop 2024-04-24T05:46:14.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.212+0000 7f258d800640 1 -- 172.21.15.123:0/428262357 shutdown_connections 2024-04-24T05:46:14.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.212+0000 7f258d800640 1 --2- 172.21.15.123:0/428262357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25880af040 0x7f2588164f30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.212+0000 7f258d800640 1 --2- 172.21.15.123:0/428262357 >> v2:172.21.15.155:3300/0 conn(0x7f258815c470 0x7f25880aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.212+0000 7f258d800640 1 -- 172.21.15.123:0/428262357 >> 172.21.15.123:0/428262357 conn(0x7f25880b1cf0 msgr2=0x7f25880b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:14.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.213+0000 7f258d800640 1 -- 172.21.15.123:0/428262357 shutdown_connections 2024-04-24T05:46:14.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.213+0000 7f258d800640 1 -- 172.21.15.123:0/428262357 wait complete. 2024-04-24T05:46:14.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.214+0000 7f258d800640 1 Processor -- start 2024-04-24T05:46:14.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.214+0000 7f258d800640 1 -- start start 2024-04-24T05:46:14.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.215+0000 7f258d800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25880af040 0x7f2588158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.215+0000 7f258d800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258815b980 0x7f25880c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.215+0000 7f258d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f258815c470 0x7f258814f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.215+0000 7f258d800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25880af4d0 con 0x7f258815c470 2024-04-24T05:46:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.215+0000 7f258d800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f25880af830 con 0x7f25880af040 2024-04-24T05:46:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.215+0000 7f258d800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f258806f920 con 0x7f258815b980 2024-04-24T05:46:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.216+0000 7f258ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f258815c470 0x7f258814f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.216+0000 7f258ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f258815c470 0x7f258814f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60232/0 (socket says 172.21.15.123:60232) 2024-04-24T05:46:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.216+0000 7f258ce00640 1 -- 172.21.15.123:0/1431329706 learned_addr learned my addr 172.21.15.123:0/1431329706 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.216+0000 7f2587e00640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25880af040 0x7f2588158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.216+0000 7f258ce00640 1 -- 172.21.15.123:0/1431329706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258815b980 msgr2=0x7f25880c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.216+0000 7f2587400640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258815b980 0x7f25880c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.216+0000 7f258ce00640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258815b980 0x7f25880c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.216+0000 7f258ce00640 1 -- 172.21.15.123:0/1431329706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25880af040 msgr2=0x7f2588158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:14.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.216+0000 7f258ce00640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25880af040 0x7f2588158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.216+0000 7f258ce00640 1 -- 172.21.15.123:0/1431329706 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25880c99d0 con 0x7f258815c470 2024-04-24T05:46:14.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.216+0000 7f2587e00640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25880af040 0x7f2588158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:14.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.217+0000 7f258ce00640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f258815c470 0x7f258814f500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f258002ce50 tx=0x7f2580004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:14.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.217+0000 7f257fe00640 1 -- 172.21.15.123:0/1431329706 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580039040 con 0x7f258815c470 2024-04-24T05:46:14.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.217+0000 7f257fe00640 1 -- 172.21.15.123:0/1431329706 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2580009370 con 0x7f258815c470 2024-04-24T05:46:14.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.217+0000 7f258d800640 1 -- 172.21.15.123:0/1431329706 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25880c9410 con 0x7f258815c470 2024-04-24T05:46:14.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.217+0000 7f257fe00640 1 -- 172.21.15.123:0/1431329706 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258000b720 con 0x7f258815c470 2024-04-24T05:46:14.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.217+0000 7f258d800640 1 -- 172.21.15.123:0/1431329706 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2588155f20 con 0x7f258815c470 2024-04-24T05:46:14.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.218+0000 7f258d800640 1 -- 172.21.15.123:0/1431329706 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2588068a20 con 0x7f258815c470 2024-04-24T05:46:14.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.219+0000 7f257fe00640 1 -- 172.21.15.123:0/1431329706 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f258003f890 con 0x7f258815c470 2024-04-24T05:46:14.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.220+0000 7f257fe00640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f256806c8f0 0x7f256806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.220+0000 7f257fe00640 1 -- 172.21.15.123:0/1431329706 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f25800ac790 con 0x7f258815c470 2024-04-24T05:46:14.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.220+0000 7f2587e00640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f256806c8f0 0x7f256806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.221+0000 7f2587e00640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f256806c8f0 0x7f256806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2588159220 tx=0x7f2578009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:14.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.225+0000 7f257fe00640 1 -- 172.21.15.123:0/1431329706 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f258004d020 con 0x7f258815c470 2024-04-24T05:46:14.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.427+0000 7f258d800640 1 -- 172.21.15.123:0/1431329706 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f25880ca240 con 0x7f258815c470 2024-04-24T05:46:14.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.428+0000 7f257fe00640 1 -- 172.21.15.123:0/1431329706 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f258007a870 con 0x7f258815c470 2024-04-24T05:46:14.428 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:14.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.430+0000 7f258d800640 1 -- 172.21.15.123:0/1431329706 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f256806c8f0 msgr2=0x7f256806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:14.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.430+0000 7f258d800640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f256806c8f0 0x7f256806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2588159220 tx=0x7f2578009040 comp rx=0 tx=0).stop 2024-04-24T05:46:14.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.430+0000 7f258d800640 1 -- 172.21.15.123:0/1431329706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f258815c470 msgr2=0x7f258814f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:14.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.430+0000 7f258d800640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f258815c470 0x7f258814f500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f258002ce50 tx=0x7f2580004a40 comp rx=0 tx=0).stop 2024-04-24T05:46:14.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.430+0000 7f258d800640 1 -- 172.21.15.123:0/1431329706 shutdown_connections 2024-04-24T05:46:14.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.430+0000 7f258d800640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f256806c8f0 0x7f256806edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.430+0000 7f258d800640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f258815c470 0x7f258814f500 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.430+0000 7f258d800640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258815b980 0x7f25880c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.430+0000 7f258d800640 1 --2- 172.21.15.123:0/1431329706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25880af040 0x7f2588158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.430+0000 7f258d800640 1 -- 172.21.15.123:0/1431329706 >> 172.21.15.123:0/1431329706 conn(0x7f25880b1cf0 msgr2=0x7f258815bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:14.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.431+0000 7f258d800640 1 -- 172.21.15.123:0/1431329706 shutdown_connections 2024-04-24T05:46:14.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.431+0000 7f258d800640 1 -- 172.21.15.123:0/1431329706 wait complete. 2024-04-24T05:46:14.448 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T05:46:14.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.647+0000 7f53f4a00640 1 Processor -- start 2024-04-24T05:46:14.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.648+0000 7f53f4a00640 1 -- start start 2024-04-24T05:46:14.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.649+0000 7f53f4a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53f015e9a0 0x7f53f00b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.649+0000 7f53f4a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f53f00d2260 0x7f53f00d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.649+0000 7f53f4a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f00d2820 con 0x7f53f00d4550 2024-04-24T05:46:14.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.649+0000 7f53f4a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f53f00d2b00 con 0x7f53f00d2260 2024-04-24T05:46:14.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.649+0000 7f53f4a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f53f00d2de0 con 0x7f53f015e9a0 2024-04-24T05:46:14.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.649+0000 7f53ee800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f53f00d4550 0x7f53f00d49e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43488/0 (socket says 172.21.15.123:43488) 2024-04-24T05:46:14.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.649+0000 7f53ee800640 1 -- 172.21.15.123:0/56860593 learned_addr learned my addr 172.21.15.123:0/56860593 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:14.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53ed400640 1 --2- 172.21.15.123:0/56860593 >> v2:172.21.15.155:3300/0 conn(0x7f53f00d2260 0x7f53f00d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53ede00640 1 --2- 172.21.15.123:0/56860593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53f015e9a0 0x7f53f00b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 -- 172.21.15.123:0/56860593 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3364638049 0 0) 0x7f53f00d2820 con 0x7f53f00d4550 2024-04-24T05:46:14.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 -- 172.21.15.123:0/56860593 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53d4003520 con 0x7f53f00d4550 2024-04-24T05:46:14.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 -- 172.21.15.123:0/56860593 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3774731160 0 0) 0x7f53d4003520 con 0x7f53f00d4550 2024-04-24T05:46:14.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 -- 172.21.15.123:0/56860593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53f015e9a0 msgr2=0x7f53f00b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:14.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 --2- 172.21.15.123:0/56860593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53f015e9a0 0x7f53f00b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 -- 172.21.15.123:0/56860593 >> v2:172.21.15.155:3300/0 conn(0x7f53f00d2260 msgr2=0x7f53f00d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:14.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 --2- 172.21.15.123:0/56860593 >> v2:172.21.15.155:3300/0 conn(0x7f53f00d2260 0x7f53f00d19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 -- 172.21.15.123:0/56860593 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f00d30c0 con 0x7f53f00d4550 2024-04-24T05:46:14.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 -- 172.21.15.123:0/56860593 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f53e8001670 con 0x7f53f00d4550 2024-04-24T05:46:14.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 -- 172.21.15.123:0/56860593 >> v1:172.21.15.123:6789/0 conn(0x7f53f00d4550 legacy=0x7f53f00d49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:14.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 --2- 172.21.15.123:0/56860593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53d40039b0 0x7f53d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 --2- 172.21.15.123:0/56860593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53d4004800 0x7f53d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 --2- 172.21.15.123:0/56860593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53f015e9a0 0x7f53d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 -- 172.21.15.123:0/56860593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53f00d2b00 con 0x7f53d40039b0 2024-04-24T05:46:14.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 -- 172.21.15.123:0/56860593 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53f00d2de0 con 0x7f53d4004800 2024-04-24T05:46:14.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.650+0000 7f53eca00640 1 -- 172.21.15.123:0/56860593 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f53d400ba10 con 0x7f53f015e9a0 2024-04-24T05:46:14.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.651+0000 7f53ede00640 1 --2- 172.21.15.123:0/56860593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53d40039b0 0x7f53d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.651+0000 7f53ed400640 1 --2- 172.21.15.123:0/56860593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53d4004800 0x7f53d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.651+0000 7f53ee800640 1 --2- 172.21.15.123:0/56860593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53f015e9a0 0x7f53d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.651+0000 7f53ede00640 1 -- 172.21.15.123:0/56860593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53f015e9a0 msgr2=0x7f53d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:14.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.651+0000 7f53ede00640 1 --2- 172.21.15.123:0/56860593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53f015e9a0 0x7f53d40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.651+0000 7f53ede00640 1 -- 172.21.15.123:0/56860593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53d4004800 msgr2=0x7f53d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:14.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.651+0000 7f53ede00640 1 --2- 172.21.15.123:0/56860593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53d4004800 0x7f53d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.651+0000 7f53ede00640 1 -- 172.21.15.123:0/56860593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d4003520 con 0x7f53d40039b0 2024-04-24T05:46:14.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.652+0000 7f53ede00640 1 --2- 172.21.15.123:0/56860593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53d40039b0 0x7f53d4003dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009af0 tx=0x7f53d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b737c66f80ad33db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:14.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.652+0000 7f53eca00640 1 -- 172.21.15.123:0/56860593 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8036070 con 0x7f53d40039b0 2024-04-24T05:46:14.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.652+0000 7f53eca00640 1 -- 172.21.15.123:0/56860593 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d8031400 con 0x7f53d40039b0 2024-04-24T05:46:14.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.652+0000 7f53eca00640 1 -- 172.21.15.123:0/56860593 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8009690 con 0x7f53d40039b0 2024-04-24T05:46:14.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.653+0000 7f53f4a00640 1 -- 172.21.15.123:0/56860593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53d40039b0 msgr2=0x7f53d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:14.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.653+0000 7f53f4a00640 1 --2- 172.21.15.123:0/56860593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53d40039b0 0x7f53d4003dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009af0 tx=0x7f53d4005830 comp rx=0 tx=0).stop 2024-04-24T05:46:14.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.653+0000 7f53ee800640 1 -- 172.21.15.123:0/56860593 reap_dead start 2024-04-24T05:46:14.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.653+0000 7f53f4a00640 1 -- 172.21.15.123:0/56860593 shutdown_connections 2024-04-24T05:46:14.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.653+0000 7f53f4a00640 1 -- 172.21.15.123:0/56860593 >> 172.21.15.123:0/56860593 conn(0x7f53f009d940 msgr2=0x7f53f00b9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:14.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.653+0000 7f53f4a00640 1 -- 172.21.15.123:0/56860593 shutdown_connections 2024-04-24T05:46:14.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.653+0000 7f53f4a00640 1 -- 172.21.15.123:0/56860593 wait complete. 2024-04-24T05:46:14.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.655+0000 7f53f4a00640 1 Processor -- start 2024-04-24T05:46:14.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.655+0000 7f53f4a00640 1 -- start start 2024-04-24T05:46:14.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.656+0000 7f53f4a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40039b0 0x7f53f00cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.656+0000 7f53f4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f0149870 0x7f53f00c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.656+0000 7f53f4a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f014b900 0x7f53f014bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.656+0000 7f53f4a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53f015ed80 con 0x7f53f0149870 2024-04-24T05:46:14.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.656+0000 7f53f4a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53f00cfc80 con 0x7f53f014b900 2024-04-24T05:46:14.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.656+0000 7f53f4a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f53f00b11c0 con 0x7f53d40039b0 2024-04-24T05:46:14.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.656+0000 7f53ed400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f0149870 0x7f53f00c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.656+0000 7f53ed400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f0149870 0x7f53f00c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60260/0 (socket says 172.21.15.123:60260) 2024-04-24T05:46:14.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.656+0000 7f53ed400640 1 -- 172.21.15.123:0/2539947038 learned_addr learned my addr 172.21.15.123:0/2539947038 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:14.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53ede00640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40039b0 0x7f53f00cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53ee800640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f014b900 0x7f53f014bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53ed400640 1 -- 172.21.15.123:0/2539947038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40039b0 msgr2=0x7f53f00cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:14.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53ed400640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40039b0 0x7f53f00cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53ed400640 1 -- 172.21.15.123:0/2539947038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f014b900 msgr2=0x7f53f014bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:14.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53ed400640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f014b900 0x7f53f014bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53ed400640 1 -- 172.21.15.123:0/2539947038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f0149cf0 con 0x7f53f0149870 2024-04-24T05:46:14.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53ede00640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40039b0 0x7f53f00cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:14.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53ee800640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f014b900 0x7f53f014bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:14.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53ed400640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f0149870 0x7f53f00c78e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f53e0007f30 tx=0x7f53e0011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:14.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53e6000640 1 -- 172.21.15.123:0/2539947038 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e001a020 con 0x7f53f0149870 2024-04-24T05:46:14.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53e6000640 1 -- 172.21.15.123:0/2539947038 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e0016470 con 0x7f53f0149870 2024-04-24T05:46:14.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53e6000640 1 -- 172.21.15.123:0/2539947038 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0016860 con 0x7f53f0149870 2024-04-24T05:46:14.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53f4a00640 1 -- 172.21.15.123:0/2539947038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f015c590 con 0x7f53f0149870 2024-04-24T05:46:14.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.657+0000 7f53f4a00640 1 -- 172.21.15.123:0/2539947038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f0159880 con 0x7f53f0149870 2024-04-24T05:46:14.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.659+0000 7f53f4a00640 1 -- 172.21.15.123:0/2539947038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f0068a20 con 0x7f53f0149870 2024-04-24T05:46:14.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.660+0000 7f53e6000640 1 -- 172.21.15.123:0/2539947038 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53e0005d10 con 0x7f53f0149870 2024-04-24T05:46:14.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.661+0000 7f53e6000640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f53cc06c8f0 0x7f53cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:14.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.661+0000 7f53e6000640 1 -- 172.21.15.123:0/2539947038 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f53e008ccb0 con 0x7f53f0149870 2024-04-24T05:46:14.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.661+0000 7f53ede00640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f53cc06c8f0 0x7f53cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:14.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.662+0000 7f53ede00640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f53cc06c8f0 0x7f53cc06edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53d8033000 tx=0x7f53d8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:14.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.666+0000 7f53e6000640 1 -- 172.21.15.123:0/2539947038 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f53e0024030 con 0x7f53f0149870 2024-04-24T05:46:14.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.879+0000 7f53f4a00640 1 -- 172.21.15.123:0/2539947038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f53f006f320 con 0x7f53f0149870 2024-04-24T05:46:14.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.880+0000 7f53e6000640 1 -- 172.21.15.123:0/2539947038 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f53e005ae30 con 0x7f53f0149870 2024-04-24T05:46:14.881 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-24T05:46:14.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.883+0000 7f53f4a00640 1 -- 172.21.15.123:0/2539947038 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f53cc06c8f0 msgr2=0x7f53cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:14.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.883+0000 7f53f4a00640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f53cc06c8f0 0x7f53cc06edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53d8033000 tx=0x7f53d8008030 comp rx=0 tx=0).stop 2024-04-24T05:46:14.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.883+0000 7f53f4a00640 1 -- 172.21.15.123:0/2539947038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f0149870 msgr2=0x7f53f00c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:14.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.883+0000 7f53f4a00640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f0149870 0x7f53f00c78e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f53e0007f30 tx=0x7f53e0011660 comp rx=0 tx=0).stop 2024-04-24T05:46:14.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.884+0000 7f53f4a00640 1 -- 172.21.15.123:0/2539947038 shutdown_connections 2024-04-24T05:46:14.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.884+0000 7f53f4a00640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f53cc06c8f0 0x7f53cc06edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.884+0000 7f53f4a00640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f014b900 0x7f53f014bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.884+0000 7f53f4a00640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f0149870 0x7f53f00c78e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.884+0000 7f53f4a00640 1 --2- 172.21.15.123:0/2539947038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40039b0 0x7f53f00cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:14.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.884+0000 7f53f4a00640 1 -- 172.21.15.123:0/2539947038 >> 172.21.15.123:0/2539947038 conn(0x7f53f009d940 msgr2=0x7f53f0160560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:14.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.884+0000 7f53f4a00640 1 -- 172.21.15.123:0/2539947038 shutdown_connections 2024-04-24T05:46:14.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:14.884+0000 7f53f4a00640 1 -- 172.21.15.123:0/2539947038 wait complete. 2024-04-24T05:46:14.903 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:46:14.903 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T05:46:14.952 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:15.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.188+0000 7f0988c00640 1 Processor -- start 2024-04-24T05:46:15.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.188+0000 7f0988c00640 1 -- start start 2024-04-24T05:46:15.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.189+0000 7f0988c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f098415bf80 0x7f0984072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:15.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.189+0000 7f0988c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f098406c330 con 0x7f0984072bc0 2024-04-24T05:46:15.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.189+0000 7f0988c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f098406c610 con 0x7f09841588f0 2024-04-24T05:46:15.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.189+0000 7f0988c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0984072340 con 0x7f098415bf80 2024-04-24T05:46:15.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.189+0000 7f0982800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0984072bc0 0x7f098406dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43500/0 (socket says 172.21.15.123:43500) 2024-04-24T05:46:15.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.189+0000 7f0982800640 1 -- 172.21.15.123:0/118479593 learned_addr learned my addr 172.21.15.123:0/118479593 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:15.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0981400640 1 --2- 172.21.15.123:0/118479593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f098415bf80 0x7f0984072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:15.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 -- 172.21.15.123:0/118479593 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2007531334 0 0) 0x7f098406c330 con 0x7f0984072bc0 2024-04-24T05:46:15.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 -- 172.21.15.123:0/118479593 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0968003520 con 0x7f0984072bc0 2024-04-24T05:46:15.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 -- 172.21.15.123:0/118479593 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3868822655 0 0) 0x7f0968003520 con 0x7f0984072bc0 2024-04-24T05:46:15.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 -- 172.21.15.123:0/118479593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f098415bf80 msgr2=0x7f0984072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:15.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 --2- 172.21.15.123:0/118479593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f098415bf80 0x7f0984072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 -- 172.21.15.123:0/118479593 >> v1:172.21.15.155:6789/0 conn(0x7f09841588f0 legacy=0x7f0984071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:15.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 -- 172.21.15.123:0/118479593 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098406c8f0 con 0x7f0984072bc0 2024-04-24T05:46:15.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 -- 172.21.15.123:0/118479593 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f097c0019f0 con 0x7f0984072bc0 2024-04-24T05:46:15.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 -- 172.21.15.123:0/118479593 >> v1:172.21.15.123:6789/0 conn(0x7f0984072bc0 legacy=0x7f098406dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:15.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 --2- 172.21.15.123:0/118479593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09680039e0 0x7f0968003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:15.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 --2- 172.21.15.123:0/118479593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f098415bf80 0x7f0968004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:15.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 --2- 172.21.15.123:0/118479593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0968004d40 0x7f09680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:15.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 -- 172.21.15.123:0/118479593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0984072340 con 0x7f0968004d40 2024-04-24T05:46:15.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 -- 172.21.15.123:0/118479593 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f096800bb20 con 0x7f09680039e0 2024-04-24T05:46:15.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.190+0000 7f0980a00640 1 -- 172.21.15.123:0/118479593 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f096800bd20 con 0x7f098415bf80 2024-04-24T05:46:15.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.191+0000 7f0982800640 1 --2- 172.21.15.123:0/118479593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0968004d40 0x7f09680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:15.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.191+0000 7f0981e00640 1 --2- 172.21.15.123:0/118479593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09680039e0 0x7f0968003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:15.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.191+0000 7f0981400640 1 --2- 172.21.15.123:0/118479593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f098415bf80 0x7f0968004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:15.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.191+0000 7f0982800640 1 -- 172.21.15.123:0/118479593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f098415bf80 msgr2=0x7f0968004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:15.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.191+0000 7f0982800640 1 --2- 172.21.15.123:0/118479593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f098415bf80 0x7f0968004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.191+0000 7f0982800640 1 -- 172.21.15.123:0/118479593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09680039e0 msgr2=0x7f0968003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:15.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.191+0000 7f0982800640 1 --2- 172.21.15.123:0/118479593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09680039e0 0x7f0968003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.191+0000 7f0982800640 1 -- 172.21.15.123:0/118479593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0968003520 con 0x7f0968004d40 2024-04-24T05:46:15.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.192+0000 7f0982800640 1 --2- 172.21.15.123:0/118479593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0968004d40 0x7f09680051c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f097c00a870 tx=0x7f0968005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e674e4b16ed4ae56 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:15.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.192+0000 7f0980a00640 1 -- 172.21.15.123:0/118479593 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c00a4a0 con 0x7f0968004d40 2024-04-24T05:46:15.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.192+0000 7f0980a00640 1 -- 172.21.15.123:0/118479593 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f097c031070 con 0x7f0968004d40 2024-04-24T05:46:15.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.192+0000 7f0980a00640 1 -- 172.21.15.123:0/118479593 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c036470 con 0x7f0968004d40 2024-04-24T05:46:15.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.193+0000 7f0988c00640 1 -- 172.21.15.123:0/118479593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0968004d40 msgr2=0x7f09680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:15.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.193+0000 7f0988c00640 1 --2- 172.21.15.123:0/118479593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0968004d40 0x7f09680051c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f097c00a870 tx=0x7f0968005830 comp rx=0 tx=0).stop 2024-04-24T05:46:15.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.193+0000 7f0982800640 1 -- 172.21.15.123:0/118479593 reap_dead start 2024-04-24T05:46:15.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.193+0000 7f0988c00640 1 -- 172.21.15.123:0/118479593 shutdown_connections 2024-04-24T05:46:15.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.193+0000 7f0988c00640 1 -- 172.21.15.123:0/118479593 >> 172.21.15.123:0/118479593 conn(0x7f09840b1e70 msgr2=0x7f09840b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:15.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.193+0000 7f0988c00640 1 -- 172.21.15.123:0/118479593 shutdown_connections 2024-04-24T05:46:15.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.193+0000 7f0988c00640 1 -- 172.21.15.123:0/118479593 wait complete. 2024-04-24T05:46:15.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.195+0000 7f0988c00640 1 Processor -- start 2024-04-24T05:46:15.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.195+0000 7f0988c00640 1 -- start start 2024-04-24T05:46:15.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.196+0000 7f0988c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0968004d40 0x7f0984142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:15.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.196+0000 7f0988c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0984153220 0x7f0984153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:15.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.196+0000 7f0988c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0984154340 0x7f0984156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.196+0000 7f0988c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f098406cf40 con 0x7f0984153220 2024-04-24T05:46:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.196+0000 7f0988c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0984158cd0 con 0x7f0968004d40 2024-04-24T05:46:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.196+0000 7f0988c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0984072680 con 0x7f0984154340 2024-04-24T05:46:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.196+0000 7f0981400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0984153220 0x7f0984153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.196+0000 7f0981400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0984153220 0x7f0984153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60266/0 (socket says 172.21.15.123:60266) 2024-04-24T05:46:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.196+0000 7f0981400640 1 -- 172.21.15.123:0/2715009449 learned_addr learned my addr 172.21.15.123:0/2715009449 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.197+0000 7f0981e00640 1 --2- 172.21.15.123:0/2715009449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0968004d40 0x7f0984142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.197+0000 7f0981400640 1 -- 172.21.15.123:0/2715009449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0984154340 msgr2=0x7f0984156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.197+0000 7f0982800640 1 --2- 172.21.15.123:0/2715009449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0984154340 0x7f0984156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.197+0000 7f0981400640 1 --2- 172.21.15.123:0/2715009449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0984154340 0x7f0984156210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.197+0000 7f0981400640 1 -- 172.21.15.123:0/2715009449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0968004d40 msgr2=0x7f0984142590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.197+0000 7f0981400640 1 --2- 172.21.15.123:0/2715009449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0968004d40 0x7f0984142590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.197+0000 7f0981400640 1 -- 172.21.15.123:0/2715009449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0984151670 con 0x7f0984153220 2024-04-24T05:46:15.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.197+0000 7f0981400640 1 --2- 172.21.15.123:0/2715009449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0984153220 0x7f0984153e00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f096c0107c0 tx=0x7f096c010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:15.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.197+0000 7f097a000640 1 -- 172.21.15.123:0/2715009449 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c00b040 con 0x7f0984153220 2024-04-24T05:46:15.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.198+0000 7f097a000640 1 -- 172.21.15.123:0/2715009449 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096c00ecc0 con 0x7f0984153220 2024-04-24T05:46:15.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.198+0000 7f0988c00640 1 -- 172.21.15.123:0/2715009449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09841f35e0 con 0x7f0984153220 2024-04-24T05:46:15.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.198+0000 7f097a000640 1 -- 172.21.15.123:0/2715009449 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c012630 con 0x7f0984153220 2024-04-24T05:46:15.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.198+0000 7f0988c00640 1 -- 172.21.15.123:0/2715009449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09841f3ba0 con 0x7f0984153220 2024-04-24T05:46:15.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.199+0000 7f0988c00640 1 -- 172.21.15.123:0/2715009449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0984068a20 con 0x7f0984153220 2024-04-24T05:46:15.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.200+0000 7f097a000640 1 -- 172.21.15.123:0/2715009449 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f096c0127d0 con 0x7f0984153220 2024-04-24T05:46:15.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.201+0000 7f097a000640 1 --2- 172.21.15.123:0/2715009449 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f096006c8f0 0x7f096006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:15.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.201+0000 7f097a000640 1 -- 172.21.15.123:0/2715009449 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f096c08c8a0 con 0x7f0984153220 2024-04-24T05:46:15.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.201+0000 7f0981e00640 1 --2- 172.21.15.123:0/2715009449 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f096006c8f0 0x7f096006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:15.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.202+0000 7f0981e00640 1 --2- 172.21.15.123:0/2715009449 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f096006c8f0 0x7f096006edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0974005490 tx=0x7f0974009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:15.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.206+0000 7f097a000640 1 -- 172.21.15.123:0/2715009449 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0984151670 con 0x7f0984153220 2024-04-24T05:46:15.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.406+0000 7f0988c00640 1 -- 172.21.15.123:0/2715009449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0984154fe0 con 0x7f0984153220 2024-04-24T05:46:15.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.409+0000 7f097a000640 1 -- 172.21.15.123:0/2715009449 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f096c05a8f0 con 0x7f0984153220 2024-04-24T05:46:15.410 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:15.410 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:04.715857+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:15.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.412+0000 7f0988c00640 1 -- 172.21.15.123:0/2715009449 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f096006c8f0 msgr2=0x7f096006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:15.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.412+0000 7f0988c00640 1 --2- 172.21.15.123:0/2715009449 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f096006c8f0 0x7f096006edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0974005490 tx=0x7f0974009040 comp rx=0 tx=0).stop 2024-04-24T05:46:15.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.412+0000 7f0988c00640 1 -- 172.21.15.123:0/2715009449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0984153220 msgr2=0x7f0984153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:15.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.412+0000 7f0988c00640 1 --2- 172.21.15.123:0/2715009449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0984153220 0x7f0984153e00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f096c0107c0 tx=0x7f096c010c80 comp rx=0 tx=0).stop 2024-04-24T05:46:15.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.412+0000 7f0988c00640 1 -- 172.21.15.123:0/2715009449 shutdown_connections 2024-04-24T05:46:15.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.412+0000 7f0988c00640 1 --2- 172.21.15.123:0/2715009449 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f096006c8f0 0x7f096006edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.412+0000 7f0988c00640 1 --2- 172.21.15.123:0/2715009449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0984154340 0x7f0984156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.412+0000 7f0988c00640 1 --2- 172.21.15.123:0/2715009449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0984153220 0x7f0984153e00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.412+0000 7f0988c00640 1 --2- 172.21.15.123:0/2715009449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0968004d40 0x7f0984142590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.412+0000 7f0988c00640 1 -- 172.21.15.123:0/2715009449 >> 172.21.15.123:0/2715009449 conn(0x7f09840b1e70 msgr2=0x7f09841557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:15.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.412+0000 7f0988c00640 1 -- 172.21.15.123:0/2715009449 shutdown_connections 2024-04-24T05:46:15.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.412+0000 7f0988c00640 1 -- 172.21.15.123:0/2715009449 wait complete. 2024-04-24T05:46:15.431 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:46:15.431 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:46:15.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.631+0000 7f8394c00640 1 Processor -- start 2024-04-24T05:46:15.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.632+0000 7f8394c00640 1 -- start start 2024-04-24T05:46:15.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.633+0000 7f8394c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f839006c510 0x7f8390071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:15.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.633+0000 7f8394c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f839006c8f0 0x7f839006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:15.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.633+0000 7f8394c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83900cf170 con 0x7f83900ccbb0 2024-04-24T05:46:15.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.633+0000 7f8394c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f83900cf450 con 0x7f839006c510 2024-04-24T05:46:15.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.633+0000 7f8394c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f83900cf730 con 0x7f839006c8f0 2024-04-24T05:46:15.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.633+0000 7f838de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f839006c510 0x7f8390071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:15.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.633+0000 7f838de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f839006c510 0x7f8390071c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60270/0 (socket says 172.21.15.123:60270) 2024-04-24T05:46:15.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.633+0000 7f838de00640 1 -- 172.21.15.123:0/1161236780 learned_addr learned my addr 172.21.15.123:0/1161236780 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.634+0000 7f838d400640 1 --2- 172.21.15.123:0/1161236780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f839006c8f0 0x7f839006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.634+0000 7f838de00640 1 -- 172.21.15.123:0/1161236780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f839006c8f0 msgr2=0x7f839006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.634+0000 7f838de00640 1 --2- 172.21.15.123:0/1161236780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f839006c8f0 0x7f839006cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.634+0000 7f838de00640 1 -- 172.21.15.123:0/1161236780 >> v1:172.21.15.155:6789/0 conn(0x7f83900ccbb0 legacy=0x7f83900d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.634+0000 7f838de00640 1 -- 172.21.15.123:0/1161236780 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83900cfa10 con 0x7f839006c510 2024-04-24T05:46:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.634+0000 7f838d400640 1 --2- 172.21.15.123:0/1161236780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f839006c8f0 0x7f839006cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.635+0000 7f838de00640 1 --2- 172.21.15.123:0/1161236780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f839006c510 0x7f8390071c50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8380009e50 tx=0x7f838002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34894dbff27a72b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:15.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.635+0000 7f838ca00640 1 -- 172.21.15.123:0/1161236780 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380036070 con 0x7f839006c510 2024-04-24T05:46:15.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.635+0000 7f838ca00640 1 -- 172.21.15.123:0/1161236780 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8380031600 con 0x7f839006c510 2024-04-24T05:46:15.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.635+0000 7f838ca00640 1 -- 172.21.15.123:0/1161236780 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83800319d0 con 0x7f839006c510 2024-04-24T05:46:15.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.636+0000 7f8394c00640 1 -- 172.21.15.123:0/1161236780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f839006c510 msgr2=0x7f8390071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:15.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.636+0000 7f8394c00640 1 --2- 172.21.15.123:0/1161236780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f839006c510 0x7f8390071c50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8380009e50 tx=0x7f838002a6c0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.636+0000 7f8394c00640 1 -- 172.21.15.123:0/1161236780 shutdown_connections 2024-04-24T05:46:15.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.636+0000 7f8394c00640 1 --2- 172.21.15.123:0/1161236780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f839006c8f0 0x7f839006cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.636+0000 7f8394c00640 1 --2- 172.21.15.123:0/1161236780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f839006c510 0x7f8390071c50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.636+0000 7f8394c00640 1 -- 172.21.15.123:0/1161236780 >> 172.21.15.123:0/1161236780 conn(0x7f83900b51f0 msgr2=0x7f83900b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:15.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.636+0000 7f8394c00640 1 -- 172.21.15.123:0/1161236780 shutdown_connections 2024-04-24T05:46:15.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.636+0000 7f8394c00640 1 -- 172.21.15.123:0/1161236780 wait complete. 2024-04-24T05:46:15.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.638+0000 7f8394c00640 1 Processor -- start 2024-04-24T05:46:15.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.638+0000 7f8394c00640 1 -- start start 2024-04-24T05:46:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.639+0000 7f8394c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f839006c510 0x7f8390158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.639+0000 7f8394c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f839006c8f0 0x7f83900ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.639+0000 7f8394c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83900ccbb0 0x7f83900ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.639+0000 7f8394c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83900cee80 con 0x7f839006c510 2024-04-24T05:46:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.639+0000 7f8394c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f83900ced00 con 0x7f83900ccbb0 2024-04-24T05:46:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.639+0000 7f8394c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f83900cd040 con 0x7f839006c8f0 2024-04-24T05:46:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.639+0000 7f838de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f839006c510 0x7f8390158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.639+0000 7f838de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f839006c510 0x7f8390158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60280/0 (socket says 172.21.15.123:60280) 2024-04-24T05:46:15.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.639+0000 7f838de00640 1 -- 172.21.15.123:0/1686660021 learned_addr learned my addr 172.21.15.123:0/1686660021 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:15.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.640+0000 7f838d400640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f839006c8f0 0x7f83900ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:15.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.640+0000 7f838e800640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83900ccbb0 0x7f83900ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:15.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.640+0000 7f838de00640 1 -- 172.21.15.123:0/1686660021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f839006c8f0 msgr2=0x7f83900ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.640+0000 7f838de00640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f839006c8f0 0x7f83900ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.640+0000 7f838de00640 1 -- 172.21.15.123:0/1686660021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83900ccbb0 msgr2=0x7f83900ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.640+0000 7f838de00640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83900ccbb0 0x7f83900ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.640+0000 7f838de00640 1 -- 172.21.15.123:0/1686660021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83900c7340 con 0x7f839006c510 2024-04-24T05:46:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.640+0000 7f838d400640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f839006c8f0 0x7f83900ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.640+0000 7f838de00640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f839006c510 0x7f8390158bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8380009e50 tx=0x7f8380004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.640+0000 7f8386000640 1 -- 172.21.15.123:0/1686660021 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380041020 con 0x7f839006c510 2024-04-24T05:46:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.640+0000 7f8386000640 1 -- 172.21.15.123:0/1686660021 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8380007820 con 0x7f839006c510 2024-04-24T05:46:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.640+0000 7f8386000640 1 -- 172.21.15.123:0/1686660021 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380036040 con 0x7f839006c510 2024-04-24T05:46:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.640+0000 7f8394c00640 1 -- 172.21.15.123:0/1686660021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83900c7620 con 0x7f839006c510 2024-04-24T05:46:15.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.641+0000 7f8394c00640 1 -- 172.21.15.123:0/1686660021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83900c7be0 con 0x7f839006c510 2024-04-24T05:46:15.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.642+0000 7f8394c00640 1 -- 172.21.15.123:0/1686660021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8390068a20 con 0x7f839006c510 2024-04-24T05:46:15.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.643+0000 7f8386000640 1 -- 172.21.15.123:0/1686660021 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8380044020 con 0x7f839006c510 2024-04-24T05:46:15.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.644+0000 7f8386000640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f836c06c8f0 0x7f836c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:15.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.644+0000 7f8386000640 1 -- 172.21.15.123:0/1686660021 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f83800acc30 con 0x7f839006c510 2024-04-24T05:46:15.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.644+0000 7f838d400640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f836c06c8f0 0x7f836c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:15.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.645+0000 7f838d400640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f836c06c8f0 0x7f836c06edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8378006fd0 tx=0x7f8378008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:15.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.649+0000 7f8386000640 1 -- 172.21.15.123:0/1686660021 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f838004b030 con 0x7f839006c510 2024-04-24T05:46:15.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.847+0000 7f8394c00640 1 -- 172.21.15.123:0/1686660021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f83900ca460 con 0x7f839006c510 2024-04-24T05:46:15.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.848+0000 7f8386000640 1 -- 172.21.15.123:0/1686660021 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f838007adb0 con 0x7f839006c510 2024-04-24T05:46:15.848 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:46:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.850+0000 7f8394c00640 1 -- 172.21.15.123:0/1686660021 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f836c06c8f0 msgr2=0x7f836c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.850+0000 7f8394c00640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f836c06c8f0 0x7f836c06edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8378006fd0 tx=0x7f8378008040 comp rx=0 tx=0).stop 2024-04-24T05:46:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.850+0000 7f8394c00640 1 -- 172.21.15.123:0/1686660021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f839006c510 msgr2=0x7f8390158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.850+0000 7f8394c00640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f839006c510 0x7f8390158bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8380009e50 tx=0x7f8380004a40 comp rx=0 tx=0).stop 2024-04-24T05:46:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.850+0000 7f8394c00640 1 -- 172.21.15.123:0/1686660021 shutdown_connections 2024-04-24T05:46:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.850+0000 7f8394c00640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f836c06c8f0 0x7f836c06edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.850+0000 7f8394c00640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83900ccbb0 0x7f83900ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.850+0000 7f8394c00640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f839006c8f0 0x7f83900ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.850+0000 7f8394c00640 1 --2- 172.21.15.123:0/1686660021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f839006c510 0x7f8390158bf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:15.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.850+0000 7f8394c00640 1 -- 172.21.15.123:0/1686660021 >> 172.21.15.123:0/1686660021 conn(0x7f83900b51f0 msgr2=0x7f839006f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:15.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.851+0000 7f8394c00640 1 -- 172.21.15.123:0/1686660021 shutdown_connections 2024-04-24T05:46:15.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:15.851+0000 7f8394c00640 1 -- 172.21.15.123:0/1686660021 wait complete. 2024-04-24T05:46:15.870 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:46:16.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.077+0000 7f64e6600640 1 Processor -- start 2024-04-24T05:46:16.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.077+0000 7f64e6600640 1 -- start start 2024-04-24T05:46:16.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.079+0000 7f64e6600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64e00b38c0 0x7f64e00b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.079+0000 7f64e6600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f64e00d21e0 0x7f64e00b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.079+0000 7f64e6600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e00d3470 con 0x7f64e00d4350 2024-04-24T05:46:16.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.079+0000 7f64e6600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f64e00b1340 con 0x7f64e00b38c0 2024-04-24T05:46:16.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.079+0000 7f64e6600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f64e00d3750 con 0x7f64e00d21e0 2024-04-24T05:46:16.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.079+0000 7f64d7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64e00b38c0 0x7f64e00b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.079+0000 7f64d7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64e00b38c0 0x7f64e00b17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60294/0 (socket says 172.21.15.123:60294) 2024-04-24T05:46:16.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.079+0000 7f64d7e00640 1 -- 172.21.15.123:0/171723112 learned_addr learned my addr 172.21.15.123:0/171723112 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:16.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.080+0000 7f64d7e00640 1 -- 172.21.15.123:0/171723112 >> v2:172.21.15.155:3300/0 conn(0x7f64e00d21e0 msgr2=0x7f64e00b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:16.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.080+0000 7f64d7e00640 1 --2- 172.21.15.123:0/171723112 >> v2:172.21.15.155:3300/0 conn(0x7f64e00d21e0 0x7f64e00b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.080+0000 7f64d7e00640 1 -- 172.21.15.123:0/171723112 >> v1:172.21.15.155:6789/0 conn(0x7f64e00d4350 legacy=0x7f64e00d4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:16.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.080+0000 7f64d7e00640 1 -- 172.21.15.123:0/171723112 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e00d3a30 con 0x7f64e00b38c0 2024-04-24T05:46:16.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.081+0000 7f64d7e00640 1 --2- 172.21.15.123:0/171723112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e00b38c0 0x7f64e00b17d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f64e00b3fe0 tx=0x7f64c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a3e4859352b5c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:16.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.081+0000 7f64dea00640 1 -- 172.21.15.123:0/171723112 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8037070 con 0x7f64e00b38c0 2024-04-24T05:46:16.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.081+0000 7f64dea00640 1 -- 172.21.15.123:0/171723112 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64c8032440 con 0x7f64e00b38c0 2024-04-24T05:46:16.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.081+0000 7f64dea00640 1 -- 172.21.15.123:0/171723112 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8032810 con 0x7f64e00b38c0 2024-04-24T05:46:16.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.082+0000 7f64e6600640 1 -- 172.21.15.123:0/171723112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e00b38c0 msgr2=0x7f64e00b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.082+0000 7f64e6600640 1 --2- 172.21.15.123:0/171723112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e00b38c0 0x7f64e00b17d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f64e00b3fe0 tx=0x7f64c802a1a0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.082+0000 7f64e6600640 1 -- 172.21.15.123:0/171723112 shutdown_connections 2024-04-24T05:46:16.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.082+0000 7f64e6600640 1 --2- 172.21.15.123:0/171723112 >> v2:172.21.15.155:3300/0 conn(0x7f64e00d21e0 0x7f64e00b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.082+0000 7f64e6600640 1 --2- 172.21.15.123:0/171723112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e00b38c0 0x7f64e00b17d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.082+0000 7f64e6600640 1 -- 172.21.15.123:0/171723112 >> 172.21.15.123:0/171723112 conn(0x7f64e009d970 msgr2=0x7f64e00b95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:16.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.082+0000 7f64e6600640 1 -- 172.21.15.123:0/171723112 shutdown_connections 2024-04-24T05:46:16.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.082+0000 7f64e6600640 1 -- 172.21.15.123:0/171723112 wait complete. 2024-04-24T05:46:16.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.084+0000 7f64e6600640 1 Processor -- start 2024-04-24T05:46:16.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.084+0000 7f64e6600640 1 -- start start 2024-04-24T05:46:16.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.086+0000 7f64e6600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64e00b38c0 0x7f64e00cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.086+0000 7f64e6600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e00d21e0 0x7f64e00b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.086+0000 7f64e6600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64e00d4350 0x7f64e015ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.086+0000 7f64e6600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64e015eb50 con 0x7f64e00d21e0 2024-04-24T05:46:16.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.086+0000 7f64e6600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f64e00d3100 con 0x7f64e00d4350 2024-04-24T05:46:16.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.086+0000 7f64e6600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f64e015ecd0 con 0x7f64e00b38c0 2024-04-24T05:46:16.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.086+0000 7f64d7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e00d21e0 0x7f64e00b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.086+0000 7f64d7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e00d21e0 0x7f64e00b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60308/0 (socket says 172.21.15.123:60308) 2024-04-24T05:46:16.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.086+0000 7f64d7e00640 1 -- 172.21.15.123:0/1846584967 learned_addr learned my addr 172.21.15.123:0/1846584967 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:16.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.086+0000 7f64df400640 1 --2- 172.21.15.123:0/1846584967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64e00b38c0 0x7f64e00cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.086+0000 7f64dfe00640 1 --2- 172.21.15.123:0/1846584967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64e00d4350 0x7f64e015ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.086+0000 7f64d7e00640 1 -- 172.21.15.123:0/1846584967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64e00b38c0 msgr2=0x7f64e00cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.086+0000 7f64d7e00640 1 --2- 172.21.15.123:0/1846584967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64e00b38c0 0x7f64e00cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.087+0000 7f64d7e00640 1 -- 172.21.15.123:0/1846584967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64e00d4350 msgr2=0x7f64e015ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.087+0000 7f64d7e00640 1 --2- 172.21.15.123:0/1846584967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64e00d4350 0x7f64e015ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.087+0000 7f64d7e00640 1 -- 172.21.15.123:0/1846584967 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e00b5b10 con 0x7f64e00d21e0 2024-04-24T05:46:16.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.087+0000 7f64d7e00640 1 --2- 172.21.15.123:0/1846584967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e00d21e0 0x7f64e00b50c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f64c8000c00 tx=0x7f64c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:16.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.087+0000 7f64d7400640 1 -- 172.21.15.123:0/1846584967 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8041070 con 0x7f64e00d21e0 2024-04-24T05:46:16.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.087+0000 7f64d7400640 1 -- 172.21.15.123:0/1846584967 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64c803c870 con 0x7f64e00d21e0 2024-04-24T05:46:16.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.087+0000 7f64e6600640 1 -- 172.21.15.123:0/1846584967 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e00b5df0 con 0x7f64e00d21e0 2024-04-24T05:46:16.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.087+0000 7f64d7400640 1 -- 172.21.15.123:0/1846584967 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8037040 con 0x7f64e00d21e0 2024-04-24T05:46:16.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.087+0000 7f64e6600640 1 -- 172.21.15.123:0/1846584967 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e00b63b0 con 0x7f64e00d21e0 2024-04-24T05:46:16.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.089+0000 7f64e6600640 1 -- 172.21.15.123:0/1846584967 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e0068a20 con 0x7f64e00d21e0 2024-04-24T05:46:16.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.090+0000 7f64d7400640 1 -- 172.21.15.123:0/1846584967 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64c8044070 con 0x7f64e00d21e0 2024-04-24T05:46:16.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.091+0000 7f64d7400640 1 --2- 172.21.15.123:0/1846584967 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f64bc06c8f0 0x7f64bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.092+0000 7f64d7400640 1 -- 172.21.15.123:0/1846584967 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f64c80acc00 con 0x7f64e00d21e0 2024-04-24T05:46:16.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.092+0000 7f64df400640 1 --2- 172.21.15.123:0/1846584967 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f64bc06c8f0 0x7f64bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.092+0000 7f64df400640 1 --2- 172.21.15.123:0/1846584967 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f64bc06c8f0 0x7f64bc06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f64d0007fd0 tx=0x7f64d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:16.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.097+0000 7f64d7400640 1 -- 172.21.15.123:0/1846584967 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64c807ae00 con 0x7f64e00d21e0 2024-04-24T05:46:16.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.303+0000 7f64e6600640 1 -- 172.21.15.123:0/1846584967 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f64e00b99e0 con 0x7f64e00d21e0 2024-04-24T05:46:16.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.304+0000 7f64d7400640 1 -- 172.21.15.123:0/1846584967 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f64c807a700 con 0x7f64e00d21e0 2024-04-24T05:46:16.305 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:16.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.307+0000 7f64e6600640 1 -- 172.21.15.123:0/1846584967 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f64bc06c8f0 msgr2=0x7f64bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.307+0000 7f64e6600640 1 --2- 172.21.15.123:0/1846584967 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f64bc06c8f0 0x7f64bc06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f64d0007fd0 tx=0x7f64d0009040 comp rx=0 tx=0).stop 2024-04-24T05:46:16.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.307+0000 7f64e6600640 1 -- 172.21.15.123:0/1846584967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e00d21e0 msgr2=0x7f64e00b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.307+0000 7f64e6600640 1 --2- 172.21.15.123:0/1846584967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e00d21e0 0x7f64e00b50c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f64c8000c00 tx=0x7f64c80046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.307+0000 7f64e6600640 1 -- 172.21.15.123:0/1846584967 shutdown_connections 2024-04-24T05:46:16.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.307+0000 7f64e6600640 1 --2- 172.21.15.123:0/1846584967 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f64bc06c8f0 0x7f64bc06edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.307+0000 7f64e6600640 1 --2- 172.21.15.123:0/1846584967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64e00d4350 0x7f64e015ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.307+0000 7f64e6600640 1 --2- 172.21.15.123:0/1846584967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64e00d21e0 0x7f64e00b50c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.307+0000 7f64e6600640 1 --2- 172.21.15.123:0/1846584967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64e00b38c0 0x7f64e00cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.307+0000 7f64e6600640 1 -- 172.21.15.123:0/1846584967 >> 172.21.15.123:0/1846584967 conn(0x7f64e009d970 msgr2=0x7f64e00d18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:16.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.307+0000 7f64e6600640 1 -- 172.21.15.123:0/1846584967 shutdown_connections 2024-04-24T05:46:16.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.308+0000 7f64e6600640 1 -- 172.21.15.123:0/1846584967 wait complete. 2024-04-24T05:46:16.327 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:46:16.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.528+0000 7fbc97200640 1 Processor -- start 2024-04-24T05:46:16.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.528+0000 7fbc97200640 1 -- start start 2024-04-24T05:46:16.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.530+0000 7fbc97200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbc9815f870 0x7fbc980d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.530+0000 7fbc97200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc980d1420 0x7fbc980cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.530+0000 7fbc97200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc98162a60 con 0x7fbc9815e5f0 2024-04-24T05:46:16.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.530+0000 7fbc97200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbc980d18a0 con 0x7fbc9815f870 2024-04-24T05:46:16.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.530+0000 7fbc97200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbc980ceb50 con 0x7fbc980d1420 2024-04-24T05:46:16.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.530+0000 7fbc8fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbc9815f870 0x7fbc980d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.530+0000 7fbc8fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbc9815f870 0x7fbc980d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60312/0 (socket says 172.21.15.123:60312) 2024-04-24T05:46:16.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.530+0000 7fbc8fe00640 1 -- 172.21.15.123:0/3514942063 learned_addr learned my addr 172.21.15.123:0/3514942063 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:16.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.530+0000 7fbc96800640 1 --2- 172.21.15.123:0/3514942063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc980d1420 0x7fbc980cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.531+0000 7fbc8fe00640 1 -- 172.21.15.123:0/3514942063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc980d1420 msgr2=0x7fbc980cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:16.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.531+0000 7fbc8fe00640 1 --2- 172.21.15.123:0/3514942063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc980d1420 0x7fbc980cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.531+0000 7fbc8fe00640 1 -- 172.21.15.123:0/3514942063 >> v1:172.21.15.155:6789/0 conn(0x7fbc9815e5f0 legacy=0x7fbc98162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:16.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.531+0000 7fbc8fe00640 1 -- 172.21.15.123:0/3514942063 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc98162d40 con 0x7fbc9815f870 2024-04-24T05:46:16.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.532+0000 7fbc8fe00640 1 --2- 172.21.15.123:0/3514942063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815f870 0x7fbc980d0ee0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009870 tx=0x7fbc8002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb6286ad96163 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:16.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.532+0000 7fbc95400640 1 -- 172.21.15.123:0/3514942063 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80037070 con 0x7fbc9815f870 2024-04-24T05:46:16.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.532+0000 7fbc95400640 1 -- 172.21.15.123:0/3514942063 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc80032440 con 0x7fbc9815f870 2024-04-24T05:46:16.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.532+0000 7fbc95400640 1 -- 172.21.15.123:0/3514942063 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80032810 con 0x7fbc9815f870 2024-04-24T05:46:16.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.533+0000 7fbc97200640 1 -- 172.21.15.123:0/3514942063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815f870 msgr2=0x7fbc980d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.533+0000 7fbc97200640 1 --2- 172.21.15.123:0/3514942063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815f870 0x7fbc980d0ee0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009870 tx=0x7fbc8002a210 comp rx=0 tx=0).stop 2024-04-24T05:46:16.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.533+0000 7fbc97200640 1 -- 172.21.15.123:0/3514942063 shutdown_connections 2024-04-24T05:46:16.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.533+0000 7fbc97200640 1 --2- 172.21.15.123:0/3514942063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc980d1420 0x7fbc980cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.533+0000 7fbc97200640 1 --2- 172.21.15.123:0/3514942063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815f870 0x7fbc980d0ee0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.533+0000 7fbc97200640 1 -- 172.21.15.123:0/3514942063 >> 172.21.15.123:0/3514942063 conn(0x7fbc980b9db0 msgr2=0x7fbc9809d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:16.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.533+0000 7fbc97200640 1 -- 172.21.15.123:0/3514942063 shutdown_connections 2024-04-24T05:46:16.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.533+0000 7fbc97200640 1 -- 172.21.15.123:0/3514942063 wait complete. 2024-04-24T05:46:16.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.534+0000 7fbc97200640 1 Processor -- start 2024-04-24T05:46:16.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.535+0000 7fbc97200640 1 -- start start 2024-04-24T05:46:16.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.535+0000 7fbc97200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc980d1420 0x7fbc980b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.535+0000 7fbc97200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc9815e5f0 0x7fbc9814baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.535+0000 7fbc97200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815f870 0x7fbc9806bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.535+0000 7fbc97200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9815e9d0 con 0x7fbc9815f870 2024-04-24T05:46:16.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.535+0000 7fbc97200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbc980cee30 con 0x7fbc9815e5f0 2024-04-24T05:46:16.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.535+0000 7fbc97200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbc9815eb50 con 0x7fbc980d1420 2024-04-24T05:46:16.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.536+0000 7fbc96800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815f870 0x7fbc9806bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.536+0000 7fbc96800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815f870 0x7fbc9806bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60326/0 (socket says 172.21.15.123:60326) 2024-04-24T05:46:16.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.536+0000 7fbc96800640 1 -- 172.21.15.123:0/1547804273 learned_addr learned my addr 172.21.15.123:0/1547804273 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:16.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.536+0000 7fbc8fe00640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc9815e5f0 0x7fbc9814baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.536+0000 7fbc95e00640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc980d1420 0x7fbc980b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.536+0000 7fbc96800640 1 -- 172.21.15.123:0/1547804273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc980d1420 msgr2=0x7fbc980b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.536+0000 7fbc96800640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc980d1420 0x7fbc980b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.536+0000 7fbc96800640 1 -- 172.21.15.123:0/1547804273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc9815e5f0 msgr2=0x7fbc9814baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.536+0000 7fbc96800640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc9815e5f0 0x7fbc9814baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.536+0000 7fbc96800640 1 -- 172.21.15.123:0/1547804273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc980cc420 con 0x7fbc9815f870 2024-04-24T05:46:16.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.536+0000 7fbc95e00640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc980d1420 0x7fbc980b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:16.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.537+0000 7fbc96800640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815f870 0x7fbc9806bfe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbc90007e70 tx=0x7fbc9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:16.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.537+0000 7fbc8e000640 1 -- 172.21.15.123:0/1547804273 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90014620 con 0x7fbc9815f870 2024-04-24T05:46:16.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.537+0000 7fbc8e000640 1 -- 172.21.15.123:0/1547804273 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc90019070 con 0x7fbc9815f870 2024-04-24T05:46:16.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.537+0000 7fbc97200640 1 -- 172.21.15.123:0/1547804273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc980cc700 con 0x7fbc9815f870 2024-04-24T05:46:16.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.537+0000 7fbc8e000640 1 -- 172.21.15.123:0/1547804273 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90014a20 con 0x7fbc9815f870 2024-04-24T05:46:16.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.537+0000 7fbc97200640 1 -- 172.21.15.123:0/1547804273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc980cccc0 con 0x7fbc9815f870 2024-04-24T05:46:16.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.538+0000 7fbc97200640 1 -- 172.21.15.123:0/1547804273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc98068a20 con 0x7fbc9815f870 2024-04-24T05:46:16.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.539+0000 7fbc8e000640 1 -- 172.21.15.123:0/1547804273 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc90014bc0 con 0x7fbc9815f870 2024-04-24T05:46:16.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.540+0000 7fbc8e000640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbc7406c8f0 0x7fbc7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.540+0000 7fbc8e000640 1 -- 172.21.15.123:0/1547804273 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbc9008c7a0 con 0x7fbc9815f870 2024-04-24T05:46:16.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.541+0000 7fbc95e00640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbc7406c8f0 0x7fbc7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.541+0000 7fbc95e00640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbc7406c8f0 0x7fbc7406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbc88003eb0 tx=0x7fbc88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:16.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.544+0000 7fbc8e000640 1 -- 172.21.15.123:0/1547804273 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc9005a880 con 0x7fbc9815f870 2024-04-24T05:46:16.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.746+0000 7fbc97200640 1 -- 172.21.15.123:0/1547804273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbc9806eff0 con 0x7fbc9815f870 2024-04-24T05:46:16.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.747+0000 7fbc8e000640 1 -- 172.21.15.123:0/1547804273 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fbc9005a180 con 0x7fbc9815f870 2024-04-24T05:46:16.747 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:46:16.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.749+0000 7fbc97200640 1 -- 172.21.15.123:0/1547804273 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbc7406c8f0 msgr2=0x7fbc7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.749+0000 7fbc97200640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbc7406c8f0 0x7fbc7406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbc88003eb0 tx=0x7fbc88009040 comp rx=0 tx=0).stop 2024-04-24T05:46:16.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.750+0000 7fbc97200640 1 -- 172.21.15.123:0/1547804273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815f870 msgr2=0x7fbc9806bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.750+0000 7fbc97200640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815f870 0x7fbc9806bfe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbc90007e70 tx=0x7fbc9000e4b0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.750+0000 7fbc97200640 1 -- 172.21.15.123:0/1547804273 shutdown_connections 2024-04-24T05:46:16.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.750+0000 7fbc97200640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbc7406c8f0 0x7fbc7406edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.750+0000 7fbc97200640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc9815f870 0x7fbc9806bfe0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.750+0000 7fbc97200640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc9815e5f0 0x7fbc9814baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.750+0000 7fbc97200640 1 --2- 172.21.15.123:0/1547804273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc980d1420 0x7fbc980b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.750+0000 7fbc97200640 1 -- 172.21.15.123:0/1547804273 >> 172.21.15.123:0/1547804273 conn(0x7fbc980b9db0 msgr2=0x7fbc980b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:16.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.750+0000 7fbc97200640 1 -- 172.21.15.123:0/1547804273 shutdown_connections 2024-04-24T05:46:16.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.750+0000 7fbc97200640 1 -- 172.21.15.123:0/1547804273 wait complete. 2024-04-24T05:46:16.770 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:46:16.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.973+0000 7f81b9800640 1 Processor -- start 2024-04-24T05:46:16.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.974+0000 7f81b9800640 1 -- start start 2024-04-24T05:46:16.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.975+0000 7f81b9800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b4072600 0x7f81b4072a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.975+0000 7f81b9800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81b415f110 con 0x7f81b406e530 2024-04-24T05:46:16.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.975+0000 7f81b9800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81b415f3f0 con 0x7f81b415aa40 2024-04-24T05:46:16.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.975+0000 7f81b9800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f81b4072fa0 con 0x7f81b4072600 2024-04-24T05:46:16.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.975+0000 7f81b8e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f81b406e530 0x7f81b406e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43512/0 (socket says 172.21.15.123:43512) 2024-04-24T05:46:16.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.975+0000 7f81b8e00640 1 -- 172.21.15.123:0/2329530923 learned_addr learned my addr 172.21.15.123:0/2329530923 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:16.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.975+0000 7f81ab400640 1 --2- 172.21.15.123:0/2329530923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b4072600 0x7f81b4072a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.976+0000 7f81b3400640 1 -- 172.21.15.123:0/2329530923 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3967183561 0 0) 0x7f81b415f110 con 0x7f81b406e530 2024-04-24T05:46:16.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.976+0000 7f81b3400640 1 -- 172.21.15.123:0/2329530923 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8198003520 con 0x7f81b406e530 2024-04-24T05:46:16.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.976+0000 7f81ab400640 1 -- 172.21.15.123:0/2329530923 >> v1:172.21.15.155:6789/0 conn(0x7f81b415aa40 legacy=0x7f81b415e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.976+0000 7f81ab400640 1 -- 172.21.15.123:0/2329530923 >> v1:172.21.15.123:6789/0 conn(0x7f81b406e530 legacy=0x7f81b406e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.976+0000 7f81b3400640 1 -- 172.21.15.123:0/2329530923 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 928296468 0 0) 0x7f8198003520 con 0x7f81b406e530 2024-04-24T05:46:16.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.976+0000 7f81ab400640 1 -- 172.21.15.123:0/2329530923 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b406dd30 con 0x7f81b4072600 2024-04-24T05:46:16.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.978+0000 7f81ab400640 1 --2- 172.21.15.123:0/2329530923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b4072600 0x7f81b4072a60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f819c0097d0 tx=0x7f819c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c61dfcdc9d17a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:16.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.978+0000 7f81b3400640 1 -- 172.21.15.123:0/2329530923 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c037070 con 0x7f81b4072600 2024-04-24T05:46:16.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.978+0000 7f81b3400640 1 -- 172.21.15.123:0/2329530923 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f819c02ac30 con 0x7f81b4072600 2024-04-24T05:46:16.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.978+0000 7f81b3400640 1 -- 172.21.15.123:0/2329530923 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac0019f0 con 0x7f81b4072600 2024-04-24T05:46:16.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.979+0000 7f81b9800640 1 -- 172.21.15.123:0/2329530923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b4072600 msgr2=0x7f81b4072a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.979+0000 7f81b9800640 1 --2- 172.21.15.123:0/2329530923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b4072600 0x7f81b4072a60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f819c0097d0 tx=0x7f819c02a1a0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.979+0000 7f81b9800640 1 -- 172.21.15.123:0/2329530923 shutdown_connections 2024-04-24T05:46:16.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.979+0000 7f81b9800640 1 --2- 172.21.15.123:0/2329530923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b4072600 0x7f81b4072a60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.979+0000 7f81b9800640 1 -- 172.21.15.123:0/2329530923 >> 172.21.15.123:0/2329530923 conn(0x7f81b40b1cf0 msgr2=0x7f81b40b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:16.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.979+0000 7f81b9800640 1 -- 172.21.15.123:0/2329530923 shutdown_connections 2024-04-24T05:46:16.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.979+0000 7f81b9800640 1 -- 172.21.15.123:0/2329530923 wait complete. 2024-04-24T05:46:16.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.980+0000 7f81b9800640 1 Processor -- start 2024-04-24T05:46:16.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.981+0000 7f81b9800640 1 -- start start 2024-04-24T05:46:16.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.981+0000 7f81b9800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b406e530 0x7f81b415dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.981+0000 7f81b9800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81b4072600 0x7f81b41e5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.981+0000 7f81b9800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b415aa40 0x7f81b41e5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.981+0000 7f81b9800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81b415ae20 con 0x7f81b4072600 2024-04-24T05:46:16.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.981+0000 7f81b9800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4071dc0 con 0x7f81b415aa40 2024-04-24T05:46:16.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.981+0000 7f81b9800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f81b415baa0 con 0x7f81b406e530 2024-04-24T05:46:16.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.982+0000 7f81ab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81b4072600 0x7f81b41e5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.982+0000 7f81ab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81b4072600 0x7f81b41e5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60332/0 (socket says 172.21.15.123:60332) 2024-04-24T05:46:16.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.982+0000 7f81ab400640 1 -- 172.21.15.123:0/125055768 learned_addr learned my addr 172.21.15.123:0/125055768 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:16.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.982+0000 7f81b3e00640 1 --2- 172.21.15.123:0/125055768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b406e530 0x7f81b415dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.982+0000 7f81ab400640 1 -- 172.21.15.123:0/125055768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b406e530 msgr2=0x7f81b415dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.982+0000 7f81ab400640 1 --2- 172.21.15.123:0/125055768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b406e530 0x7f81b415dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.982+0000 7f81ab400640 1 -- 172.21.15.123:0/125055768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b415aa40 msgr2=0x7f81b41e5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:16.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.982+0000 7f81ab400640 1 --2- 172.21.15.123:0/125055768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b415aa40 0x7f81b41e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:16.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.982+0000 7f81ab400640 1 -- 172.21.15.123:0/125055768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b415bd80 con 0x7f81b4072600 2024-04-24T05:46:16.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.983+0000 7f81ab400640 1 --2- 172.21.15.123:0/125055768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81b4072600 0x7f81b41e5340 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f819c035040 tx=0x7f819c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:16.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.983+0000 7f81b0c00640 1 -- 172.21.15.123:0/125055768 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c041070 con 0x7f81b4072600 2024-04-24T05:46:16.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.983+0000 7f81b0c00640 1 -- 172.21.15.123:0/125055768 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f819c03c680 con 0x7f81b4072600 2024-04-24T05:46:16.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.983+0000 7f81b0c00640 1 -- 172.21.15.123:0/125055768 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c037040 con 0x7f81b4072600 2024-04-24T05:46:16.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.983+0000 7f81b9800640 1 -- 172.21.15.123:0/125055768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b415c060 con 0x7f81b4072600 2024-04-24T05:46:16.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.983+0000 7f81b9800640 1 -- 172.21.15.123:0/125055768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b415e2d0 con 0x7f81b4072600 2024-04-24T05:46:16.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.984+0000 7f81b9800640 1 -- 172.21.15.123:0/125055768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b4068a20 con 0x7f81b4072600 2024-04-24T05:46:16.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.985+0000 7f81b0c00640 1 -- 172.21.15.123:0/125055768 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f819c044070 con 0x7f81b4072600 2024-04-24T05:46:16.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.986+0000 7f81b0c00640 1 --2- 172.21.15.123:0/125055768 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f819006c8f0 0x7f819006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:16.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.986+0000 7f81b0c00640 1 -- 172.21.15.123:0/125055768 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f819c0aca80 con 0x7f81b4072600 2024-04-24T05:46:16.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.986+0000 7f81b3e00640 1 --2- 172.21.15.123:0/125055768 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f819006c8f0 0x7f819006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:16.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.987+0000 7f81b3e00640 1 --2- 172.21.15.123:0/125055768 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f819006c8f0 0x7f819006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81a4000f60 tx=0x7f81a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:16.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:16.991+0000 7f81b0c00640 1 -- 172.21.15.123:0/125055768 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f819c0aee20 con 0x7f81b4072600 2024-04-24T05:46:17.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.192+0000 7f81b9800640 1 -- 172.21.15.123:0/125055768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f81b40b29c0 con 0x7f81b4072600 2024-04-24T05:46:17.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.193+0000 7f81b0c00640 1 -- 172.21.15.123:0/125055768 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f819c07ac80 con 0x7f81b4072600 2024-04-24T05:46:17.195 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:17.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.196+0000 7f81b9800640 1 -- 172.21.15.123:0/125055768 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f819006c8f0 msgr2=0x7f819006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:17.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.196+0000 7f81b9800640 1 --2- 172.21.15.123:0/125055768 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f819006c8f0 0x7f819006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81a4000f60 tx=0x7f81a4009040 comp rx=0 tx=0).stop 2024-04-24T05:46:17.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.196+0000 7f81b9800640 1 -- 172.21.15.123:0/125055768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81b4072600 msgr2=0x7f81b41e5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:17.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.196+0000 7f81b9800640 1 --2- 172.21.15.123:0/125055768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81b4072600 0x7f81b41e5340 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f819c035040 tx=0x7f819c0046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.196+0000 7f81b9800640 1 -- 172.21.15.123:0/125055768 shutdown_connections 2024-04-24T05:46:17.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.196+0000 7f81b9800640 1 --2- 172.21.15.123:0/125055768 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f819006c8f0 0x7f819006edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.196+0000 7f81b9800640 1 --2- 172.21.15.123:0/125055768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81b415aa40 0x7f81b41e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.196+0000 7f81b9800640 1 --2- 172.21.15.123:0/125055768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81b4072600 0x7f81b41e5340 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.196+0000 7f81b9800640 1 --2- 172.21.15.123:0/125055768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81b406e530 0x7f81b415dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.196+0000 7f81b9800640 1 -- 172.21.15.123:0/125055768 >> 172.21.15.123:0/125055768 conn(0x7f81b40b1cf0 msgr2=0x7f81b406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:17.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.197+0000 7f81b9800640 1 -- 172.21.15.123:0/125055768 shutdown_connections 2024-04-24T05:46:17.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.197+0000 7f81b9800640 1 -- 172.21.15.123:0/125055768 wait complete. 2024-04-24T05:46:17.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T05:46:17.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.408+0000 7f2edee00640 1 Processor -- start 2024-04-24T05:46:17.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.408+0000 7f2edee00640 1 -- start start 2024-04-24T05:46:17.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.409+0000 7f2edee00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2ed806ba00 0x7f2ed806cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:17.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.409+0000 7f2edee00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2ed815d5d0 0x7f2ed806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:17.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.409+0000 7f2edee00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed806be90 con 0x7f2ed80ad150 2024-04-24T05:46:17.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.409+0000 7f2edee00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2ed806c170 con 0x7f2ed806ba00 2024-04-24T05:46:17.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.409+0000 7f2edee00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2ed806c450 con 0x7f2ed815d5d0 2024-04-24T05:46:17.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.410+0000 7f2ecf400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2ed806ba00 0x7f2ed806cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:17.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.410+0000 7f2ed7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f2ed80ad150 0x7f2ed80ad530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43518/0 (socket says 172.21.15.123:43518) 2024-04-24T05:46:17.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.410+0000 7f2ed7e00640 1 -- 172.21.15.123:0/2598028428 learned_addr learned my addr 172.21.15.123:0/2598028428 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:17.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.410+0000 7f2edce00640 1 --2- 172.21.15.123:0/2598028428 >> v2:172.21.15.155:3300/0 conn(0x7f2ed815d5d0 0x7f2ed806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:17.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.410+0000 7f2ecf400640 1 -- 172.21.15.123:0/2598028428 >> v2:172.21.15.155:3300/0 conn(0x7f2ed815d5d0 msgr2=0x7f2ed806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:17.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.410+0000 7f2ecf400640 1 --2- 172.21.15.123:0/2598028428 >> v2:172.21.15.155:3300/0 conn(0x7f2ed815d5d0 0x7f2ed806dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.410+0000 7f2ecf400640 1 -- 172.21.15.123:0/2598028428 >> v1:172.21.15.123:6789/0 conn(0x7f2ed80ad150 legacy=0x7f2ed80ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:17.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.410+0000 7f2ecf400640 1 -- 172.21.15.123:0/2598028428 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed806c730 con 0x7f2ed806ba00 2024-04-24T05:46:17.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.410+0000 7f2edce00640 1 --2- 172.21.15.123:0/2598028428 >> v2:172.21.15.155:3300/0 conn(0x7f2ed815d5d0 0x7f2ed806dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:17.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.411+0000 7f2ecf400640 1 --2- 172.21.15.123:0/2598028428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed806ba00 0x7f2ed806cf30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80d12a0 tx=0x7f2ec002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d9c1dce7288a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.412+0000 7f2ed7400640 1 -- 172.21.15.123:0/2598028428 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0037070 con 0x7f2ed806ba00 2024-04-24T05:46:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.412+0000 7f2ed7400640 1 -- 172.21.15.123:0/2598028428 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec0032440 con 0x7f2ed806ba00 2024-04-24T05:46:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.412+0000 7f2ed7400640 1 -- 172.21.15.123:0/2598028428 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0032810 con 0x7f2ed806ba00 2024-04-24T05:46:17.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.413+0000 7f2edee00640 1 -- 172.21.15.123:0/2598028428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed806ba00 msgr2=0x7f2ed806cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:17.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.413+0000 7f2edee00640 1 --2- 172.21.15.123:0/2598028428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed806ba00 0x7f2ed806cf30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80d12a0 tx=0x7f2ec002a1a0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.413+0000 7f2edee00640 1 -- 172.21.15.123:0/2598028428 shutdown_connections 2024-04-24T05:46:17.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.413+0000 7f2edee00640 1 --2- 172.21.15.123:0/2598028428 >> v2:172.21.15.155:3300/0 conn(0x7f2ed815d5d0 0x7f2ed806dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.413+0000 7f2edee00640 1 --2- 172.21.15.123:0/2598028428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed806ba00 0x7f2ed806cf30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.413+0000 7f2edee00640 1 -- 172.21.15.123:0/2598028428 >> 172.21.15.123:0/2598028428 conn(0x7f2ed80b1e70 msgr2=0x7f2ed80b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:17.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.413+0000 7f2edee00640 1 -- 172.21.15.123:0/2598028428 shutdown_connections 2024-04-24T05:46:17.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.413+0000 7f2edee00640 1 -- 172.21.15.123:0/2598028428 wait complete. 2024-04-24T05:46:17.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.415+0000 7f2edee00640 1 Processor -- start 2024-04-24T05:46:17.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.415+0000 7f2edee00640 1 -- start start 2024-04-24T05:46:17.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2edee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed80ad150 0x7f2ed81f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:17.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2edee00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ed815d5d0 0x7f2ed8151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:17.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2edee00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ed80cbb40 0x7f2ed809d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2edee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed80af360 con 0x7f2ed80ad150 2024-04-24T05:46:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2edee00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed815da20 con 0x7f2ed80cbb40 2024-04-24T05:46:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2edee00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed81520d0 con 0x7f2ed815d5d0 2024-04-24T05:46:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2ed7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed80ad150 0x7f2ed81f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2ed7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed80ad150 0x7f2ed81f5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60338/0 (socket says 172.21.15.123:60338) 2024-04-24T05:46:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2ed7e00640 1 -- 172.21.15.123:0/952196785 learned_addr learned my addr 172.21.15.123:0/952196785 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2edce00640 1 --2- 172.21.15.123:0/952196785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ed80cbb40 0x7f2ed809d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2ecf400640 1 --2- 172.21.15.123:0/952196785 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ed815d5d0 0x7f2ed8151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2ed7e00640 1 -- 172.21.15.123:0/952196785 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ed815d5d0 msgr2=0x7f2ed8151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2ed7e00640 1 --2- 172.21.15.123:0/952196785 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ed815d5d0 0x7f2ed8151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2ed7e00640 1 -- 172.21.15.123:0/952196785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ed80cbb40 msgr2=0x7f2ed809d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2ed7e00640 1 --2- 172.21.15.123:0/952196785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ed80cbb40 0x7f2ed809d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.416+0000 7f2ed7e00640 1 -- 172.21.15.123:0/952196785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed81f5b50 con 0x7f2ed80ad150 2024-04-24T05:46:17.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.417+0000 7f2ed7e00640 1 --2- 172.21.15.123:0/952196785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed80ad150 0x7f2ed81f5330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800c400 tx=0x7f2ec800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.417+0000 7f2ed4c00640 1 -- 172.21.15.123:0/952196785 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8017070 con 0x7f2ed80ad150 2024-04-24T05:46:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.417+0000 7f2ed4c00640 1 -- 172.21.15.123:0/952196785 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec8005ce0 con 0x7f2ed80ad150 2024-04-24T05:46:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.417+0000 7f2ed4c00640 1 -- 172.21.15.123:0/952196785 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec80072f0 con 0x7f2ed80ad150 2024-04-24T05:46:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.417+0000 7f2edee00640 1 -- 172.21.15.123:0/952196785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed81f5e30 con 0x7f2ed80ad150 2024-04-24T05:46:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.417+0000 7f2edee00640 1 -- 172.21.15.123:0/952196785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed80c5260 con 0x7f2ed80ad150 2024-04-24T05:46:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.418+0000 7f2edee00640 1 -- 172.21.15.123:0/952196785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed80ae680 con 0x7f2ed80ad150 2024-04-24T05:46:17.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.419+0000 7f2ed4c00640 1 -- 172.21.15.123:0/952196785 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ec801a070 con 0x7f2ed80ad150 2024-04-24T05:46:17.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.421+0000 7f2ed4c00640 1 --2- 172.21.15.123:0/952196785 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2eb406c8f0 0x7f2eb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:17.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.421+0000 7f2ed4c00640 1 -- 172.21.15.123:0/952196785 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2ec808b240 con 0x7f2ed80ad150 2024-04-24T05:46:17.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.421+0000 7f2ecf400640 1 --2- 172.21.15.123:0/952196785 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2eb406c8f0 0x7f2eb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:17.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.422+0000 7f2ecf400640 1 --2- 172.21.15.123:0/952196785 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2eb406c8f0 0x7f2eb406edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0034000 tx=0x7f2ec002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:17.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.425+0000 7f2ed4c00640 1 -- 172.21.15.123:0/952196785 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ed80ae680 con 0x7f2ed80ad150 2024-04-24T05:46:17.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.632+0000 7f2edee00640 1 -- 172.21.15.123:0/952196785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2ed80ada70 con 0x7f2ed80ad150 2024-04-24T05:46:17.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.633+0000 7f2ed4c00640 1 -- 172.21.15.123:0/952196785 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f2ed80ada70 con 0x7f2ed80ad150 2024-04-24T05:46:17.634 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-24T05:46:17.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.636+0000 7f2edee00640 1 -- 172.21.15.123:0/952196785 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2eb406c8f0 msgr2=0x7f2eb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:17.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.636+0000 7f2edee00640 1 --2- 172.21.15.123:0/952196785 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2eb406c8f0 0x7f2eb406edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0034000 tx=0x7f2ec002e000 comp rx=0 tx=0).stop 2024-04-24T05:46:17.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.636+0000 7f2edee00640 1 -- 172.21.15.123:0/952196785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed80ad150 msgr2=0x7f2ed81f5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:17.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.636+0000 7f2edee00640 1 --2- 172.21.15.123:0/952196785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed80ad150 0x7f2ed81f5330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800c400 tx=0x7f2ec800c8c0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.636+0000 7f2edee00640 1 -- 172.21.15.123:0/952196785 shutdown_connections 2024-04-24T05:46:17.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.636+0000 7f2edee00640 1 --2- 172.21.15.123:0/952196785 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2eb406c8f0 0x7f2eb406edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.636+0000 7f2edee00640 1 --2- 172.21.15.123:0/952196785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ed80cbb40 0x7f2ed809d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.636+0000 7f2edee00640 1 --2- 172.21.15.123:0/952196785 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ed815d5d0 0x7f2ed8151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.636+0000 7f2edee00640 1 --2- 172.21.15.123:0/952196785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed80ad150 0x7f2ed81f5330 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:17.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.636+0000 7f2edee00640 1 -- 172.21.15.123:0/952196785 >> 172.21.15.123:0/952196785 conn(0x7f2ed80b1e70 msgr2=0x7f2ed80afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:17.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.636+0000 7f2edee00640 1 -- 172.21.15.123:0/952196785 shutdown_connections 2024-04-24T05:46:17.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:17.636+0000 7f2edee00640 1 -- 172.21.15.123:0/952196785 wait complete. 2024-04-24T05:46:17.655 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2024-04-24T05:46:17.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:17.655 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-24T05:46:17.730 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi123.stderr:ceph-fuse[40366]: fuse finished with error 0 and tester_r 0 2024-04-24T05:46:17.736 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T05:46:17.806 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-04-24T05:46:17.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:17.807 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-24T05:46:17.865 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:18.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.061+0000 7f8ba5c00640 1 Processor -- start 2024-04-24T05:46:18.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.061+0000 7f8ba5c00640 1 -- start start 2024-04-24T05:46:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.062+0000 7f8ba5c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ba015a2a0 0x7f8ba00ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.062+0000 7f8ba5c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ba00ad690 0x7f8ba00af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.062+0000 7f8ba5c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8ba00af780 0x7f8ba0141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.062+0000 7f8ba5c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8ba00adaf0 con 0x7f8ba00ad690 2024-04-24T05:46:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.062+0000 7f8ba5c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8ba015a680 con 0x7f8ba00af780 2024-04-24T05:46:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.062+0000 7f8ba5c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba015da00 con 0x7f8ba015a2a0 2024-04-24T05:46:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.063+0000 7f8b9e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ba00ad690 0x7f8ba00af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.063+0000 7f8b9e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ba00ad690 0x7f8ba00af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60342/0 (socket says 172.21.15.123:60342) 2024-04-24T05:46:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.063+0000 7f8b9e000640 1 -- 172.21.15.123:0/4009629132 learned_addr learned my addr 172.21.15.123:0/4009629132 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.063+0000 7f8b9f400640 1 --2- 172.21.15.123:0/4009629132 >> v2:172.21.15.155:3300/0 conn(0x7f8ba00af780 0x7f8ba0141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.063+0000 7f8b9ea00640 1 --2- 172.21.15.123:0/4009629132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ba015a2a0 0x7f8ba00ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.063+0000 7f8b9e000640 1 -- 172.21.15.123:0/4009629132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ba015a2a0 msgr2=0x7f8ba00ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.063+0000 7f8b9e000640 1 --2- 172.21.15.123:0/4009629132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ba015a2a0 0x7f8ba00ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.063+0000 7f8b9e000640 1 -- 172.21.15.123:0/4009629132 >> v2:172.21.15.155:3300/0 conn(0x7f8ba00af780 msgr2=0x7f8ba0141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.063+0000 7f8b9e000640 1 --2- 172.21.15.123:0/4009629132 >> v2:172.21.15.155:3300/0 conn(0x7f8ba00af780 0x7f8ba0141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.063+0000 7f8b9e000640 1 -- 172.21.15.123:0/4009629132 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba00afbf0 con 0x7f8ba00ad690 2024-04-24T05:46:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.063+0000 7f8b9f400640 1 --2- 172.21.15.123:0/4009629132 >> v2:172.21.15.155:3300/0 conn(0x7f8ba00af780 0x7f8ba0141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:18.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.063+0000 7f8b9ea00640 1 --2- 172.21.15.123:0/4009629132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ba015a2a0 0x7f8ba00ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:18.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.064+0000 7f8b9e000640 1 --2- 172.21.15.123:0/4009629132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00ad690 0x7f8ba00af240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00b1e00 tx=0x7f8b8802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee8abd9ff34a98f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:18.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.065+0000 7f8b9d600640 1 -- 172.21.15.123:0/4009629132 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88037070 con 0x7f8ba00ad690 2024-04-24T05:46:18.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.065+0000 7f8b9d600640 1 -- 172.21.15.123:0/4009629132 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b88032440 con 0x7f8ba00ad690 2024-04-24T05:46:18.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.065+0000 7f8b9d600640 1 -- 172.21.15.123:0/4009629132 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88032810 con 0x7f8ba00ad690 2024-04-24T05:46:18.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.065+0000 7f8ba5c00640 1 -- 172.21.15.123:0/4009629132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00ad690 msgr2=0x7f8ba00af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.065+0000 7f8ba5c00640 1 --2- 172.21.15.123:0/4009629132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00ad690 0x7f8ba00af240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00b1e00 tx=0x7f8b8802a140 comp rx=0 tx=0).stop 2024-04-24T05:46:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.065+0000 7f8ba5c00640 1 -- 172.21.15.123:0/4009629132 shutdown_connections 2024-04-24T05:46:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.065+0000 7f8ba5c00640 1 --2- 172.21.15.123:0/4009629132 >> v2:172.21.15.155:3300/0 conn(0x7f8ba00af780 0x7f8ba0141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.065+0000 7f8ba5c00640 1 --2- 172.21.15.123:0/4009629132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00ad690 0x7f8ba00af240 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.065+0000 7f8ba5c00640 1 --2- 172.21.15.123:0/4009629132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ba015a2a0 0x7f8ba00ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.065+0000 7f8ba5c00640 1 -- 172.21.15.123:0/4009629132 >> 172.21.15.123:0/4009629132 conn(0x7f8ba00b1e70 msgr2=0x7f8ba00b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:18.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.065+0000 7f8ba5c00640 1 -- 172.21.15.123:0/4009629132 shutdown_connections 2024-04-24T05:46:18.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.065+0000 7f8ba5c00640 1 -- 172.21.15.123:0/4009629132 wait complete. 2024-04-24T05:46:18.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.068+0000 7f8ba5c00640 1 Processor -- start 2024-04-24T05:46:18.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.068+0000 7f8ba5c00640 1 -- start start 2024-04-24T05:46:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.069+0000 7f8ba5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00ad690 0x7f8ba01f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.069+0000 7f8ba5c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ba00af780 0x7f8ba01514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.069+0000 7f8ba5c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ba015a2a0 0x7f8ba0151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.069+0000 7f8ba5c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba015d3b0 con 0x7f8ba00ad690 2024-04-24T05:46:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.069+0000 7f8ba5c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba015d0d0 con 0x7f8ba00af780 2024-04-24T05:46:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.069+0000 7f8ba5c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba015d690 con 0x7f8ba015a2a0 2024-04-24T05:46:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.069+0000 7f8b9ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00ad690 0x7f8ba01f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.069+0000 7f8b9ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00ad690 0x7f8ba01f20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60354/0 (socket says 172.21.15.123:60354) 2024-04-24T05:46:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.069+0000 7f8b9ea00640 1 -- 172.21.15.123:0/3460451280 learned_addr learned my addr 172.21.15.123:0/3460451280 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.070+0000 7f8b9e000640 1 --2- 172.21.15.123:0/3460451280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ba00af780 0x7f8ba01514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.070+0000 7f8b9ea00640 1 -- 172.21.15.123:0/3460451280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ba015a2a0 msgr2=0x7f8ba0151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.070+0000 7f8b9ea00640 1 --2- 172.21.15.123:0/3460451280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ba015a2a0 0x7f8ba0151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.070+0000 7f8b9ea00640 1 -- 172.21.15.123:0/3460451280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ba00af780 msgr2=0x7f8ba01514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:18.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.070+0000 7f8b9ea00640 1 --2- 172.21.15.123:0/3460451280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ba00af780 0x7f8ba01514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.070+0000 7f8b9ea00640 1 -- 172.21.15.123:0/3460451280 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba00cc590 con 0x7f8ba00ad690 2024-04-24T05:46:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.070+0000 7f8b9ea00640 1 --2- 172.21.15.123:0/3460451280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00ad690 0x7f8ba01f20f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000c7f0 tx=0x7f8b9000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.070+0000 7f8b96a00640 1 -- 172.21.15.123:0/3460451280 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90015070 con 0x7f8ba00ad690 2024-04-24T05:46:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.070+0000 7f8b96a00640 1 -- 172.21.15.123:0/3460451280 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b90007a30 con 0x7f8ba00ad690 2024-04-24T05:46:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.070+0000 7f8b96a00640 1 -- 172.21.15.123:0/3460451280 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90007e00 con 0x7f8ba00ad690 2024-04-24T05:46:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.070+0000 7f8ba5c00640 1 -- 172.21.15.123:0/3460451280 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba00cc870 con 0x7f8ba00ad690 2024-04-24T05:46:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.070+0000 7f8ba5c00640 1 -- 172.21.15.123:0/3460451280 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba00cce30 con 0x7f8ba00ad690 2024-04-24T05:46:18.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.072+0000 7f8ba5c00640 1 -- 172.21.15.123:0/3460451280 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba0068a20 con 0x7f8ba00ad690 2024-04-24T05:46:18.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.072+0000 7f8b96a00640 1 -- 172.21.15.123:0/3460451280 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b90018070 con 0x7f8ba00ad690 2024-04-24T05:46:18.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.073+0000 7f8b96a00640 1 --2- 172.21.15.123:0/3460451280 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8b7c06c8f0 0x7f8b7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.073+0000 7f8b96a00640 1 -- 172.21.15.123:0/3460451280 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8b9008be70 con 0x7f8ba00ad690 2024-04-24T05:46:18.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.074+0000 7f8b9e000640 1 --2- 172.21.15.123:0/3460451280 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8b7c06c8f0 0x7f8b7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:18.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.074+0000 7f8b9e000640 1 --2- 172.21.15.123:0/3460451280 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8b7c06c8f0 0x7f8b7c06edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8b88034000 tx=0x7f8b8802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:18.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.078+0000 7f8b96a00640 1 -- 172.21.15.123:0/3460451280 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b90059f40 con 0x7f8ba00ad690 2024-04-24T05:46:18.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.279+0000 7f8ba5c00640 1 -- 172.21.15.123:0/3460451280 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ba00cffd0 con 0x7f8ba00ad690 2024-04-24T05:46:18.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.282+0000 7f8b96a00640 1 -- 172.21.15.123:0/3460451280 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f8b90059840 con 0x7f8ba00ad690 2024-04-24T05:46:18.284 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:18.284 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:04.715857+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.285+0000 7f8ba5c00640 1 -- 172.21.15.123:0/3460451280 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8b7c06c8f0 msgr2=0x7f8b7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.285+0000 7f8ba5c00640 1 --2- 172.21.15.123:0/3460451280 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8b7c06c8f0 0x7f8b7c06edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8b88034000 tx=0x7f8b8802e000 comp rx=0 tx=0).stop 2024-04-24T05:46:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.285+0000 7f8ba5c00640 1 -- 172.21.15.123:0/3460451280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00ad690 msgr2=0x7f8ba01f20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.285+0000 7f8ba5c00640 1 --2- 172.21.15.123:0/3460451280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00ad690 0x7f8ba01f20f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000c7f0 tx=0x7f8b9000cbf0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.285+0000 7f8ba5c00640 1 -- 172.21.15.123:0/3460451280 shutdown_connections 2024-04-24T05:46:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.285+0000 7f8ba5c00640 1 --2- 172.21.15.123:0/3460451280 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8b7c06c8f0 0x7f8b7c06edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.285+0000 7f8ba5c00640 1 --2- 172.21.15.123:0/3460451280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ba015a2a0 0x7f8ba0151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.285+0000 7f8ba5c00640 1 --2- 172.21.15.123:0/3460451280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ba00af780 0x7f8ba01514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.285+0000 7f8ba5c00640 1 --2- 172.21.15.123:0/3460451280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba00ad690 0x7f8ba01f20f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.285+0000 7f8ba5c00640 1 -- 172.21.15.123:0/3460451280 >> 172.21.15.123:0/3460451280 conn(0x7f8ba00b1e70 msgr2=0x7f8ba0154b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:18.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.285+0000 7f8ba5c00640 1 -- 172.21.15.123:0/3460451280 shutdown_connections 2024-04-24T05:46:18.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.285+0000 7f8ba5c00640 1 -- 172.21.15.123:0/3460451280 wait complete. 2024-04-24T05:46:18.305 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:18.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.496+0000 7f800ee00640 1 Processor -- start 2024-04-24T05:46:18.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.496+0000 7f800ee00640 1 -- start start 2024-04-24T05:46:18.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.498+0000 7f800ee00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f80080ad860 0x7f80080af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.498+0000 7f800ee00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f800815cdf0 con 0x7f80080af780 2024-04-24T05:46:18.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.498+0000 7f800ee00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f800815d0d0 con 0x7f800815a2a0 2024-04-24T05:46:18.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.498+0000 7f800ee00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f800815a680 con 0x7f80080ad860 2024-04-24T05:46:18.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.498+0000 7f8007400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f80080ad860 0x7f80080af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:18.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.498+0000 7f800ce00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f80080af780 0x7f800806bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43524/0 (socket says 172.21.15.123:43524) 2024-04-24T05:46:18.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.498+0000 7f800ce00640 1 -- 172.21.15.123:0/704649327 learned_addr learned my addr 172.21.15.123:0/704649327 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:18.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.499+0000 7f8007400640 1 -- 172.21.15.123:0/704649327 >> v1:172.21.15.155:6789/0 conn(0x7f800815a2a0 legacy=0x7f80080ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:18.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.499+0000 7f8007400640 1 -- 172.21.15.123:0/704649327 >> v1:172.21.15.123:6789/0 conn(0x7f80080af780 legacy=0x7f800806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:18.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.499+0000 7f8007400640 1 -- 172.21.15.123:0/704649327 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800815d3b0 con 0x7f80080ad860 2024-04-24T05:46:18.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.500+0000 7f8007400640 1 --2- 172.21.15.123:0/704649327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80080ad860 0x7f80080af240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40097a0 tx=0x7f7ff402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fac1f36dbbadf19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:18.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.500+0000 7f8006a00640 1 -- 172.21.15.123:0/704649327 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4037070 con 0x7f80080ad860 2024-04-24T05:46:18.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.500+0000 7f8006a00640 1 -- 172.21.15.123:0/704649327 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff402ae50 con 0x7f80080ad860 2024-04-24T05:46:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.500+0000 7f8006a00640 1 -- 172.21.15.123:0/704649327 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4032660 con 0x7f80080ad860 2024-04-24T05:46:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.501+0000 7f800ee00640 1 -- 172.21.15.123:0/704649327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80080ad860 msgr2=0x7f80080af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.501+0000 7f800ee00640 1 --2- 172.21.15.123:0/704649327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80080ad860 0x7f80080af240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40097a0 tx=0x7f7ff402a140 comp rx=0 tx=0).stop 2024-04-24T05:46:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.501+0000 7f800ee00640 1 -- 172.21.15.123:0/704649327 shutdown_connections 2024-04-24T05:46:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.501+0000 7f800ee00640 1 --2- 172.21.15.123:0/704649327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80080ad860 0x7f80080af240 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.501+0000 7f800ee00640 1 -- 172.21.15.123:0/704649327 >> 172.21.15.123:0/704649327 conn(0x7f80080b1e70 msgr2=0x7f80080b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.501+0000 7f800ee00640 1 -- 172.21.15.123:0/704649327 shutdown_connections 2024-04-24T05:46:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.501+0000 7f800ee00640 1 -- 172.21.15.123:0/704649327 wait complete. 2024-04-24T05:46:18.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.503+0000 7f800ee00640 1 Processor -- start 2024-04-24T05:46:18.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.503+0000 7f800ee00640 1 -- start start 2024-04-24T05:46:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f800ee00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80080ad860 0x7f80080cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f800ee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80080af780 0x7f80081f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f800ee00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f800815a2a0 0x7f80080ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f800ee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80080afb90 con 0x7f80080af780 2024-04-24T05:46:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f800ee00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f800815da00 con 0x7f80080ad860 2024-04-24T05:46:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f800ee00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f800806c210 con 0x7f800815a2a0 2024-04-24T05:46:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f8007400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80080af780 0x7f80081f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f8007400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80080af780 0x7f80081f3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60378/0 (socket says 172.21.15.123:60378) 2024-04-24T05:46:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f8007400640 1 -- 172.21.15.123:0/394866013 learned_addr learned my addr 172.21.15.123:0/394866013 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f8007e00640 1 --2- 172.21.15.123:0/394866013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80080ad860 0x7f80080cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f8007400640 1 -- 172.21.15.123:0/394866013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f800815a2a0 msgr2=0x7f80080ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f8007400640 1 --2- 172.21.15.123:0/394866013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f800815a2a0 0x7f80080ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f8007400640 1 -- 172.21.15.123:0/394866013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80080ad860 msgr2=0x7f80080cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f8007400640 1 --2- 172.21.15.123:0/394866013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80080ad860 0x7f80080cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.504+0000 7f8007400640 1 -- 172.21.15.123:0/394866013 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80080ca240 con 0x7f80080af780 2024-04-24T05:46:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.505+0000 7f8007400640 1 --2- 172.21.15.123:0/394866013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80080af780 0x7f80081f3fd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4000c00 tx=0x7f7ff40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.505+0000 7f7fffe00640 1 -- 172.21.15.123:0/394866013 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4041070 con 0x7f80080af780 2024-04-24T05:46:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.505+0000 7f7fffe00640 1 -- 172.21.15.123:0/394866013 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff403c670 con 0x7f80080af780 2024-04-24T05:46:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.505+0000 7f7fffe00640 1 -- 172.21.15.123:0/394866013 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4037040 con 0x7f80080af780 2024-04-24T05:46:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.505+0000 7f800ee00640 1 -- 172.21.15.123:0/394866013 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80080ca520 con 0x7f80080af780 2024-04-24T05:46:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.505+0000 7f800ee00640 1 -- 172.21.15.123:0/394866013 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80080caae0 con 0x7f80080af780 2024-04-24T05:46:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.506+0000 7f800ee00640 1 -- 172.21.15.123:0/394866013 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8008068a20 con 0x7f80080af780 2024-04-24T05:46:18.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.507+0000 7f7fffe00640 1 -- 172.21.15.123:0/394866013 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ff4044070 con 0x7f80080af780 2024-04-24T05:46:18.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.508+0000 7f7fffe00640 1 --2- 172.21.15.123:0/394866013 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7fe806c8f0 0x7f7fe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.508+0000 7f7fffe00640 1 -- 172.21.15.123:0/394866013 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7ff40ac680 con 0x7f80080af780 2024-04-24T05:46:18.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.509+0000 7f8007e00640 1 --2- 172.21.15.123:0/394866013 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7fe806c8f0 0x7f7fe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:18.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.509+0000 7f8007e00640 1 --2- 172.21.15.123:0/394866013 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7fe806c8f0 0x7f7fe806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8007bf0 tx=0x7f7ff8007240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:18.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.513+0000 7f7fffe00640 1 -- 172.21.15.123:0/394866013 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ff407a760 con 0x7f80080af780 2024-04-24T05:46:18.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.715+0000 7f800ee00640 1 -- 172.21.15.123:0/394866013 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f800806fdf0 con 0x7f80080af780 2024-04-24T05:46:18.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.718+0000 7f7fffe00640 1 -- 172.21.15.123:0/394866013 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f7ff407a060 con 0x7f80080af780 2024-04-24T05:46:18.720 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:18.720 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:04.715857+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:18.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.721+0000 7f800ee00640 1 -- 172.21.15.123:0/394866013 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7fe806c8f0 msgr2=0x7f7fe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:18.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.721+0000 7f800ee00640 1 --2- 172.21.15.123:0/394866013 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7fe806c8f0 0x7f7fe806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8007bf0 tx=0x7f7ff8007240 comp rx=0 tx=0).stop 2024-04-24T05:46:18.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.721+0000 7f800ee00640 1 -- 172.21.15.123:0/394866013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80080af780 msgr2=0x7f80081f3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:18.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.721+0000 7f800ee00640 1 --2- 172.21.15.123:0/394866013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80080af780 0x7f80081f3fd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4000c00 tx=0x7f7ff40046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.721+0000 7f800ee00640 1 -- 172.21.15.123:0/394866013 shutdown_connections 2024-04-24T05:46:18.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.721+0000 7f800ee00640 1 --2- 172.21.15.123:0/394866013 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7fe806c8f0 0x7f7fe806edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.721+0000 7f800ee00640 1 --2- 172.21.15.123:0/394866013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f800815a2a0 0x7f80080ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.721+0000 7f800ee00640 1 --2- 172.21.15.123:0/394866013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80080af780 0x7f80081f3fd0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.721+0000 7f800ee00640 1 --2- 172.21.15.123:0/394866013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80080ad860 0x7f80080cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.721+0000 7f800ee00640 1 -- 172.21.15.123:0/394866013 >> 172.21.15.123:0/394866013 conn(0x7f80080b1e70 msgr2=0x7f800815b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:18.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.721+0000 7f800ee00640 1 -- 172.21.15.123:0/394866013 shutdown_connections 2024-04-24T05:46:18.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.721+0000 7f800ee00640 1 -- 172.21.15.123:0/394866013 wait complete. 2024-04-24T05:46:18.742 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T05:46:18.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.945+0000 7fa250a00640 1 Processor -- start 2024-04-24T05:46:18.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.945+0000 7fa250a00640 1 -- start start 2024-04-24T05:46:18.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.946+0000 7fa250a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa24c15e640 0x7fa24c06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.947+0000 7fa250a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa24c06f730 0x7fa24c0acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.947+0000 7fa250a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa24c06baa0 con 0x7fa24c06d080 2024-04-24T05:46:18.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.947+0000 7fa250a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa24c15ea20 con 0x7fa24c15e640 2024-04-24T05:46:18.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.947+0000 7fa250a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa24c06bd80 con 0x7fa24c06f730 2024-04-24T05:46:18.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.947+0000 7fa24b400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa24c15e640 0x7fa24c06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:18.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.947+0000 7fa24b400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa24c15e640 0x7fa24c06cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56296/0 (socket says 172.21.15.123:56296) 2024-04-24T05:46:18.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.947+0000 7fa24b400640 1 -- 172.21.15.123:0/3428325613 learned_addr learned my addr 172.21.15.123:0/3428325613 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:18.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.947+0000 7fa24b400640 1 -- 172.21.15.123:0/3428325613 >> v2:172.21.15.155:3300/0 conn(0x7fa24c06f730 msgr2=0x7fa24c0acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:18.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.947+0000 7fa24b400640 1 --2- 172.21.15.123:0/3428325613 >> v2:172.21.15.155:3300/0 conn(0x7fa24c06f730 0x7fa24c0acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.947+0000 7fa24b400640 1 -- 172.21.15.123:0/3428325613 >> v1:172.21.15.155:6789/0 conn(0x7fa24c06d080 legacy=0x7fa24c071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:18.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.947+0000 7fa24b400640 1 -- 172.21.15.123:0/3428325613 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c06c060 con 0x7fa24c15e640 2024-04-24T05:46:18.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.948+0000 7fa24b400640 1 --2- 172.21.15.123:0/3428325613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa24c15e640 0x7fa24c06cb40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa24c074180 tx=0x7fa23c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c84fc0d4b6dad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:18.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.949+0000 7fa24aa00640 1 -- 172.21.15.123:0/3428325613 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c036070 con 0x7fa24c15e640 2024-04-24T05:46:18.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.949+0000 7fa24aa00640 1 -- 172.21.15.123:0/3428325613 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa23c031600 con 0x7fa24c15e640 2024-04-24T05:46:18.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.949+0000 7fa24aa00640 1 -- 172.21.15.123:0/3428325613 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0319d0 con 0x7fa24c15e640 2024-04-24T05:46:18.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.949+0000 7fa250a00640 1 -- 172.21.15.123:0/3428325613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa24c15e640 msgr2=0x7fa24c06cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:18.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.949+0000 7fa250a00640 1 --2- 172.21.15.123:0/3428325613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa24c15e640 0x7fa24c06cb40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa24c074180 tx=0x7fa23c02a660 comp rx=0 tx=0).stop 2024-04-24T05:46:18.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.949+0000 7fa250a00640 1 -- 172.21.15.123:0/3428325613 shutdown_connections 2024-04-24T05:46:18.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.949+0000 7fa250a00640 1 --2- 172.21.15.123:0/3428325613 >> v2:172.21.15.155:3300/0 conn(0x7fa24c06f730 0x7fa24c0acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.949+0000 7fa250a00640 1 --2- 172.21.15.123:0/3428325613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa24c15e640 0x7fa24c06cb40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.949+0000 7fa250a00640 1 -- 172.21.15.123:0/3428325613 >> 172.21.15.123:0/3428325613 conn(0x7fa24c0b1cf0 msgr2=0x7fa24c0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:18.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.950+0000 7fa250a00640 1 -- 172.21.15.123:0/3428325613 shutdown_connections 2024-04-24T05:46:18.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.950+0000 7fa250a00640 1 -- 172.21.15.123:0/3428325613 wait complete. 2024-04-24T05:46:18.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.951+0000 7fa250a00640 1 Processor -- start 2024-04-24T05:46:18.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.951+0000 7fa250a00640 1 -- start start 2024-04-24T05:46:18.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.952+0000 7fa250a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa24c06d080 0x7fa24c1430f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.952+0000 7fa250a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa24c06f730 0x7fa24c159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.952+0000 7fa250a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa24c15e640 0x7fa24c15a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.952+0000 7fa250a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0727b0 con 0x7fa24c15e640 2024-04-24T05:46:18.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.952+0000 7fa250a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c06c620 con 0x7fa24c06d080 2024-04-24T05:46:18.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.952+0000 7fa250a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa24c072930 con 0x7fa24c06f730 2024-04-24T05:46:18.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.953+0000 7fa24be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa24c15e640 0x7fa24c15a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:18.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.953+0000 7fa24be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa24c15e640 0x7fa24c15a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56300/0 (socket says 172.21.15.123:56300) 2024-04-24T05:46:18.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.953+0000 7fa24be00640 1 -- 172.21.15.123:0/2434245360 learned_addr learned my addr 172.21.15.123:0/2434245360 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:18.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.953+0000 7fa24b400640 1 --2- 172.21.15.123:0/2434245360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa24c06d080 0x7fa24c1430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:18.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.953+0000 7fa24be00640 1 -- 172.21.15.123:0/2434245360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa24c06f730 msgr2=0x7fa24c159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:18.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.953+0000 7fa24be00640 1 --2- 172.21.15.123:0/2434245360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa24c06f730 0x7fa24c159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.953+0000 7fa24be00640 1 -- 172.21.15.123:0/2434245360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa24c06d080 msgr2=0x7fa24c1430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:18.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.953+0000 7fa24be00640 1 --2- 172.21.15.123:0/2434245360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa24c06d080 0x7fa24c1430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:18.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.953+0000 7fa24be00640 1 -- 172.21.15.123:0/2434245360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c15aa40 con 0x7fa24c15e640 2024-04-24T05:46:18.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.953+0000 7fa24be00640 1 --2- 172.21.15.123:0/2434245360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa24c15e640 0x7fa24c15a150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa24400b440 tx=0x7fa24400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:18.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.954+0000 7fa243e00640 1 -- 172.21.15.123:0/2434245360 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244015020 con 0x7fa24c15e640 2024-04-24T05:46:18.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.954+0000 7fa243e00640 1 -- 172.21.15.123:0/2434245360 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa244011850 con 0x7fa24c15e640 2024-04-24T05:46:18.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.954+0000 7fa243e00640 1 -- 172.21.15.123:0/2434245360 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244011c40 con 0x7fa24c15e640 2024-04-24T05:46:18.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.954+0000 7fa250a00640 1 -- 172.21.15.123:0/2434245360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24c15ad20 con 0x7fa24c15e640 2024-04-24T05:46:18.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.954+0000 7fa250a00640 1 -- 172.21.15.123:0/2434245360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24c155b30 con 0x7fa24c15e640 2024-04-24T05:46:18.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.955+0000 7fa250a00640 1 -- 172.21.15.123:0/2434245360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24c155d30 con 0x7fa24c15e640 2024-04-24T05:46:18.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.956+0000 7fa243e00640 1 -- 172.21.15.123:0/2434245360 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa244018020 con 0x7fa24c15e640 2024-04-24T05:46:18.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.957+0000 7fa243e00640 1 --2- 172.21.15.123:0/2434245360 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa22806c8f0 0x7fa22806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:18.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.957+0000 7fa243e00640 1 -- 172.21.15.123:0/2434245360 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa24408bc60 con 0x7fa24c15e640 2024-04-24T05:46:18.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.957+0000 7fa24b400640 1 --2- 172.21.15.123:0/2434245360 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa22806c8f0 0x7fa22806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:18.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.958+0000 7fa24b400640 1 --2- 172.21.15.123:0/2434245360 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa22806c8f0 0x7fa22806edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa23c033000 tx=0x7fa23c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:18.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:18.961+0000 7fa243e00640 1 -- 172.21.15.123:0/2434245360 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa24c155d30 con 0x7fa24c15e640 2024-04-24T05:46:19.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.202+0000 7fa250a00640 1 -- 172.21.15.123:0/2434245360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa24c068a20 con 0x7fa24c15e640 2024-04-24T05:46:19.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.332+0000 7fa243e00640 1 -- 172.21.15.123:0/2434245360 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fa24401f020 con 0x7fa24c15e640 2024-04-24T05:46:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.335+0000 7fa250a00640 1 -- 172.21.15.123:0/2434245360 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa22806c8f0 msgr2=0x7fa22806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.335+0000 7fa250a00640 1 --2- 172.21.15.123:0/2434245360 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa22806c8f0 0x7fa22806edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa23c033000 tx=0x7fa23c009000 comp rx=0 tx=0).stop 2024-04-24T05:46:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.335+0000 7fa250a00640 1 -- 172.21.15.123:0/2434245360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa24c15e640 msgr2=0x7fa24c15a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.335+0000 7fa250a00640 1 --2- 172.21.15.123:0/2434245360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa24c15e640 0x7fa24c15a150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa24400b440 tx=0x7fa24400b900 comp rx=0 tx=0).stop 2024-04-24T05:46:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.336+0000 7fa250a00640 1 -- 172.21.15.123:0/2434245360 shutdown_connections 2024-04-24T05:46:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.336+0000 7fa250a00640 1 --2- 172.21.15.123:0/2434245360 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa22806c8f0 0x7fa22806edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.336+0000 7fa250a00640 1 --2- 172.21.15.123:0/2434245360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa24c15e640 0x7fa24c15a150 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.336+0000 7fa250a00640 1 --2- 172.21.15.123:0/2434245360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa24c06f730 0x7fa24c159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.336+0000 7fa250a00640 1 --2- 172.21.15.123:0/2434245360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa24c06d080 0x7fa24c1430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:19.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.336+0000 7fa250a00640 1 -- 172.21.15.123:0/2434245360 >> 172.21.15.123:0/2434245360 conn(0x7fa24c0b1cf0 msgr2=0x7fa24c071b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:19.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.336+0000 7fa250a00640 1 -- 172.21.15.123:0/2434245360 shutdown_connections 2024-04-24T05:46:19.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.337+0000 7fa250a00640 1 -- 172.21.15.123:0/2434245360 wait complete. 2024-04-24T05:46:19.357 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-24T05:46:19.358 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T05:46:19.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.559+0000 7fdd48800640 1 Processor -- start 2024-04-24T05:46:19.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.560+0000 7fdd48800640 1 -- start start 2024-04-24T05:46:19.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.561+0000 7fdd48800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdd400ae990 0x7fdd400d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:19.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.561+0000 7fdd48800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd4006c410 0x7fdd400ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:19.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.561+0000 7fdd48800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdd400ce610 0x7fdd4015f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:19.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.561+0000 7fdd48800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdd400cea70 con 0x7fdd400ce610 2024-04-24T05:46:19.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.561+0000 7fdd48800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdd4006dcb0 con 0x7fdd400ae990 2024-04-24T05:46:19.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.561+0000 7fdd48800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdd4006e270 con 0x7fdd4006c410 2024-04-24T05:46:19.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.561+0000 7fdd46800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdd400ce610 0x7fdd4015f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:19.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.561+0000 7fdd46800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdd400ce610 0x7fdd4015f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56302/0 (socket says 172.21.15.123:56302) 2024-04-24T05:46:19.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.561+0000 7fdd46800640 1 -- 172.21.15.123:0/4150650666 learned_addr learned my addr 172.21.15.123:0/4150650666 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:19.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.561+0000 7fdd45e00640 1 --2- 172.21.15.123:0/4150650666 >> v2:172.21.15.155:3300/0 conn(0x7fdd400ae990 0x7fdd400d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:19.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.562+0000 7fdd45400640 1 --2- 172.21.15.123:0/4150650666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd4006c410 0x7fdd400ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:19.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.562+0000 7fdd46800640 1 -- 172.21.15.123:0/4150650666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd4006c410 msgr2=0x7fdd400ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:19.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.562+0000 7fdd46800640 1 --2- 172.21.15.123:0/4150650666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd4006c410 0x7fdd400ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:19.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.562+0000 7fdd46800640 1 -- 172.21.15.123:0/4150650666 >> v2:172.21.15.155:3300/0 conn(0x7fdd400ae990 msgr2=0x7fdd400d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:19.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.562+0000 7fdd46800640 1 --2- 172.21.15.123:0/4150650666 >> v2:172.21.15.155:3300/0 conn(0x7fdd400ae990 0x7fdd400d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:19.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.562+0000 7fdd46800640 1 -- 172.21.15.123:0/4150650666 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd400d0740 con 0x7fdd400ce610 2024-04-24T05:46:19.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.563+0000 7fdd46800640 1 --2- 172.21.15.123:0/4150650666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400ce610 0x7fdd4015f410 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdd4006ff40 tx=0x7fdd3c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557437d62c2c45c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:19.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.563+0000 7fdd44a00640 1 -- 172.21.15.123:0/4150650666 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c036070 con 0x7fdd400ce610 2024-04-24T05:46:19.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.563+0000 7fdd44a00640 1 -- 172.21.15.123:0/4150650666 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd3c031600 con 0x7fdd400ce610 2024-04-24T05:46:19.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.563+0000 7fdd44a00640 1 -- 172.21.15.123:0/4150650666 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0319d0 con 0x7fdd400ce610 2024-04-24T05:46:19.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.564+0000 7fdd48800640 1 -- 172.21.15.123:0/4150650666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400ce610 msgr2=0x7fdd4015f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:19.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.564+0000 7fdd48800640 1 --2- 172.21.15.123:0/4150650666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400ce610 0x7fdd4015f410 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdd4006ff40 tx=0x7fdd3c02a6a0 comp rx=0 tx=0).stop 2024-04-24T05:46:19.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.564+0000 7fdd48800640 1 -- 172.21.15.123:0/4150650666 shutdown_connections 2024-04-24T05:46:19.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.564+0000 7fdd48800640 1 --2- 172.21.15.123:0/4150650666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400ce610 0x7fdd4015f410 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:19.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.564+0000 7fdd48800640 1 --2- 172.21.15.123:0/4150650666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd4006c410 0x7fdd400ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:19.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.564+0000 7fdd48800640 1 --2- 172.21.15.123:0/4150650666 >> v2:172.21.15.155:3300/0 conn(0x7fdd400ae990 0x7fdd400d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:19.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.564+0000 7fdd48800640 1 -- 172.21.15.123:0/4150650666 >> 172.21.15.123:0/4150650666 conn(0x7fdd400b51f0 msgr2=0x7fdd400b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:19.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.564+0000 7fdd48800640 1 -- 172.21.15.123:0/4150650666 shutdown_connections 2024-04-24T05:46:19.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.564+0000 7fdd48800640 1 -- 172.21.15.123:0/4150650666 wait complete. 2024-04-24T05:46:19.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.566+0000 7fdd48800640 1 Processor -- start 2024-04-24T05:46:19.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.566+0000 7fdd48800640 1 -- start start 2024-04-24T05:46:19.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd48800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd4006c410 0x7fdd401f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:19.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd48800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400ae990 0x7fdd400cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:19.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd48800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd400ce610 0x7fdd400af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:19.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd48800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdd400aed70 con 0x7fdd400ae990 2024-04-24T05:46:19.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd48800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4006df00 con 0x7fdd4006c410 2024-04-24T05:46:19.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd48800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdd4006c870 con 0x7fdd400ce610 2024-04-24T05:46:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd45400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400ae990 0x7fdd400cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd45400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400ae990 0x7fdd400cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56316/0 (socket says 172.21.15.123:56316) 2024-04-24T05:46:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd45400640 1 -- 172.21.15.123:0/1625895676 learned_addr learned my addr 172.21.15.123:0/1625895676 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd45e00640 1 --2- 172.21.15.123:0/1625895676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd4006c410 0x7fdd401f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd46800640 1 --2- 172.21.15.123:0/1625895676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd400ce610 0x7fdd400af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd45400640 1 -- 172.21.15.123:0/1625895676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd400ce610 msgr2=0x7fdd400af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd45400640 1 --2- 172.21.15.123:0/1625895676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd400ce610 0x7fdd400af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd45400640 1 -- 172.21.15.123:0/1625895676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd4006c410 msgr2=0x7fdd401f2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd45400640 1 --2- 172.21.15.123:0/1625895676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd4006c410 0x7fdd401f2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.567+0000 7fdd45400640 1 -- 172.21.15.123:0/1625895676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd401f33b0 con 0x7fdd400ae990 2024-04-24T05:46:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.568+0000 7fdd45400640 1 --2- 172.21.15.123:0/1625895676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400ae990 0x7fdd400cce30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000d800 tx=0x7fdd3000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.568+0000 7fdd3a000640 1 -- 172.21.15.123:0/1625895676 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30018020 con 0x7fdd400ae990 2024-04-24T05:46:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.568+0000 7fdd3a000640 1 -- 172.21.15.123:0/1625895676 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd300096a0 con 0x7fdd400ae990 2024-04-24T05:46:19.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.568+0000 7fdd3a000640 1 -- 172.21.15.123:0/1625895676 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30009a70 con 0x7fdd400ae990 2024-04-24T05:46:19.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.568+0000 7fdd48800640 1 -- 172.21.15.123:0/1625895676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd4015cf00 con 0x7fdd400ae990 2024-04-24T05:46:19.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.568+0000 7fdd48800640 1 -- 172.21.15.123:0/1625895676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd4015d4c0 con 0x7fdd400ae990 2024-04-24T05:46:19.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.569+0000 7fdd48800640 1 -- 172.21.15.123:0/1625895676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd40068a20 con 0x7fdd400ae990 2024-04-24T05:46:19.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.570+0000 7fdd3a000640 1 -- 172.21.15.123:0/1625895676 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd3001b020 con 0x7fdd400ae990 2024-04-24T05:46:19.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.571+0000 7fdd3a000640 1 --2- 172.21.15.123:0/1625895676 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdd2406c8f0 0x7fdd2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:19.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.571+0000 7fdd3a000640 1 -- 172.21.15.123:0/1625895676 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdd3008c090 con 0x7fdd400ae990 2024-04-24T05:46:19.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.572+0000 7fdd45e00640 1 --2- 172.21.15.123:0/1625895676 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdd2406c8f0 0x7fdd2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:19.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.572+0000 7fdd45e00640 1 --2- 172.21.15.123:0/1625895676 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdd2406c8f0 0x7fdd2406edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd34006fd0 tx=0x7fdd34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:19.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.576+0000 7fdd3a000640 1 -- 172.21.15.123:0/1625895676 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd30022030 con 0x7fdd400ae990 2024-04-24T05:46:19.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:19.811+0000 7fdd48800640 1 -- 172.21.15.123:0/1625895676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdd400cc660 con 0x7fdd400ae990 2024-04-24T05:46:20.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.336+0000 7fdd3a000640 1 -- 172.21.15.123:0/1625895676 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7fdd401f33b0 con 0x7fdd400ae990 2024-04-24T05:46:20.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.339+0000 7fdd48800640 1 -- 172.21.15.123:0/1625895676 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdd2406c8f0 msgr2=0x7fdd2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:20.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.339+0000 7fdd48800640 1 --2- 172.21.15.123:0/1625895676 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdd2406c8f0 0x7fdd2406edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd34006fd0 tx=0x7fdd34008040 comp rx=0 tx=0).stop 2024-04-24T05:46:20.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.339+0000 7fdd48800640 1 -- 172.21.15.123:0/1625895676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400ae990 msgr2=0x7fdd400cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:20.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.339+0000 7fdd48800640 1 --2- 172.21.15.123:0/1625895676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400ae990 0x7fdd400cce30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000d800 tx=0x7fdd3000dcc0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.339+0000 7fdd48800640 1 -- 172.21.15.123:0/1625895676 shutdown_connections 2024-04-24T05:46:20.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.339+0000 7fdd48800640 1 --2- 172.21.15.123:0/1625895676 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdd2406c8f0 0x7fdd2406edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.339+0000 7fdd48800640 1 --2- 172.21.15.123:0/1625895676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd400ce610 0x7fdd400af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.339+0000 7fdd48800640 1 --2- 172.21.15.123:0/1625895676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400ae990 0x7fdd400cce30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.339+0000 7fdd48800640 1 --2- 172.21.15.123:0/1625895676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd4006c410 0x7fdd401f2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.339+0000 7fdd48800640 1 -- 172.21.15.123:0/1625895676 >> 172.21.15.123:0/1625895676 conn(0x7fdd400b51f0 msgr2=0x7fdd400ad040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:20.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.340+0000 7fdd48800640 1 -- 172.21.15.123:0/1625895676 shutdown_connections 2024-04-24T05:46:20.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.340+0000 7fdd48800640 1 -- 172.21.15.123:0/1625895676 wait complete. 2024-04-24T05:46:20.364 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T05:46:20.364 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T05:46:20.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.559+0000 7f7a8be00640 1 Processor -- start 2024-04-24T05:46:20.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.559+0000 7f7a8be00640 1 -- start start 2024-04-24T05:46:20.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.561+0000 7f7a8be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7a8c0d04b0 0x7f7a8c162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:20.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.561+0000 7f7a8be00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a8c0b6cf0 con 0x7f7a8c15e7f0 2024-04-24T05:46:20.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.561+0000 7f7a8be00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a8c0cee10 con 0x7f7a8c15ebd0 2024-04-24T05:46:20.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.561+0000 7f7a8be00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7a8c163480 con 0x7f7a8c0d04b0 2024-04-24T05:46:20.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.561+0000 7f7a8aa00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7a8c15e7f0 0x7f7a8c0cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35458/0 (socket says 172.21.15.123:35458) 2024-04-24T05:46:20.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.561+0000 7f7a8aa00640 1 -- 172.21.15.123:0/2381606326 learned_addr learned my addr 172.21.15.123:0/2381606326 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:20.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.561+0000 7f7a8a000640 1 --2- 172.21.15.123:0/2381606326 >> v2:172.21.15.155:3300/0 conn(0x7f7a8c0d04b0 0x7f7a8c162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:20.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 -- 172.21.15.123:0/2381606326 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4114896671 0 0) 0x7f7a8c0b6cf0 con 0x7f7a8c15e7f0 2024-04-24T05:46:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 -- 172.21.15.123:0/2381606326 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a70003520 con 0x7f7a8c15e7f0 2024-04-24T05:46:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 -- 172.21.15.123:0/2381606326 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 910757492 0 0) 0x7f7a70003520 con 0x7f7a8c15e7f0 2024-04-24T05:46:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 -- 172.21.15.123:0/2381606326 >> v2:172.21.15.155:3300/0 conn(0x7f7a8c0d04b0 msgr2=0x7f7a8c162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 --2- 172.21.15.123:0/2381606326 >> v2:172.21.15.155:3300/0 conn(0x7f7a8c0d04b0 0x7f7a8c162f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 -- 172.21.15.123:0/2381606326 >> v1:172.21.15.155:6789/0 conn(0x7f7a8c15ebd0 legacy=0x7f7a8c168860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 -- 172.21.15.123:0/2381606326 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c0cf0f0 con 0x7f7a8c15e7f0 2024-04-24T05:46:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 -- 172.21.15.123:0/2381606326 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f7a74001670 con 0x7f7a8c15e7f0 2024-04-24T05:46:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 -- 172.21.15.123:0/2381606326 >> v1:172.21.15.123:6789/0 conn(0x7f7a8c15e7f0 legacy=0x7f7a8c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 --2- 172.21.15.123:0/2381606326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a70003a00 0x7f7a70003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 --2- 172.21.15.123:0/2381606326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a70004800 0x7f7a70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 --2- 172.21.15.123:0/2381606326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a70005210 0x7f7a7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 -- 172.21.15.123:0/2381606326 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c163480 con 0x7f7a70005210 2024-04-24T05:46:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 -- 172.21.15.123:0/2381606326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7000c160 con 0x7f7a70003a00 2024-04-24T05:46:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.562+0000 7f7a89600640 1 -- 172.21.15.123:0/2381606326 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a7000c360 con 0x7f7a70004800 2024-04-24T05:46:20.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.563+0000 7f7a8b400640 1 --2- 172.21.15.123:0/2381606326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a70005210 0x7f7a7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:20.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.563+0000 7f7a8aa00640 1 --2- 172.21.15.123:0/2381606326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a70003a00 0x7f7a70003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:20.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.563+0000 7f7a8b400640 1 -- 172.21.15.123:0/2381606326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a70004800 msgr2=0x7f7a70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:20.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.563+0000 7f7a8b400640 1 --2- 172.21.15.123:0/2381606326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a70004800 0x7f7a70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.563+0000 7f7a8b400640 1 -- 172.21.15.123:0/2381606326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a70003a00 msgr2=0x7f7a70003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:20.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.563+0000 7f7a8b400640 1 --2- 172.21.15.123:0/2381606326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a70003a00 0x7f7a70003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.563+0000 7f7a8b400640 1 -- 172.21.15.123:0/2381606326 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a70003520 con 0x7f7a70005210 2024-04-24T05:46:20.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.563+0000 7f7a8aa00640 1 --2- 172.21.15.123:0/2381606326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a70003a00 0x7f7a70003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:20.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.564+0000 7f7a8b400640 1 -- 172.21.15.123:0/2381606326 reap_dead start 2024-04-24T05:46:20.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.564+0000 7f7a8b400640 1 --2- 172.21.15.123:0/2381606326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a70005210 0x7f7a7000b9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c15f000 tx=0x7f7a70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21bde769ee7a53a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:20.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.564+0000 7f7a89600640 1 -- 172.21.15.123:0/2381606326 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a84001370 con 0x7f7a70005210 2024-04-24T05:46:20.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.564+0000 7f7a89600640 1 -- 172.21.15.123:0/2381606326 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a840052d0 con 0x7f7a70005210 2024-04-24T05:46:20.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.564+0000 7f7a89600640 1 -- 172.21.15.123:0/2381606326 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8402ab50 con 0x7f7a70005210 2024-04-24T05:46:20.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.565+0000 7f7a8be00640 1 -- 172.21.15.123:0/2381606326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a70005210 msgr2=0x7f7a7000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:20.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.565+0000 7f7a8be00640 1 --2- 172.21.15.123:0/2381606326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a70005210 0x7f7a7000b9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c15f000 tx=0x7f7a70005830 comp rx=0 tx=0).stop 2024-04-24T05:46:20.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.565+0000 7f7a8be00640 1 -- 172.21.15.123:0/2381606326 shutdown_connections 2024-04-24T05:46:20.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.565+0000 7f7a8be00640 1 --2- 172.21.15.123:0/2381606326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a70005210 0x7f7a7000b9c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.565+0000 7f7a8be00640 1 -- 172.21.15.123:0/2381606326 >> 172.21.15.123:0/2381606326 conn(0x7f7a8c0b9530 msgr2=0x7f7a8c0b9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:20.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.565+0000 7f7a8be00640 1 -- 172.21.15.123:0/2381606326 shutdown_connections 2024-04-24T05:46:20.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.565+0000 7f7a8be00640 1 -- 172.21.15.123:0/2381606326 wait complete. 2024-04-24T05:46:20.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.566+0000 7f7a8be00640 1 Processor -- start 2024-04-24T05:46:20.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.567+0000 7f7a8be00640 1 -- start start 2024-04-24T05:46:20.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.567+0000 7f7a8be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a70005210 0x7f7a8c0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.567+0000 7f7a8be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a8c0cc4a0 0x7f7a8c1611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.567+0000 7f7a8be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a8c1616f0 0x7f7a8c1f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.568+0000 7f7a8be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c0cf8f0 con 0x7f7a8c0cc4a0 2024-04-24T05:46:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.568+0000 7f7a8be00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c0cebc0 con 0x7f7a70005210 2024-04-24T05:46:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.568+0000 7f7a8be00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8c0cfa70 con 0x7f7a8c1616f0 2024-04-24T05:46:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.568+0000 7f7a8a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a8c0cc4a0 0x7f7a8c1611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.568+0000 7f7a8a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a8c0cc4a0 0x7f7a8c1611b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56336/0 (socket says 172.21.15.123:56336) 2024-04-24T05:46:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.568+0000 7f7a8a000640 1 -- 172.21.15.123:0/4219546625 learned_addr learned my addr 172.21.15.123:0/4219546625 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.568+0000 7f7a8aa00640 1 --2- 172.21.15.123:0/4219546625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a70005210 0x7f7a8c0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.568+0000 7f7a8b400640 1 --2- 172.21.15.123:0/4219546625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a8c1616f0 0x7f7a8c1f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.568+0000 7f7a8a000640 1 -- 172.21.15.123:0/4219546625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a8c1616f0 msgr2=0x7f7a8c1f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.568+0000 7f7a8a000640 1 --2- 172.21.15.123:0/4219546625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a8c1616f0 0x7f7a8c1f4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.568+0000 7f7a8a000640 1 -- 172.21.15.123:0/4219546625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a70005210 msgr2=0x7f7a8c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.568+0000 7f7a8a000640 1 --2- 172.21.15.123:0/4219546625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a70005210 0x7f7a8c0cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.568+0000 7f7a8a000640 1 -- 172.21.15.123:0/4219546625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c15d7e0 con 0x7f7a8c0cc4a0 2024-04-24T05:46:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.569+0000 7f7a8a000640 1 --2- 172.21.15.123:0/4219546625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a8c0cc4a0 0x7f7a8c1611b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00ef70 tx=0x7f7a7c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.569+0000 7f7a82a00640 1 -- 172.21.15.123:0/4219546625 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c018020 con 0x7f7a8c0cc4a0 2024-04-24T05:46:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.569+0000 7f7a82a00640 1 -- 172.21.15.123:0/4219546625 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a7c014400 con 0x7f7a8c0cc4a0 2024-04-24T05:46:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.569+0000 7f7a82a00640 1 -- 172.21.15.123:0/4219546625 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0147f0 con 0x7f7a8c0cc4a0 2024-04-24T05:46:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.569+0000 7f7a8be00640 1 -- 172.21.15.123:0/4219546625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c1f53c0 con 0x7f7a8c0cc4a0 2024-04-24T05:46:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.569+0000 7f7a8be00640 1 -- 172.21.15.123:0/4219546625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c0cb9c0 con 0x7f7a8c0cc4a0 2024-04-24T05:46:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.570+0000 7f7a8be00640 1 -- 172.21.15.123:0/4219546625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8c075b70 con 0x7f7a8c0cc4a0 2024-04-24T05:46:20.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.571+0000 7f7a82a00640 1 -- 172.21.15.123:0/4219546625 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a7c005d10 con 0x7f7a8c0cc4a0 2024-04-24T05:46:20.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.572+0000 7f7a82a00640 1 --2- 172.21.15.123:0/4219546625 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7a6806c8f0 0x7f7a6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:20.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.572+0000 7f7a82a00640 1 -- 172.21.15.123:0/4219546625 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7a7c08c930 con 0x7f7a8c0cc4a0 2024-04-24T05:46:20.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.572+0000 7f7a8aa00640 1 --2- 172.21.15.123:0/4219546625 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7a6806c8f0 0x7f7a6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:20.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.573+0000 7f7a8aa00640 1 --2- 172.21.15.123:0/4219546625 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7a6806c8f0 0x7f7a6806edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7a740017e0 tx=0x7f7a74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:20.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.577+0000 7f7a82a00640 1 -- 172.21.15.123:0/4219546625 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a7c022030 con 0x7f7a8c0cc4a0 2024-04-24T05:46:20.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.795+0000 7f7a8be00640 1 -- 172.21.15.123:0/4219546625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a8c0b9b20 con 0x7f7a8c0cc4a0 2024-04-24T05:46:20.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.795+0000 7f7a82a00640 1 -- 172.21.15.123:0/4219546625 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f7a8c15d7e0 con 0x7f7a8c0cc4a0 2024-04-24T05:46:20.797 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:20.797 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2743935658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2743935658},{"type":"v1","addr":"172.21.15.155:6834","nonce":2743935658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1148561953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1148561953},{"type":"v1","addr":"172.21.15.123:6836","nonce":1148561953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2992052647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2992052647},{"type":"v1","addr":"172.21.15.155:6835","nonce":2992052647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T05:45:59.639548+0000","modified":"2024-04-24T05:46:19.813408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/3370584410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3370584410},{"type":"v1","addr":"172.21.15.123:6837","nonce":3370584410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T05:46:20.798 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-24T05:46:20.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.798+0000 7f7a8be00640 1 -- 172.21.15.123:0/4219546625 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7a6806c8f0 msgr2=0x7f7a6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:20.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.798+0000 7f7a8be00640 1 --2- 172.21.15.123:0/4219546625 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7a6806c8f0 0x7f7a6806edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7a740017e0 tx=0x7f7a74009040 comp rx=0 tx=0).stop 2024-04-24T05:46:20.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.798+0000 7f7a8be00640 1 -- 172.21.15.123:0/4219546625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a8c0cc4a0 msgr2=0x7f7a8c1611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:20.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.798+0000 7f7a8be00640 1 --2- 172.21.15.123:0/4219546625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a8c0cc4a0 0x7f7a8c1611b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00ef70 tx=0x7f7a7c00b480 comp rx=0 tx=0).stop 2024-04-24T05:46:20.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.798+0000 7f7a8be00640 1 -- 172.21.15.123:0/4219546625 shutdown_connections 2024-04-24T05:46:20.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.798+0000 7f7a8be00640 1 --2- 172.21.15.123:0/4219546625 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7a6806c8f0 0x7f7a6806edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.798+0000 7f7a8be00640 1 --2- 172.21.15.123:0/4219546625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a8c1616f0 0x7f7a8c1f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.798+0000 7f7a8be00640 1 --2- 172.21.15.123:0/4219546625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a8c0cc4a0 0x7f7a8c1611b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.798+0000 7f7a8be00640 1 --2- 172.21.15.123:0/4219546625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a70005210 0x7f7a8c0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:20.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.798+0000 7f7a8be00640 1 -- 172.21.15.123:0/4219546625 >> 172.21.15.123:0/4219546625 conn(0x7f7a8c0b9530 msgr2=0x7f7a8c0b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:20.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.798+0000 7f7a8be00640 1 -- 172.21.15.123:0/4219546625 shutdown_connections 2024-04-24T05:46:20.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:20.798+0000 7f7a8be00640 1 -- 172.21.15.123:0/4219546625 wait complete. 2024-04-24T05:46:20.818 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:21.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.009+0000 7fb38f000640 1 Processor -- start 2024-04-24T05:46:21.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.010+0000 7fb38f000640 1 -- start start 2024-04-24T05:46:21.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.011+0000 7fb38f000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb38815a2a0 0x7fb3880ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.011+0000 7fb38f000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb3880af950 0x7fb38806bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.011+0000 7fb38f000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb38815cdf0 con 0x7fb3880ad690 2024-04-24T05:46:21.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.011+0000 7fb38f000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb38815a680 con 0x7fb38815a2a0 2024-04-24T05:46:21.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.011+0000 7fb38f000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb38815da00 con 0x7fb3880af950 2024-04-24T05:46:21.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.011+0000 7fb387e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb38815a2a0 0x7fb3880ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.011+0000 7fb37f400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb3880ad690 0x7fb3880af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35460/0 (socket says 172.21.15.123:35460) 2024-04-24T05:46:21.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.011+0000 7fb37f400640 1 -- 172.21.15.123:0/781559452 learned_addr learned my addr 172.21.15.123:0/781559452 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:21.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.012+0000 7fb38d000640 1 --2- 172.21.15.123:0/781559452 >> v2:172.21.15.155:3300/0 conn(0x7fb3880af950 0x7fb38806bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.012+0000 7fb387e00640 1 -- 172.21.15.123:0/781559452 >> v2:172.21.15.155:3300/0 conn(0x7fb3880af950 msgr2=0x7fb38806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:21.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.012+0000 7fb387e00640 1 --2- 172.21.15.123:0/781559452 >> v2:172.21.15.155:3300/0 conn(0x7fb3880af950 0x7fb38806bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.012+0000 7fb387e00640 1 -- 172.21.15.123:0/781559452 >> v1:172.21.15.123:6789/0 conn(0x7fb3880ad690 legacy=0x7fb3880af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.012+0000 7fb387e00640 1 -- 172.21.15.123:0/781559452 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38815d0d0 con 0x7fb38815a2a0 2024-04-24T05:46:21.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.013+0000 7fb387e00640 1 --2- 172.21.15.123:0/781559452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb38815a2a0 0x7fb3880ad150 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb388075290 tx=0x7fb37802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26539e75de429913 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:21.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.013+0000 7fb387400640 1 -- 172.21.15.123:0/781559452 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378037070 con 0x7fb38815a2a0 2024-04-24T05:46:21.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.013+0000 7fb387400640 1 -- 172.21.15.123:0/781559452 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb378032440 con 0x7fb38815a2a0 2024-04-24T05:46:21.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.013+0000 7fb387400640 1 -- 172.21.15.123:0/781559452 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37802ae60 con 0x7fb38815a2a0 2024-04-24T05:46:21.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.014+0000 7fb38f000640 1 -- 172.21.15.123:0/781559452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb38815a2a0 msgr2=0x7fb3880ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.014+0000 7fb38f000640 1 --2- 172.21.15.123:0/781559452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb38815a2a0 0x7fb3880ad150 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb388075290 tx=0x7fb37802a110 comp rx=0 tx=0).stop 2024-04-24T05:46:21.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.014+0000 7fb38f000640 1 -- 172.21.15.123:0/781559452 shutdown_connections 2024-04-24T05:46:21.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.014+0000 7fb38f000640 1 --2- 172.21.15.123:0/781559452 >> v2:172.21.15.155:3300/0 conn(0x7fb3880af950 0x7fb38806bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.014+0000 7fb38f000640 1 --2- 172.21.15.123:0/781559452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb38815a2a0 0x7fb3880ad150 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.014+0000 7fb38f000640 1 -- 172.21.15.123:0/781559452 >> 172.21.15.123:0/781559452 conn(0x7fb3880b1e70 msgr2=0x7fb3880b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:21.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.014+0000 7fb38f000640 1 -- 172.21.15.123:0/781559452 shutdown_connections 2024-04-24T05:46:21.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.014+0000 7fb38f000640 1 -- 172.21.15.123:0/781559452 wait complete. 2024-04-24T05:46:21.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.016+0000 7fb38f000640 1 Processor -- start 2024-04-24T05:46:21.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.016+0000 7fb38f000640 1 -- start start 2024-04-24T05:46:21.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb38f000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3880ad690 0x7fb388153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb38f000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3880af950 0x7fb388156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb38f000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb38815a2a0 0x7fb3881f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb38f000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb38806c040 con 0x7fb3880af950 2024-04-24T05:46:21.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb38f000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb38815d690 con 0x7fb38815a2a0 2024-04-24T05:46:21.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb38f000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb38806c1c0 con 0x7fb3880ad690 2024-04-24T05:46:21.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb37f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3880af950 0x7fb388156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb37f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3880af950 0x7fb388156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56342/0 (socket says 172.21.15.123:56342) 2024-04-24T05:46:21.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb37f400640 1 -- 172.21.15.123:0/68339291 learned_addr learned my addr 172.21.15.123:0/68339291 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:21.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb387e00640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3880ad690 0x7fb388153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb38d000640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb38815a2a0 0x7fb3881f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb37f400640 1 -- 172.21.15.123:0/68339291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3880ad690 msgr2=0x7fb388153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb37f400640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3880ad690 0x7fb388153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb37f400640 1 -- 172.21.15.123:0/68339291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb38815a2a0 msgr2=0x7fb3881f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb37f400640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb38815a2a0 0x7fb3881f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb37f400640 1 -- 172.21.15.123:0/68339291 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb388157410 con 0x7fb3880af950 2024-04-24T05:46:21.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.017+0000 7fb38d000640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb38815a2a0 0x7fb3881f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:21.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.018+0000 7fb37f400640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3880af950 0x7fb388156d50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c430 tx=0x7fb37000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:21.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.018+0000 7fb384c00640 1 -- 172.21.15.123:0/68339291 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370017070 con 0x7fb3880af950 2024-04-24T05:46:21.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.018+0000 7fb384c00640 1 -- 172.21.15.123:0/68339291 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb370005ce0 con 0x7fb3880af950 2024-04-24T05:46:21.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.018+0000 7fb384c00640 1 -- 172.21.15.123:0/68339291 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3700072f0 con 0x7fb3880af950 2024-04-24T05:46:21.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.018+0000 7fb38f000640 1 -- 172.21.15.123:0/68339291 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3881576f0 con 0x7fb3880af950 2024-04-24T05:46:21.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.018+0000 7fb38f000640 1 -- 172.21.15.123:0/68339291 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3880ce8a0 con 0x7fb3880af950 2024-04-24T05:46:21.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.019+0000 7fb38f000640 1 -- 172.21.15.123:0/68339291 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3880cf580 con 0x7fb3880af950 2024-04-24T05:46:21.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.020+0000 7fb384c00640 1 -- 172.21.15.123:0/68339291 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb37001a070 con 0x7fb3880af950 2024-04-24T05:46:21.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.021+0000 7fb384c00640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb36406c8f0 0x7fb36406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.021+0000 7fb384c00640 1 -- 172.21.15.123:0/68339291 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb37008b400 con 0x7fb3880af950 2024-04-24T05:46:21.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.021+0000 7fb387e00640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb36406c8f0 0x7fb36406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.022+0000 7fb387e00640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb36406c8f0 0x7fb36406edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb378034000 tx=0x7fb37802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:21.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.025+0000 7fb384c00640 1 -- 172.21.15.123:0/68339291 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3700593c0 con 0x7fb3880af950 2024-04-24T05:46:21.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.226+0000 7fb38f000640 1 -- 172.21.15.123:0/68339291 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3880ceaa0 con 0x7fb3880af950 2024-04-24T05:46:21.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.228+0000 7fb384c00640 1 -- 172.21.15.123:0/68339291 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7fb3880ceaa0 con 0x7fb3880af950 2024-04-24T05:46:21.230 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:21.230 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:04.715857+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:21.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.230+0000 7fb38f000640 1 -- 172.21.15.123:0/68339291 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb36406c8f0 msgr2=0x7fb36406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.230+0000 7fb38f000640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb36406c8f0 0x7fb36406edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb378034000 tx=0x7fb37802e000 comp rx=0 tx=0).stop 2024-04-24T05:46:21.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.230+0000 7fb38f000640 1 -- 172.21.15.123:0/68339291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3880af950 msgr2=0x7fb388156d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.230+0000 7fb38f000640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3880af950 0x7fb388156d50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c430 tx=0x7fb37000c8f0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.231+0000 7fb38f000640 1 -- 172.21.15.123:0/68339291 shutdown_connections 2024-04-24T05:46:21.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.231+0000 7fb38f000640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb36406c8f0 0x7fb36406edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.231+0000 7fb38f000640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb38815a2a0 0x7fb3881f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.231+0000 7fb38f000640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3880af950 0x7fb388156d50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.231+0000 7fb38f000640 1 --2- 172.21.15.123:0/68339291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb3880ad690 0x7fb388153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.231+0000 7fb38f000640 1 -- 172.21.15.123:0/68339291 >> 172.21.15.123:0/68339291 conn(0x7fb3880b1e70 msgr2=0x7fb3881552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:21.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.231+0000 7fb38f000640 1 -- 172.21.15.123:0/68339291 shutdown_connections 2024-04-24T05:46:21.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.231+0000 7fb38f000640 1 -- 172.21.15.123:0/68339291 wait complete. 2024-04-24T05:46:21.250 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T05:46:21.251 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T05:46:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.440+0000 7f7758a00640 1 Processor -- start 2024-04-24T05:46:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.440+0000 7f7758a00640 1 -- start start 2024-04-24T05:46:21.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.441+0000 7f7758a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7754145aa0 0x7f775415c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.441+0000 7f7758a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f775406fa50 con 0x7f77540735f0 2024-04-24T05:46:21.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.441+0000 7f7758a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f775406fd30 con 0x7f7754145e80 2024-04-24T05:46:21.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.441+0000 7f7758a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f77540cf740 con 0x7f7754145aa0 2024-04-24T05:46:21.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.441+0000 7f7752800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7754145aa0 0x7f775415c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.442+0000 7f7753200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f77540735f0 0x7f775406ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35474/0 (socket says 172.21.15.123:35474) 2024-04-24T05:46:21.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.442+0000 7f7752800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7754145aa0 0x7f775415c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56348/0 (socket says 172.21.15.123:56348) 2024-04-24T05:46:21.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.442+0000 7f7753200640 1 -- 172.21.15.123:0/2938507129 learned_addr learned my addr 172.21.15.123:0/2938507129 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:21.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.442+0000 7f7752800640 1 -- 172.21.15.123:0/2938507129 >> v1:172.21.15.155:6789/0 conn(0x7f7754145e80 legacy=0x7f7754072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:21.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.442+0000 7f7752800640 1 -- 172.21.15.123:0/2938507129 >> v1:172.21.15.123:6789/0 conn(0x7f77540735f0 legacy=0x7f775406ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.442+0000 7f7751400640 1 -- 172.21.15.123:0/2938507129 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1016769706 0 0) 0x7f775406fa50 con 0x7f77540735f0 2024-04-24T05:46:21.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.442+0000 7f7752800640 1 -- 172.21.15.123:0/2938507129 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77540cf990 con 0x7f7754145aa0 2024-04-24T05:46:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.443+0000 7f7752800640 1 --2- 172.21.15.123:0/2938507129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7754145aa0 0x7f775415c8c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f774c0097a0 tx=0x7f774c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b974b62c558bae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.443+0000 7f7751400640 1 -- 172.21.15.123:0/2938507129 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c037070 con 0x7f7754145aa0 2024-04-24T05:46:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.443+0000 7f7751400640 1 -- 172.21.15.123:0/2938507129 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774c02ae50 con 0x7f7754145aa0 2024-04-24T05:46:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.443+0000 7f7751400640 1 -- 172.21.15.123:0/2938507129 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c032660 con 0x7f7754145aa0 2024-04-24T05:46:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.444+0000 7f7758a00640 1 -- 172.21.15.123:0/2938507129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7754145aa0 msgr2=0x7f775415c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.444+0000 7f7758a00640 1 --2- 172.21.15.123:0/2938507129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7754145aa0 0x7f775415c8c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f774c0097a0 tx=0x7f774c02a140 comp rx=0 tx=0).stop 2024-04-24T05:46:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.444+0000 7f7758a00640 1 -- 172.21.15.123:0/2938507129 shutdown_connections 2024-04-24T05:46:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.444+0000 7f7758a00640 1 --2- 172.21.15.123:0/2938507129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7754145aa0 0x7f775415c8c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.444+0000 7f7758a00640 1 -- 172.21.15.123:0/2938507129 >> 172.21.15.123:0/2938507129 conn(0x7f77540b3100 msgr2=0x7f77540b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.444+0000 7f7758a00640 1 -- 172.21.15.123:0/2938507129 shutdown_connections 2024-04-24T05:46:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.444+0000 7f7758a00640 1 -- 172.21.15.123:0/2938507129 wait complete. 2024-04-24T05:46:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.446+0000 7f7758a00640 1 Processor -- start 2024-04-24T05:46:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.446+0000 7f7758a00640 1 -- start start 2024-04-24T05:46:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.447+0000 7f7758a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77540735f0 0x7f77540c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.447+0000 7f7758a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7754145aa0 0x7f77540cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.447+0000 7f7758a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7754145e80 0x7f77541f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.447+0000 7f7758a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f775406f810 con 0x7f7754145e80 2024-04-24T05:46:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.447+0000 7f7758a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f775406f690 con 0x7f7754145aa0 2024-04-24T05:46:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.447+0000 7f7758a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f77540cfd00 con 0x7f77540735f0 2024-04-24T05:46:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.447+0000 7f7753200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7754145e80 0x7f77541f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.447+0000 7f7753200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7754145e80 0x7f77541f4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56358/0 (socket says 172.21.15.123:56358) 2024-04-24T05:46:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.447+0000 7f7753200640 1 -- 172.21.15.123:0/3107011850 learned_addr learned my addr 172.21.15.123:0/3107011850 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.447+0000 7f7751e00640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7754145aa0 0x7f77540cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.448+0000 7f7752800640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77540735f0 0x7f77540c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.448+0000 7f7753200640 1 -- 172.21.15.123:0/3107011850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77540735f0 msgr2=0x7f77540c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.448+0000 7f7753200640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77540735f0 0x7f77540c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.448+0000 7f7753200640 1 -- 172.21.15.123:0/3107011850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7754145aa0 msgr2=0x7f77540cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.448+0000 7f7753200640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7754145aa0 0x7f77540cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.448+0000 7f7753200640 1 -- 172.21.15.123:0/3107011850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77540cceb0 con 0x7f7754145e80 2024-04-24T05:46:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.448+0000 7f7751e00640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7754145aa0 0x7f77540cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:46:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.448+0000 7f7753200640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7754145e80 0x7f77541f4a50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f774400c480 tx=0x7f774400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.448+0000 7f774aa00640 1 -- 172.21.15.123:0/3107011850 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744018070 con 0x7f7754145e80 2024-04-24T05:46:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.448+0000 7f774aa00640 1 -- 172.21.15.123:0/3107011850 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77440074e0 con 0x7f7754145e80 2024-04-24T05:46:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.448+0000 7f774aa00640 1 -- 172.21.15.123:0/3107011850 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744004a20 con 0x7f7754145e80 2024-04-24T05:46:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.448+0000 7f7758a00640 1 -- 172.21.15.123:0/3107011850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77541f5270 con 0x7f7754145e80 2024-04-24T05:46:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.448+0000 7f7758a00640 1 -- 172.21.15.123:0/3107011850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77541f5830 con 0x7f7754145e80 2024-04-24T05:46:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.449+0000 7f7758a00640 1 -- 172.21.15.123:0/3107011850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7754075b70 con 0x7f7754145e80 2024-04-24T05:46:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.450+0000 7f774aa00640 1 -- 172.21.15.123:0/3107011850 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7744004bc0 con 0x7f7754145e80 2024-04-24T05:46:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.451+0000 7f774aa00640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f773406c8f0 0x7f773406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.451+0000 7f774aa00640 1 -- 172.21.15.123:0/3107011850 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f774408b1e0 con 0x7f7754145e80 2024-04-24T05:46:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.452+0000 7f7752800640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f773406c8f0 0x7f773406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.452+0000 7f7752800640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f773406c8f0 0x7f773406edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f774c034000 tx=0x7f774c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:21.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.456+0000 7f774aa00640 1 -- 172.21.15.123:0/3107011850 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77540cceb0 con 0x7f7754145e80 2024-04-24T05:46:21.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.658+0000 7f7758a00640 1 -- 172.21.15.123:0/3107011850 --> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f77540c8c50 con 0x7f773406c8f0 2024-04-24T05:46:21.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.671+0000 7f774aa00640 1 -- 172.21.15.123:0/3107011850 <== mgr.4101 v2:172.21.15.123:6832/1503718323 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332574 (secure 0 0 0) 0x7f77540c8c50 con 0x7f773406c8f0 2024-04-24T05:46:21.674 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:21.680 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-24T05:46:21.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.679+0000 7f7758a00640 1 -- 172.21.15.123:0/3107011850 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f773406c8f0 msgr2=0x7f773406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.679+0000 7f7758a00640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f773406c8f0 0x7f773406edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f774c034000 tx=0x7f774c02e000 comp rx=0 tx=0).stop 2024-04-24T05:46:21.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.679+0000 7f7758a00640 1 -- 172.21.15.123:0/3107011850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7754145e80 msgr2=0x7f77541f4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.679+0000 7f7758a00640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7754145e80 0x7f77541f4a50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f774400c480 tx=0x7f774400c940 comp rx=0 tx=0).stop 2024-04-24T05:46:21.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.679+0000 7f7758a00640 1 -- 172.21.15.123:0/3107011850 shutdown_connections 2024-04-24T05:46:21.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.679+0000 7f7758a00640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f773406c8f0 0x7f773406edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.679+0000 7f7758a00640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7754145e80 0x7f77541f4a50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.679+0000 7f7758a00640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7754145aa0 0x7f77540cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.679+0000 7f7758a00640 1 --2- 172.21.15.123:0/3107011850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77540735f0 0x7f77540c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.679+0000 7f7758a00640 1 -- 172.21.15.123:0/3107011850 >> 172.21.15.123:0/3107011850 conn(0x7f77540b3100 msgr2=0x7f77540b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:21.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.679+0000 7f7758a00640 1 -- 172.21.15.123:0/3107011850 shutdown_connections 2024-04-24T05:46:21.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.679+0000 7f7758a00640 1 -- 172.21.15.123:0/3107011850 wait complete. 2024-04-24T05:46:21.699 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-24T05:46:20.517929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219476,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755244,"statfs":{"total":773094113280,"available":772869369856,"internally_reserved":0,"allocated":3383296,"data_stored":2032520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006734"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063974+0000","last_change":"2024-04-24T05:45:59.220218+0000","last_active":"2024-04-24T05:46:06.063974+0000","last_peered":"2024-04-24T05:46:06.063974+0000","last_clean":"2024-04-24T05:46:06.063974+0000","last_became_active":"2024-04-24T05:45:59.219888+0000","last_became_peered":"2024-04-24T05:45:59.219888+0000","last_unstale":"2024-04-24T05:46:06.063974+0000","last_undegraded":"2024-04-24T05:46:06.063974+0000","last_fullsized":"2024-04-24T05:46:06.063974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062489+0000","last_change":"2024-04-24T05:46:00.218964+0000","last_active":"2024-04-24T05:46:06.062489+0000","last_peered":"2024-04-24T05:46:06.062489+0000","last_clean":"2024-04-24T05:46:06.062489+0000","last_became_active":"2024-04-24T05:46:00.218589+0000","last_became_peered":"2024-04-24T05:46:00.218589+0000","last_unstale":"2024-04-24T05:46:06.062489+0000","last_undegraded":"2024-04-24T05:46:06.062489+0000","last_fullsized":"2024-04-24T05:46:06.062489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:42.688050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059820+0000","last_change":"2024-04-24T05:46:04.525402+0000","last_active":"2024-04-24T05:46:06.059820+0000","last_peered":"2024-04-24T05:46:06.059820+0000","last_clean":"2024-04-24T05:46:06.059820+0000","last_became_active":"2024-04-24T05:46:04.524690+0000","last_became_peered":"2024-04-24T05:46:04.524690+0000","last_unstale":"2024-04-24T05:46:06.059820+0000","last_undegraded":"2024-04-24T05:46:06.059820+0000","last_fullsized":"2024-04-24T05:46:06.059820+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:15:19.979319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062738+0000","last_change":"2024-04-24T05:45:59.203824+0000","last_active":"2024-04-24T05:46:06.062738+0000","last_peered":"2024-04-24T05:46:06.062738+0000","last_clean":"2024-04-24T05:46:06.062738+0000","last_became_active":"2024-04-24T05:45:59.203396+0000","last_became_peered":"2024-04-24T05:45:59.203396+0000","last_unstale":"2024-04-24T05:46:06.062738+0000","last_undegraded":"2024-04-24T05:46:06.062738+0000","last_fullsized":"2024-04-24T05:46:06.062738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:53:31.596793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063701+0000","last_change":"2024-04-24T05:46:00.221369+0000","last_active":"2024-04-24T05:46:06.063701+0000","last_peered":"2024-04-24T05:46:06.063701+0000","last_clean":"2024-04-24T05:46:06.063701+0000","last_became_active":"2024-04-24T05:46:00.220890+0000","last_became_peered":"2024-04-24T05:46:00.220890+0000","last_unstale":"2024-04-24T05:46:06.063701+0000","last_undegraded":"2024-04-24T05:46:06.063701+0000","last_fullsized":"2024-04-24T05:46:06.063701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:27.080921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.246725+0000","last_change":"2024-04-24T05:46:04.274351+0000","last_active":"2024-04-24T05:46:05.246725+0000","last_peered":"2024-04-24T05:46:05.246725+0000","last_clean":"2024-04-24T05:46:05.246725+0000","last_became_active":"2024-04-24T05:46:04.273892+0000","last_became_peered":"2024-04-24T05:46:04.273892+0000","last_unstale":"2024-04-24T05:46:05.246725+0000","last_undegraded":"2024-04-24T05:46:05.246725+0000","last_fullsized":"2024-04-24T05:46:05.246725+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:39.015857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557980+0000","last_change":"2024-04-24T05:45:59.221936+0000","last_active":"2024-04-24T05:46:05.557980+0000","last_peered":"2024-04-24T05:46:05.557980+0000","last_clean":"2024-04-24T05:46:05.557980+0000","last_became_active":"2024-04-24T05:45:59.221563+0000","last_became_peered":"2024-04-24T05:45:59.221563+0000","last_unstale":"2024-04-24T05:46:05.557980+0000","last_undegraded":"2024-04-24T05:46:05.557980+0000","last_fullsized":"2024-04-24T05:46:05.557980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:34.072620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252559+0000","last_change":"2024-04-24T05:46:00.204200+0000","last_active":"2024-04-24T05:46:05.252559+0000","last_peered":"2024-04-24T05:46:05.252559+0000","last_clean":"2024-04-24T05:46:05.252559+0000","last_became_active":"2024-04-24T05:46:00.203884+0000","last_became_peered":"2024-04-24T05:46:00.203884+0000","last_unstale":"2024-04-24T05:46:05.252559+0000","last_undegraded":"2024-04-24T05:46:05.252559+0000","last_fullsized":"2024-04-24T05:46:05.252559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:05:06.286589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248046+0000","last_change":"2024-04-24T05:46:04.279449+0000","last_active":"2024-04-24T05:46:05.248046+0000","last_peered":"2024-04-24T05:46:05.248046+0000","last_clean":"2024-04-24T05:46:05.248046+0000","last_became_active":"2024-04-24T05:46:04.278915+0000","last_became_peered":"2024-04-24T05:46:04.278915+0000","last_unstale":"2024-04-24T05:46:05.248046+0000","last_undegraded":"2024-04-24T05:46:05.248046+0000","last_fullsized":"2024-04-24T05:46:05.248046+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:42.160187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557287+0000","last_change":"2024-04-24T05:45:59.220243+0000","last_active":"2024-04-24T05:46:05.557287+0000","last_peered":"2024-04-24T05:46:05.557287+0000","last_clean":"2024-04-24T05:46:05.557287+0000","last_became_active":"2024-04-24T05:45:59.219288+0000","last_became_peered":"2024-04-24T05:45:59.219288+0000","last_unstale":"2024-04-24T05:46:05.557287+0000","last_undegraded":"2024-04-24T05:46:05.557287+0000","last_fullsized":"2024-04-24T05:46:05.557287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253052+0000","last_change":"2024-04-24T05:46:00.208516+0000","last_active":"2024-04-24T05:46:05.253052+0000","last_peered":"2024-04-24T05:46:05.253052+0000","last_clean":"2024-04-24T05:46:05.253052+0000","last_became_active":"2024-04-24T05:46:00.208217+0000","last_became_peered":"2024-04-24T05:46:00.208217+0000","last_unstale":"2024-04-24T05:46:05.253052+0000","last_undegraded":"2024-04-24T05:46:05.253052+0000","last_fullsized":"2024-04-24T05:46:05.253052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:13.907547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.247497+0000","last_change":"2024-04-24T05:46:04.274319+0000","last_active":"2024-04-24T05:46:05.247497+0000","last_peered":"2024-04-24T05:46:05.247497+0000","last_clean":"2024-04-24T05:46:05.247497+0000","last_became_active":"2024-04-24T05:46:04.273687+0000","last_became_peered":"2024-04-24T05:46:04.273687+0000","last_unstale":"2024-04-24T05:46:05.247497+0000","last_undegraded":"2024-04-24T05:46:05.247497+0000","last_fullsized":"2024-04-24T05:46:05.247497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:46.740551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058089+0000","last_change":"2024-04-24T05:46:00.222735+0000","last_active":"2024-04-24T05:46:06.058089+0000","last_peered":"2024-04-24T05:46:06.058089+0000","last_clean":"2024-04-24T05:46:06.058089+0000","last_became_active":"2024-04-24T05:46:00.222428+0000","last_became_peered":"2024-04-24T05:46:00.222428+0000","last_unstale":"2024-04-24T05:46:06.058089+0000","last_undegraded":"2024-04-24T05:46:06.058089+0000","last_fullsized":"2024-04-24T05:46:06.058089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557728+0000","last_change":"2024-04-24T05:45:59.220847+0000","last_active":"2024-04-24T05:46:05.557728+0000","last_peered":"2024-04-24T05:46:05.557728+0000","last_clean":"2024-04-24T05:46:05.557728+0000","last_became_active":"2024-04-24T05:45:59.220187+0000","last_became_peered":"2024-04-24T05:45:59.220187+0000","last_unstale":"2024-04-24T05:46:05.557728+0000","last_undegraded":"2024-04-24T05:46:05.557728+0000","last_fullsized":"2024-04-24T05:46:05.557728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:58:40.227841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.249739+0000","last_change":"2024-04-24T05:46:04.271803+0000","last_active":"2024-04-24T05:46:05.249739+0000","last_peered":"2024-04-24T05:46:05.249739+0000","last_clean":"2024-04-24T05:46:05.249739+0000","last_became_active":"2024-04-24T05:46:04.270863+0000","last_became_peered":"2024-04-24T05:46:04.270863+0000","last_unstale":"2024-04-24T05:46:05.249739+0000","last_undegraded":"2024-04-24T05:46:05.249739+0000","last_fullsized":"2024-04-24T05:46:05.249739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:34.349539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061349+0000","last_change":"2024-04-24T05:46:00.216572+0000","last_active":"2024-04-24T05:46:06.061349+0000","last_peered":"2024-04-24T05:46:06.061349+0000","last_clean":"2024-04-24T05:46:06.061349+0000","last_became_active":"2024-04-24T05:46:00.216226+0000","last_became_peered":"2024-04-24T05:46:00.216226+0000","last_unstale":"2024-04-24T05:46:06.061349+0000","last_undegraded":"2024-04-24T05:46:06.061349+0000","last_fullsized":"2024-04-24T05:46:06.061349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607418+0000","last_change":"2024-04-24T05:45:59.201895+0000","last_active":"2024-04-24T05:46:05.607418+0000","last_peered":"2024-04-24T05:46:05.607418+0000","last_clean":"2024-04-24T05:46:05.607418+0000","last_became_active":"2024-04-24T05:45:59.201064+0000","last_became_peered":"2024-04-24T05:45:59.201064+0000","last_unstale":"2024-04-24T05:46:05.607418+0000","last_undegraded":"2024-04-24T05:46:05.607418+0000","last_fullsized":"2024-04-24T05:46:05.607418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248655+0000","last_change":"2024-04-24T05:46:04.278089+0000","last_active":"2024-04-24T05:46:05.248655+0000","last_peered":"2024-04-24T05:46:05.248655+0000","last_clean":"2024-04-24T05:46:05.248655+0000","last_became_active":"2024-04-24T05:46:04.277685+0000","last_became_peered":"2024-04-24T05:46:04.277685+0000","last_unstale":"2024-04-24T05:46:05.248655+0000","last_undegraded":"2024-04-24T05:46:05.248655+0000","last_fullsized":"2024-04-24T05:46:05.248655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:05:10.339090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059639+0000","last_change":"2024-04-24T05:46:00.210339+0000","last_active":"2024-04-24T05:46:06.059639+0000","last_peered":"2024-04-24T05:46:06.059639+0000","last_clean":"2024-04-24T05:46:06.059639+0000","last_became_active":"2024-04-24T05:46:00.209637+0000","last_became_peered":"2024-04-24T05:46:00.209637+0000","last_unstale":"2024-04-24T05:46:06.059639+0000","last_undegraded":"2024-04-24T05:46:06.059639+0000","last_fullsized":"2024-04-24T05:46:06.059639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606639+0000","last_change":"2024-04-24T05:45:59.204528+0000","last_active":"2024-04-24T05:46:05.606639+0000","last_peered":"2024-04-24T05:46:05.606639+0000","last_clean":"2024-04-24T05:46:05.606639+0000","last_became_active":"2024-04-24T05:45:59.204200+0000","last_became_peered":"2024-04-24T05:45:59.204200+0000","last_unstale":"2024-04-24T05:46:05.606639+0000","last_undegraded":"2024-04-24T05:46:05.606639+0000","last_fullsized":"2024-04-24T05:46:05.606639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:26.453566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608562+0000","last_change":"2024-04-24T05:46:04.521949+0000","last_active":"2024-04-24T05:46:05.608562+0000","last_peered":"2024-04-24T05:46:05.608562+0000","last_clean":"2024-04-24T05:46:05.608562+0000","last_became_active":"2024-04-24T05:46:04.521423+0000","last_became_peered":"2024-04-24T05:46:04.521423+0000","last_unstale":"2024-04-24T05:46:05.608562+0000","last_undegraded":"2024-04-24T05:46:05.608562+0000","last_fullsized":"2024-04-24T05:46:05.608562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:44.521141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248230+0000","last_change":"2024-04-24T05:46:00.209888+0000","last_active":"2024-04-24T05:46:05.248230+0000","last_peered":"2024-04-24T05:46:05.248230+0000","last_clean":"2024-04-24T05:46:05.248230+0000","last_became_active":"2024-04-24T05:46:00.209568+0000","last_became_peered":"2024-04-24T05:46:00.209568+0000","last_unstale":"2024-04-24T05:46:05.248230+0000","last_undegraded":"2024-04-24T05:46:05.248230+0000","last_fullsized":"2024-04-24T05:46:05.248230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:21.844840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557732+0000","last_change":"2024-04-24T05:45:59.218584+0000","last_active":"2024-04-24T05:46:05.557732+0000","last_peered":"2024-04-24T05:46:05.557732+0000","last_clean":"2024-04-24T05:46:05.557732+0000","last_became_active":"2024-04-24T05:45:59.218316+0000","last_became_peered":"2024-04-24T05:45:59.218316+0000","last_unstale":"2024-04-24T05:46:05.557732+0000","last_undegraded":"2024-04-24T05:46:05.557732+0000","last_fullsized":"2024-04-24T05:46:05.557732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.249754+0000","last_change":"2024-04-24T05:46:04.281339+0000","last_active":"2024-04-24T05:46:05.249754+0000","last_peered":"2024-04-24T05:46:05.249754+0000","last_clean":"2024-04-24T05:46:05.249754+0000","last_became_active":"2024-04-24T05:46:04.280972+0000","last_became_peered":"2024-04-24T05:46:04.280972+0000","last_unstale":"2024-04-24T05:46:05.249754+0000","last_undegraded":"2024-04-24T05:46:05.249754+0000","last_fullsized":"2024-04-24T05:46:05.249754+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:17.960048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.064250+0000","last_change":"2024-04-24T05:46:00.210893+0000","last_active":"2024-04-24T05:46:06.064250+0000","last_peered":"2024-04-24T05:46:06.064250+0000","last_clean":"2024-04-24T05:46:06.064250+0000","last_became_active":"2024-04-24T05:46:00.210566+0000","last_became_peered":"2024-04-24T05:46:00.210566+0000","last_unstale":"2024-04-24T05:46:06.064250+0000","last_undegraded":"2024-04-24T05:46:06.064250+0000","last_fullsized":"2024-04-24T05:46:06.064250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059489+0000","last_change":"2024-04-24T05:45:59.213341+0000","last_active":"2024-04-24T05:46:06.059489+0000","last_peered":"2024-04-24T05:46:06.059489+0000","last_clean":"2024-04-24T05:46:06.059489+0000","last_became_active":"2024-04-24T05:45:59.212756+0000","last_became_peered":"2024-04-24T05:45:59.212756+0000","last_unstale":"2024-04-24T05:46:06.059489+0000","last_undegraded":"2024-04-24T05:46:06.059489+0000","last_fullsized":"2024-04-24T05:46:06.059489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:34.072620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.247506+0000","last_change":"2024-04-24T05:46:04.524925+0000","last_active":"2024-04-24T05:46:05.247506+0000","last_peered":"2024-04-24T05:46:05.247506+0000","last_clean":"2024-04-24T05:46:05.247506+0000","last_became_active":"2024-04-24T05:46:04.523828+0000","last_became_peered":"2024-04-24T05:46:04.523828+0000","last_unstale":"2024-04-24T05:46:05.247506+0000","last_undegraded":"2024-04-24T05:46:05.247506+0000","last_fullsized":"2024-04-24T05:46:05.247506+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:45.904568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063326+0000","last_change":"2024-04-24T05:46:00.227429+0000","last_active":"2024-04-24T05:46:06.063326+0000","last_peered":"2024-04-24T05:46:06.063326+0000","last_clean":"2024-04-24T05:46:06.063326+0000","last_became_active":"2024-04-24T05:46:00.226909+0000","last_became_peered":"2024-04-24T05:46:00.226909+0000","last_unstale":"2024-04-24T05:46:06.063326+0000","last_undegraded":"2024-04-24T05:46:06.063326+0000","last_fullsized":"2024-04-24T05:46:06.063326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:45.039541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608337+0000","last_change":"2024-04-24T05:45:59.203340+0000","last_active":"2024-04-24T05:46:05.608337+0000","last_peered":"2024-04-24T05:46:05.608337+0000","last_clean":"2024-04-24T05:46:05.608337+0000","last_became_active":"2024-04-24T05:45:59.203004+0000","last_became_peered":"2024-04-24T05:45:59.203004+0000","last_unstale":"2024-04-24T05:46:05.608337+0000","last_undegraded":"2024-04-24T05:46:05.608337+0000","last_fullsized":"2024-04-24T05:46:05.608337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:34.072620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059109+0000","last_change":"2024-04-24T05:46:04.528328+0000","last_active":"2024-04-24T05:46:06.059109+0000","last_peered":"2024-04-24T05:46:06.059109+0000","last_clean":"2024-04-24T05:46:06.059109+0000","last_became_active":"2024-04-24T05:46:04.527760+0000","last_became_peered":"2024-04-24T05:46:04.527760+0000","last_unstale":"2024-04-24T05:46:06.059109+0000","last_undegraded":"2024-04-24T05:46:06.059109+0000","last_fullsized":"2024-04-24T05:46:06.059109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:16:08.677684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607804+0000","last_change":"2024-04-24T05:46:00.215704+0000","last_active":"2024-04-24T05:46:05.607804+0000","last_peered":"2024-04-24T05:46:05.607804+0000","last_clean":"2024-04-24T05:46:05.607804+0000","last_became_active":"2024-04-24T05:46:00.214751+0000","last_became_peered":"2024-04-24T05:46:00.214751+0000","last_unstale":"2024-04-24T05:46:05.607804+0000","last_undegraded":"2024-04-24T05:46:05.607804+0000","last_fullsized":"2024-04-24T05:46:05.607804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558293+0000","last_change":"2024-04-24T05:45:59.219396+0000","last_active":"2024-04-24T05:46:05.558293+0000","last_peered":"2024-04-24T05:46:05.558293+0000","last_clean":"2024-04-24T05:46:05.558293+0000","last_became_active":"2024-04-24T05:45:59.217765+0000","last_became_peered":"2024-04-24T05:45:59.217765+0000","last_unstale":"2024-04-24T05:46:05.558293+0000","last_undegraded":"2024-04-24T05:46:05.558293+0000","last_fullsized":"2024-04-24T05:46:05.558293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:21:56.958729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061548+0000","last_change":"2024-04-24T05:46:04.518477+0000","last_active":"2024-04-24T05:46:06.061548+0000","last_peered":"2024-04-24T05:46:06.061548+0000","last_clean":"2024-04-24T05:46:06.061548+0000","last_became_active":"2024-04-24T05:46:04.515078+0000","last_became_peered":"2024-04-24T05:46:04.515078+0000","last_unstale":"2024-04-24T05:46:06.061548+0000","last_undegraded":"2024-04-24T05:46:06.061548+0000","last_fullsized":"2024-04-24T05:46:06.061548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:31.102602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253112+0000","last_change":"2024-04-24T05:46:00.212998+0000","last_active":"2024-04-24T05:46:05.253112+0000","last_peered":"2024-04-24T05:46:05.253112+0000","last_clean":"2024-04-24T05:46:05.253112+0000","last_became_active":"2024-04-24T05:46:00.212446+0000","last_became_peered":"2024-04-24T05:46:00.212446+0000","last_unstale":"2024-04-24T05:46:05.253112+0000","last_undegraded":"2024-04-24T05:46:05.253112+0000","last_fullsized":"2024-04-24T05:46:05.253112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:27.050101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607823+0000","last_change":"2024-04-24T05:45:59.197842+0000","last_active":"2024-04-24T05:46:05.607823+0000","last_peered":"2024-04-24T05:46:05.607823+0000","last_clean":"2024-04-24T05:46:05.607823+0000","last_became_active":"2024-04-24T05:45:59.197510+0000","last_became_peered":"2024-04-24T05:45:59.197510+0000","last_unstale":"2024-04-24T05:46:05.607823+0000","last_undegraded":"2024-04-24T05:46:05.607823+0000","last_fullsized":"2024-04-24T05:46:05.607823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606699+0000","last_change":"2024-04-24T05:46:04.270957+0000","last_active":"2024-04-24T05:46:05.606699+0000","last_peered":"2024-04-24T05:46:05.606699+0000","last_clean":"2024-04-24T05:46:05.606699+0000","last_became_active":"2024-04-24T05:46:04.268656+0000","last_became_peered":"2024-04-24T05:46:04.268656+0000","last_unstale":"2024-04-24T05:46:05.606699+0000","last_undegraded":"2024-04-24T05:46:05.606699+0000","last_fullsized":"2024-04-24T05:46:05.606699+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:31.102602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248701+0000","last_change":"2024-04-24T05:46:00.205128+0000","last_active":"2024-04-24T05:46:05.248701+0000","last_peered":"2024-04-24T05:46:05.248701+0000","last_clean":"2024-04-24T05:46:05.248701+0000","last_became_active":"2024-04-24T05:46:00.202044+0000","last_became_peered":"2024-04-24T05:46:00.202044+0000","last_unstale":"2024-04-24T05:46:05.248701+0000","last_undegraded":"2024-04-24T05:46:05.248701+0000","last_fullsized":"2024-04-24T05:46:05.248701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:18.772951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252226+0000","last_change":"2024-04-24T05:45:59.199320+0000","last_active":"2024-04-24T05:46:05.252226+0000","last_peered":"2024-04-24T05:46:05.252226+0000","last_clean":"2024-04-24T05:46:05.252226+0000","last_became_active":"2024-04-24T05:45:59.198382+0000","last_became_peered":"2024-04-24T05:45:59.198382+0000","last_unstale":"2024-04-24T05:46:05.252226+0000","last_undegraded":"2024-04-24T05:46:05.252226+0000","last_fullsized":"2024-04-24T05:46:05.252226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607713+0000","last_change":"2024-04-24T05:46:04.273947+0000","last_active":"2024-04-24T05:46:05.607713+0000","last_peered":"2024-04-24T05:46:05.607713+0000","last_clean":"2024-04-24T05:46:05.607713+0000","last_became_active":"2024-04-24T05:46:04.271583+0000","last_became_peered":"2024-04-24T05:46:04.271583+0000","last_unstale":"2024-04-24T05:46:05.607713+0000","last_undegraded":"2024-04-24T05:46:05.607713+0000","last_fullsized":"2024-04-24T05:46:05.607713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:45.991179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606401+0000","last_change":"2024-04-24T05:46:00.211997+0000","last_active":"2024-04-24T05:46:05.606401+0000","last_peered":"2024-04-24T05:46:05.606401+0000","last_clean":"2024-04-24T05:46:05.606401+0000","last_became_active":"2024-04-24T05:46:00.209526+0000","last_became_peered":"2024-04-24T05:46:00.209526+0000","last_unstale":"2024-04-24T05:46:05.606401+0000","last_undegraded":"2024-04-24T05:46:05.606401+0000","last_fullsized":"2024-04-24T05:46:05.606401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:27.080921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058600+0000","last_change":"2024-04-24T05:45:59.198738+0000","last_active":"2024-04-24T05:46:06.058600+0000","last_peered":"2024-04-24T05:46:06.058600+0000","last_clean":"2024-04-24T05:46:06.058600+0000","last_became_active":"2024-04-24T05:45:59.198458+0000","last_became_peered":"2024-04-24T05:45:59.198458+0000","last_unstale":"2024-04-24T05:46:06.058600+0000","last_undegraded":"2024-04-24T05:46:06.058600+0000","last_fullsized":"2024-04-24T05:46:06.058600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:11.561126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608670+0000","last_change":"2024-04-24T05:46:04.266757+0000","last_active":"2024-04-24T05:46:05.608670+0000","last_peered":"2024-04-24T05:46:05.608670+0000","last_clean":"2024-04-24T05:46:05.608670+0000","last_became_active":"2024-04-24T05:46:04.266279+0000","last_became_peered":"2024-04-24T05:46:04.266279+0000","last_unstale":"2024-04-24T05:46:05.608670+0000","last_undegraded":"2024-04-24T05:46:05.608670+0000","last_fullsized":"2024-04-24T05:46:05.608670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:06:00.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058554+0000","last_change":"2024-04-24T05:46:00.216303+0000","last_active":"2024-04-24T05:46:06.058554+0000","last_peered":"2024-04-24T05:46:06.058554+0000","last_clean":"2024-04-24T05:46:06.058554+0000","last_became_active":"2024-04-24T05:46:00.215719+0000","last_became_peered":"2024-04-24T05:46:00.215719+0000","last_unstale":"2024-04-24T05:46:06.058554+0000","last_undegraded":"2024-04-24T05:46:06.058554+0000","last_fullsized":"2024-04-24T05:46:06.058554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:47:58.509148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558034+0000","last_change":"2024-04-24T05:45:59.218643+0000","last_active":"2024-04-24T05:46:05.558034+0000","last_peered":"2024-04-24T05:46:05.558034+0000","last_clean":"2024-04-24T05:46:05.558034+0000","last_became_active":"2024-04-24T05:45:59.218346+0000","last_became_peered":"2024-04-24T05:45:59.218346+0000","last_unstale":"2024-04-24T05:46:05.558034+0000","last_undegraded":"2024-04-24T05:46:05.558034+0000","last_fullsized":"2024-04-24T05:46:05.558034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:26.453566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063814+0000","last_change":"2024-04-24T05:46:04.527188+0000","last_active":"2024-04-24T05:46:06.063814+0000","last_peered":"2024-04-24T05:46:06.063814+0000","last_clean":"2024-04-24T05:46:06.063814+0000","last_became_active":"2024-04-24T05:46:04.526002+0000","last_became_peered":"2024-04-24T05:46:04.526002+0000","last_unstale":"2024-04-24T05:46:06.063814+0000","last_undegraded":"2024-04-24T05:46:06.063814+0000","last_fullsized":"2024-04-24T05:46:06.063814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:06:00.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060620+0000","last_change":"2024-04-24T05:46:00.224034+0000","last_active":"2024-04-24T05:46:06.060620+0000","last_peered":"2024-04-24T05:46:06.060620+0000","last_clean":"2024-04-24T05:46:06.060620+0000","last_became_active":"2024-04-24T05:46:00.223714+0000","last_became_peered":"2024-04-24T05:46:00.223714+0000","last_unstale":"2024-04-24T05:46:06.060620+0000","last_undegraded":"2024-04-24T05:46:06.060620+0000","last_fullsized":"2024-04-24T05:46:06.060620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:25.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252259+0000","last_change":"2024-04-24T05:45:59.199095+0000","last_active":"2024-04-24T05:46:05.252259+0000","last_peered":"2024-04-24T05:46:05.252259+0000","last_clean":"2024-04-24T05:46:05.252259+0000","last_became_active":"2024-04-24T05:45:59.198772+0000","last_became_peered":"2024-04-24T05:45:59.198772+0000","last_unstale":"2024-04-24T05:46:05.252259+0000","last_undegraded":"2024-04-24T05:46:05.252259+0000","last_fullsized":"2024-04-24T05:46:05.252259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:11.561126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558345+0000","last_change":"2024-04-24T05:45:51.019903+0000","last_active":"2024-04-24T05:46:05.558345+0000","last_peered":"2024-04-24T05:46:05.558345+0000","last_clean":"2024-04-24T05:46:05.558345+0000","last_became_active":"2024-04-24T05:45:51.017323+0000","last_became_peered":"2024-04-24T05:45:51.017323+0000","last_unstale":"2024-04-24T05:46:05.558345+0000","last_undegraded":"2024-04-24T05:46:05.558345+0000","last_fullsized":"2024-04-24T05:46:05.558345+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:48.855871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:48.855871+0000","last_clean_scrub_stamp":"2024-04-24T05:45:48.855871+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:10:41.045203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058781+0000","last_change":"2024-04-24T05:46:04.525962+0000","last_active":"2024-04-24T05:46:06.058781+0000","last_peered":"2024-04-24T05:46:06.058781+0000","last_clean":"2024-04-24T05:46:06.058781+0000","last_became_active":"2024-04-24T05:46:04.524947+0000","last_became_peered":"2024-04-24T05:46:04.524947+0000","last_unstale":"2024-04-24T05:46:06.058781+0000","last_undegraded":"2024-04-24T05:46:06.058781+0000","last_fullsized":"2024-04-24T05:46:06.058781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:17.960048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557690+0000","last_change":"2024-04-24T05:46:00.226016+0000","last_active":"2024-04-24T05:46:05.557690+0000","last_peered":"2024-04-24T05:46:05.557690+0000","last_clean":"2024-04-24T05:46:05.557690+0000","last_became_active":"2024-04-24T05:46:00.225693+0000","last_became_peered":"2024-04-24T05:46:00.225693+0000","last_unstale":"2024-04-24T05:46:05.557690+0000","last_undegraded":"2024-04-24T05:46:05.557690+0000","last_fullsized":"2024-04-24T05:46:05.557690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:13.907547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059847+0000","last_change":"2024-04-24T05:45:59.203021+0000","last_active":"2024-04-24T05:46:06.059847+0000","last_peered":"2024-04-24T05:46:06.059847+0000","last_clean":"2024-04-24T05:46:06.059847+0000","last_became_active":"2024-04-24T05:45:59.202733+0000","last_became_peered":"2024-04-24T05:45:59.202733+0000","last_unstale":"2024-04-24T05:46:06.059847+0000","last_undegraded":"2024-04-24T05:46:06.059847+0000","last_fullsized":"2024-04-24T05:46:06.059847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058297+0000","last_change":"2024-04-24T05:46:04.521125+0000","last_active":"2024-04-24T05:46:06.058297+0000","last_peered":"2024-04-24T05:46:06.058297+0000","last_clean":"2024-04-24T05:46:06.058297+0000","last_became_active":"2024-04-24T05:46:04.519878+0000","last_became_peered":"2024-04-24T05:46:04.519878+0000","last_unstale":"2024-04-24T05:46:06.058297+0000","last_undegraded":"2024-04-24T05:46:06.058297+0000","last_fullsized":"2024-04-24T05:46:06.058297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:34.349539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252701+0000","last_change":"2024-04-24T05:46:00.214353+0000","last_active":"2024-04-24T05:46:05.252701+0000","last_peered":"2024-04-24T05:46:05.252701+0000","last_clean":"2024-04-24T05:46:05.252701+0000","last_became_active":"2024-04-24T05:46:00.213791+0000","last_became_peered":"2024-04-24T05:46:00.213791+0000","last_unstale":"2024-04-24T05:46:05.252701+0000","last_undegraded":"2024-04-24T05:46:05.252701+0000","last_fullsized":"2024-04-24T05:46:05.252701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:30.297037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608191+0000","last_change":"2024-04-24T05:45:59.216779+0000","last_active":"2024-04-24T05:46:05.608191+0000","last_peered":"2024-04-24T05:46:05.608191+0000","last_clean":"2024-04-24T05:46:05.608191+0000","last_became_active":"2024-04-24T05:45:59.216507+0000","last_became_peered":"2024-04-24T05:45:59.216507+0000","last_unstale":"2024-04-24T05:46:05.608191+0000","last_undegraded":"2024-04-24T05:46:05.608191+0000","last_fullsized":"2024-04-24T05:46:05.608191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062332+0000","last_change":"2024-04-24T05:46:04.268777+0000","last_active":"2024-04-24T05:46:06.062332+0000","last_peered":"2024-04-24T05:46:06.062332+0000","last_clean":"2024-04-24T05:46:06.062332+0000","last_became_active":"2024-04-24T05:46:04.268416+0000","last_became_peered":"2024-04-24T05:46:04.268416+0000","last_unstale":"2024-04-24T05:46:06.062332+0000","last_undegraded":"2024-04-24T05:46:06.062332+0000","last_fullsized":"2024-04-24T05:46:06.062332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:13.252851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558634+0000","last_change":"2024-04-24T05:46:00.227851+0000","last_active":"2024-04-24T05:46:05.558634+0000","last_peered":"2024-04-24T05:46:05.558634+0000","last_clean":"2024-04-24T05:46:05.558634+0000","last_became_active":"2024-04-24T05:46:00.220088+0000","last_became_peered":"2024-04-24T05:46:00.220088+0000","last_unstale":"2024-04-24T05:46:05.558634+0000","last_undegraded":"2024-04-24T05:46:05.558634+0000","last_fullsized":"2024-04-24T05:46:05.558634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:40.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059276+0000","last_change":"2024-04-24T05:45:59.196319+0000","last_active":"2024-04-24T05:46:06.059276+0000","last_peered":"2024-04-24T05:46:06.059276+0000","last_clean":"2024-04-24T05:46:06.059276+0000","last_became_active":"2024-04-24T05:45:59.196029+0000","last_became_peered":"2024-04-24T05:45:59.196029+0000","last_unstale":"2024-04-24T05:46:06.059276+0000","last_undegraded":"2024-04-24T05:46:06.059276+0000","last_fullsized":"2024-04-24T05:46:06.059276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.250996+0000","last_change":"2024-04-24T05:46:04.527040+0000","last_active":"2024-04-24T05:46:05.250996+0000","last_peered":"2024-04-24T05:46:05.250996+0000","last_clean":"2024-04-24T05:46:05.250996+0000","last_became_active":"2024-04-24T05:46:04.522889+0000","last_became_peered":"2024-04-24T05:46:04.522889+0000","last_unstale":"2024-04-24T05:46:05.250996+0000","last_undegraded":"2024-04-24T05:46:05.250996+0000","last_fullsized":"2024-04-24T05:46:05.250996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:06:00.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558335+0000","last_change":"2024-04-24T05:46:00.225225+0000","last_active":"2024-04-24T05:46:05.558335+0000","last_peered":"2024-04-24T05:46:05.558335+0000","last_clean":"2024-04-24T05:46:05.558335+0000","last_became_active":"2024-04-24T05:46:00.224932+0000","last_became_peered":"2024-04-24T05:46:00.224932+0000","last_unstale":"2024-04-24T05:46:05.558335+0000","last_undegraded":"2024-04-24T05:46:05.558335+0000","last_fullsized":"2024-04-24T05:46:05.558335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:41.938677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248435+0000","last_change":"2024-04-24T05:45:59.214909+0000","last_active":"2024-04-24T05:46:05.248435+0000","last_peered":"2024-04-24T05:46:05.248435+0000","last_clean":"2024-04-24T05:46:05.248435+0000","last_became_active":"2024-04-24T05:45:59.213540+0000","last_became_peered":"2024-04-24T05:45:59.213540+0000","last_unstale":"2024-04-24T05:46:05.248435+0000","last_undegraded":"2024-04-24T05:46:05.248435+0000","last_fullsized":"2024-04-24T05:46:05.248435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:00.340470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.246897+0000","last_change":"2024-04-24T05:46:04.275887+0000","last_active":"2024-04-24T05:46:05.246897+0000","last_peered":"2024-04-24T05:46:05.246897+0000","last_clean":"2024-04-24T05:46:05.246897+0000","last_became_active":"2024-04-24T05:46:04.275455+0000","last_became_peered":"2024-04-24T05:46:04.275455+0000","last_unstale":"2024-04-24T05:46:05.246897+0000","last_undegraded":"2024-04-24T05:46:05.246897+0000","last_fullsized":"2024-04-24T05:46:05.246897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:29.075099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557429+0000","last_change":"2024-04-24T05:46:00.217599+0000","last_active":"2024-04-24T05:46:05.557429+0000","last_peered":"2024-04-24T05:46:05.557429+0000","last_clean":"2024-04-24T05:46:05.557429+0000","last_became_active":"2024-04-24T05:46:00.217383+0000","last_became_peered":"2024-04-24T05:46:00.217383+0000","last_unstale":"2024-04-24T05:46:05.557429+0000","last_undegraded":"2024-04-24T05:46:05.557429+0000","last_fullsized":"2024-04-24T05:46:05.557429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:38.107685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.057867+0000","last_change":"2024-04-24T05:45:59.196826+0000","last_active":"2024-04-24T05:46:06.057867+0000","last_peered":"2024-04-24T05:46:06.057867+0000","last_clean":"2024-04-24T05:46:06.057867+0000","last_became_active":"2024-04-24T05:45:59.195023+0000","last_became_peered":"2024-04-24T05:45:59.195023+0000","last_unstale":"2024-04-24T05:46:06.057867+0000","last_undegraded":"2024-04-24T05:46:06.057867+0000","last_fullsized":"2024-04-24T05:46:06.057867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063777+0000","last_change":"2024-04-24T05:46:04.265949+0000","last_active":"2024-04-24T05:46:06.063777+0000","last_peered":"2024-04-24T05:46:06.063777+0000","last_clean":"2024-04-24T05:46:06.063777+0000","last_became_active":"2024-04-24T05:46:04.265546+0000","last_became_peered":"2024-04-24T05:46:04.265546+0000","last_unstale":"2024-04-24T05:46:06.063777+0000","last_undegraded":"2024-04-24T05:46:06.063777+0000","last_fullsized":"2024-04-24T05:46:06.063777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:44.521141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058813+0000","last_change":"2024-04-24T05:46:00.221203+0000","last_active":"2024-04-24T05:46:06.058813+0000","last_peered":"2024-04-24T05:46:06.058813+0000","last_clean":"2024-04-24T05:46:06.058813+0000","last_became_active":"2024-04-24T05:46:00.220709+0000","last_became_peered":"2024-04-24T05:46:00.220709+0000","last_unstale":"2024-04-24T05:46:06.058813+0000","last_undegraded":"2024-04-24T05:46:06.058813+0000","last_fullsized":"2024-04-24T05:46:06.058813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059170+0000","last_change":"2024-04-24T05:45:59.217694+0000","last_active":"2024-04-24T05:46:06.059170+0000","last_peered":"2024-04-24T05:46:06.059170+0000","last_clean":"2024-04-24T05:46:06.059170+0000","last_became_active":"2024-04-24T05:45:59.217385+0000","last_became_peered":"2024-04-24T05:45:59.217385+0000","last_unstale":"2024-04-24T05:46:06.059170+0000","last_undegraded":"2024-04-24T05:46:06.059170+0000","last_fullsized":"2024-04-24T05:46:06.059170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063100+0000","last_change":"2024-04-24T05:46:04.529839+0000","last_active":"2024-04-24T05:46:06.063100+0000","last_peered":"2024-04-24T05:46:06.063100+0000","last_clean":"2024-04-24T05:46:06.063100+0000","last_became_active":"2024-04-24T05:46:04.524578+0000","last_became_peered":"2024-04-24T05:46:04.524578+0000","last_unstale":"2024-04-24T05:46:06.063100+0000","last_undegraded":"2024-04-24T05:46:06.063100+0000","last_fullsized":"2024-04-24T05:46:06.063100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:26.692990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557529+0000","last_change":"2024-04-24T05:46:00.226661+0000","last_active":"2024-04-24T05:46:05.557529+0000","last_peered":"2024-04-24T05:46:05.557529+0000","last_clean":"2024-04-24T05:46:05.557529+0000","last_became_active":"2024-04-24T05:46:00.226357+0000","last_became_peered":"2024-04-24T05:46:00.226357+0000","last_unstale":"2024-04-24T05:46:05.557529+0000","last_undegraded":"2024-04-24T05:46:05.557529+0000","last_fullsized":"2024-04-24T05:46:05.557529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:05:06.286589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252051+0000","last_change":"2024-04-24T05:45:59.206276+0000","last_active":"2024-04-24T05:46:05.252051+0000","last_peered":"2024-04-24T05:46:05.252051+0000","last_clean":"2024-04-24T05:46:05.252051+0000","last_became_active":"2024-04-24T05:45:59.205926+0000","last_became_peered":"2024-04-24T05:45:59.205926+0000","last_unstale":"2024-04-24T05:46:05.252051+0000","last_undegraded":"2024-04-24T05:46:05.252051+0000","last_fullsized":"2024-04-24T05:46:05.252051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062045+0000","last_change":"2024-04-24T05:46:04.515144+0000","last_active":"2024-04-24T05:46:06.062045+0000","last_peered":"2024-04-24T05:46:06.062045+0000","last_clean":"2024-04-24T05:46:06.062045+0000","last_became_active":"2024-04-24T05:46:04.514326+0000","last_became_peered":"2024-04-24T05:46:04.514326+0000","last_unstale":"2024-04-24T05:46:06.062045+0000","last_undegraded":"2024-04-24T05:46:06.062045+0000","last_fullsized":"2024-04-24T05:46:06.062045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:45.847497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060827+0000","last_change":"2024-04-24T05:46:00.214864+0000","last_active":"2024-04-24T05:46:06.060827+0000","last_peered":"2024-04-24T05:46:06.060827+0000","last_clean":"2024-04-24T05:46:06.060827+0000","last_became_active":"2024-04-24T05:46:00.214051+0000","last_became_peered":"2024-04-24T05:46:00.214051+0000","last_unstale":"2024-04-24T05:46:06.060827+0000","last_undegraded":"2024-04-24T05:46:06.060827+0000","last_fullsized":"2024-04-24T05:46:06.060827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:41:43.988057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062825+0000","last_change":"2024-04-24T05:45:59.213057+0000","last_active":"2024-04-24T05:46:06.062825+0000","last_peered":"2024-04-24T05:46:06.062825+0000","last_clean":"2024-04-24T05:46:06.062825+0000","last_became_active":"2024-04-24T05:45:59.212716+0000","last_became_peered":"2024-04-24T05:45:59.212716+0000","last_unstale":"2024-04-24T05:46:06.062825+0000","last_undegraded":"2024-04-24T05:46:06.062825+0000","last_fullsized":"2024-04-24T05:46:06.062825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:53.435411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252861+0000","last_change":"2024-04-24T05:46:04.516510+0000","last_active":"2024-04-24T05:46:05.252861+0000","last_peered":"2024-04-24T05:46:05.252861+0000","last_clean":"2024-04-24T05:46:05.252861+0000","last_became_active":"2024-04-24T05:46:04.515458+0000","last_became_peered":"2024-04-24T05:46:04.515458+0000","last_unstale":"2024-04-24T05:46:05.252861+0000","last_undegraded":"2024-04-24T05:46:05.252861+0000","last_fullsized":"2024-04-24T05:46:05.252861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:16:08.677684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557591+0000","last_change":"2024-04-24T05:46:00.215087+0000","last_active":"2024-04-24T05:46:05.557591+0000","last_peered":"2024-04-24T05:46:05.557591+0000","last_clean":"2024-04-24T05:46:05.557591+0000","last_became_active":"2024-04-24T05:46:00.213783+0000","last_became_peered":"2024-04-24T05:46:00.213783+0000","last_unstale":"2024-04-24T05:46:05.557591+0000","last_undegraded":"2024-04-24T05:46:05.557591+0000","last_fullsized":"2024-04-24T05:46:05.557591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:30.297037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:17.734750+0000","last_change":"2024-04-24T05:45:59.211881+0000","last_active":"2024-04-24T05:46:17.734750+0000","last_peered":"2024-04-24T05:46:17.734750+0000","last_clean":"2024-04-24T05:46:17.734750+0000","last_became_active":"2024-04-24T05:45:59.211566+0000","last_became_peered":"2024-04-24T05:45:59.211566+0000","last_unstale":"2024-04-24T05:46:17.734750+0000","last_undegraded":"2024-04-24T05:46:17.734750+0000","last_fullsized":"2024-04-24T05:46:17.734750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607263+0000","last_change":"2024-04-24T05:46:04.264433+0000","last_active":"2024-04-24T05:46:05.607263+0000","last_peered":"2024-04-24T05:46:05.607263+0000","last_clean":"2024-04-24T05:46:05.607263+0000","last_became_active":"2024-04-24T05:46:04.263926+0000","last_became_peered":"2024-04-24T05:46:04.263926+0000","last_unstale":"2024-04-24T05:46:05.607263+0000","last_undegraded":"2024-04-24T05:46:05.607263+0000","last_fullsized":"2024-04-24T05:46:05.607263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:13.252851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248914+0000","last_change":"2024-04-24T05:46:00.218334+0000","last_active":"2024-04-24T05:46:05.248914+0000","last_peered":"2024-04-24T05:46:05.248914+0000","last_clean":"2024-04-24T05:46:05.248914+0000","last_became_active":"2024-04-24T05:46:00.213738+0000","last_became_peered":"2024-04-24T05:46:00.213738+0000","last_unstale":"2024-04-24T05:46:05.248914+0000","last_undegraded":"2024-04-24T05:46:05.248914+0000","last_fullsized":"2024-04-24T05:46:05.248914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:27.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608102+0000","last_change":"2024-04-24T05:45:59.206057+0000","last_active":"2024-04-24T05:46:05.608102+0000","last_peered":"2024-04-24T05:46:05.608102+0000","last_clean":"2024-04-24T05:46:05.608102+0000","last_became_active":"2024-04-24T05:45:59.205560+0000","last_became_peered":"2024-04-24T05:45:59.205560+0000","last_unstale":"2024-04-24T05:46:05.608102+0000","last_undegraded":"2024-04-24T05:46:05.608102+0000","last_fullsized":"2024-04-24T05:46:05.608102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248223+0000","last_change":"2024-04-24T05:46:04.274265+0000","last_active":"2024-04-24T05:46:05.248223+0000","last_peered":"2024-04-24T05:46:05.248223+0000","last_clean":"2024-04-24T05:46:05.248223+0000","last_became_active":"2024-04-24T05:46:04.273775+0000","last_became_peered":"2024-04-24T05:46:04.273775+0000","last_unstale":"2024-04-24T05:46:05.248223+0000","last_undegraded":"2024-04-24T05:46:05.248223+0000","last_fullsized":"2024-04-24T05:46:05.248223+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:49.092042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063970+0000","last_change":"2024-04-24T05:46:00.222245+0000","last_active":"2024-04-24T05:46:06.063970+0000","last_peered":"2024-04-24T05:46:06.063970+0000","last_clean":"2024-04-24T05:46:06.063970+0000","last_became_active":"2024-04-24T05:46:00.220540+0000","last_became_peered":"2024-04-24T05:46:00.220540+0000","last_unstale":"2024-04-24T05:46:06.063970+0000","last_undegraded":"2024-04-24T05:46:06.063970+0000","last_fullsized":"2024-04-24T05:46:06.063970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253552+0000","last_change":"2024-04-24T05:45:59.203808+0000","last_active":"2024-04-24T05:46:05.253552+0000","last_peered":"2024-04-24T05:46:05.253552+0000","last_clean":"2024-04-24T05:46:05.253552+0000","last_became_active":"2024-04-24T05:45:59.203023+0000","last_became_peered":"2024-04-24T05:45:59.203023+0000","last_unstale":"2024-04-24T05:46:05.253552+0000","last_undegraded":"2024-04-24T05:46:05.253552+0000","last_fullsized":"2024-04-24T05:46:05.253552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059474+0000","last_change":"2024-04-24T05:46:04.274610+0000","last_active":"2024-04-24T05:46:06.059474+0000","last_peered":"2024-04-24T05:46:06.059474+0000","last_clean":"2024-04-24T05:46:06.059474+0000","last_became_active":"2024-04-24T05:46:04.274246+0000","last_became_peered":"2024-04-24T05:46:04.274246+0000","last_unstale":"2024-04-24T05:46:06.059474+0000","last_undegraded":"2024-04-24T05:46:06.059474+0000","last_fullsized":"2024-04-24T05:46:06.059474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:31:17.536498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253481+0000","last_change":"2024-04-24T05:46:00.207959+0000","last_active":"2024-04-24T05:46:05.253481+0000","last_peered":"2024-04-24T05:46:05.253481+0000","last_clean":"2024-04-24T05:46:05.253481+0000","last_became_active":"2024-04-24T05:46:00.205225+0000","last_became_peered":"2024-04-24T05:46:00.205225+0000","last_unstale":"2024-04-24T05:46:05.253481+0000","last_undegraded":"2024-04-24T05:46:05.253481+0000","last_fullsized":"2024-04-24T05:46:05.253481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:55:08.484471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059232+0000","last_change":"2024-04-24T05:45:59.212223+0000","last_active":"2024-04-24T05:46:06.059232+0000","last_peered":"2024-04-24T05:46:06.059232+0000","last_clean":"2024-04-24T05:46:06.059232+0000","last_became_active":"2024-04-24T05:45:59.211908+0000","last_became_peered":"2024-04-24T05:45:59.211908+0000","last_unstale":"2024-04-24T05:46:06.059232+0000","last_undegraded":"2024-04-24T05:46:06.059232+0000","last_fullsized":"2024-04-24T05:46:06.059232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060243+0000","last_change":"2024-04-24T05:46:04.529148+0000","last_active":"2024-04-24T05:46:06.060243+0000","last_peered":"2024-04-24T05:46:06.060243+0000","last_clean":"2024-04-24T05:46:06.060243+0000","last_became_active":"2024-04-24T05:46:04.524775+0000","last_became_peered":"2024-04-24T05:46:04.524775+0000","last_unstale":"2024-04-24T05:46:06.060243+0000","last_undegraded":"2024-04-24T05:46:06.060243+0000","last_fullsized":"2024-04-24T05:46:06.060243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:58:44.607716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063332+0000","last_change":"2024-04-24T05:46:04.524596+0000","last_active":"2024-04-24T05:46:06.063332+0000","last_peered":"2024-04-24T05:46:06.063332+0000","last_clean":"2024-04-24T05:46:06.063332+0000","last_became_active":"2024-04-24T05:46:04.524013+0000","last_became_peered":"2024-04-24T05:46:04.524013+0000","last_unstale":"2024-04-24T05:46:06.063332+0000","last_undegraded":"2024-04-24T05:46:06.063332+0000","last_fullsized":"2024-04-24T05:46:06.063332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:45.991179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606337+0000","last_change":"2024-04-24T05:46:00.222945+0000","last_active":"2024-04-24T05:46:05.606337+0000","last_peered":"2024-04-24T05:46:05.606337+0000","last_clean":"2024-04-24T05:46:05.606337+0000","last_became_active":"2024-04-24T05:46:00.222698+0000","last_became_peered":"2024-04-24T05:46:00.222698+0000","last_unstale":"2024-04-24T05:46:05.606337+0000","last_undegraded":"2024-04-24T05:46:05.606337+0000","last_fullsized":"2024-04-24T05:46:05.606337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251487+0000","last_change":"2024-04-24T05:45:59.211291+0000","last_active":"2024-04-24T05:46:05.251487+0000","last_peered":"2024-04-24T05:46:05.251487+0000","last_clean":"2024-04-24T05:46:05.251487+0000","last_became_active":"2024-04-24T05:45:59.209448+0000","last_became_peered":"2024-04-24T05:45:59.209448+0000","last_unstale":"2024-04-24T05:46:05.251487+0000","last_undegraded":"2024-04-24T05:46:05.251487+0000","last_fullsized":"2024-04-24T05:46:05.251487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058917+0000","last_change":"2024-04-24T05:46:04.274043+0000","last_active":"2024-04-24T05:46:06.058917+0000","last_peered":"2024-04-24T05:46:06.058917+0000","last_clean":"2024-04-24T05:46:06.058917+0000","last_became_active":"2024-04-24T05:46:04.273504+0000","last_became_peered":"2024-04-24T05:46:04.273504+0000","last_unstale":"2024-04-24T05:46:06.058917+0000","last_undegraded":"2024-04-24T05:46:06.058917+0000","last_fullsized":"2024-04-24T05:46:06.058917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:55:12.536973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059394+0000","last_change":"2024-04-24T05:46:00.216386+0000","last_active":"2024-04-24T05:46:06.059394+0000","last_peered":"2024-04-24T05:46:06.059394+0000","last_clean":"2024-04-24T05:46:06.059394+0000","last_became_active":"2024-04-24T05:46:00.216078+0000","last_became_peered":"2024-04-24T05:46:00.216078+0000","last_unstale":"2024-04-24T05:46:06.059394+0000","last_undegraded":"2024-04-24T05:46:06.059394+0000","last_fullsized":"2024-04-24T05:46:06.059394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060995+0000","last_change":"2024-04-24T05:45:59.206266+0000","last_active":"2024-04-24T05:46:06.060995+0000","last_peered":"2024-04-24T05:46:06.060995+0000","last_clean":"2024-04-24T05:46:06.060995+0000","last_became_active":"2024-04-24T05:45:59.205881+0000","last_became_peered":"2024-04-24T05:45:59.205881+0000","last_unstale":"2024-04-24T05:46:06.060995+0000","last_undegraded":"2024-04-24T05:46:06.060995+0000","last_fullsized":"2024-04-24T05:46:06.060995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:53.435411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058603+0000","last_change":"2024-04-24T05:45:59.217059+0000","last_active":"2024-04-24T05:46:06.058603+0000","last_peered":"2024-04-24T05:46:06.058603+0000","last_clean":"2024-04-24T05:46:06.058603+0000","last_became_active":"2024-04-24T05:45:59.216742+0000","last_became_peered":"2024-04-24T05:45:59.216742+0000","last_unstale":"2024-04-24T05:46:06.058603+0000","last_undegraded":"2024-04-24T05:46:06.058603+0000","last_fullsized":"2024-04-24T05:46:06.058603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062704+0000","last_change":"2024-04-24T05:46:00.215564+0000","last_active":"2024-04-24T05:46:06.062704+0000","last_peered":"2024-04-24T05:46:06.062704+0000","last_clean":"2024-04-24T05:46:06.062704+0000","last_became_active":"2024-04-24T05:46:00.214442+0000","last_became_peered":"2024-04-24T05:46:00.214442+0000","last_unstale":"2024-04-24T05:46:06.062704+0000","last_undegraded":"2024-04-24T05:46:06.062704+0000","last_fullsized":"2024-04-24T05:46:06.062704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059540+0000","last_change":"2024-04-24T05:46:04.272081+0000","last_active":"2024-04-24T05:46:06.059540+0000","last_peered":"2024-04-24T05:46:06.059540+0000","last_clean":"2024-04-24T05:46:06.059540+0000","last_became_active":"2024-04-24T05:46:04.269391+0000","last_became_peered":"2024-04-24T05:46:04.269391+0000","last_unstale":"2024-04-24T05:46:06.059540+0000","last_undegraded":"2024-04-24T05:46:06.059540+0000","last_fullsized":"2024-04-24T05:46:06.059540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:24.966843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061465+0000","last_change":"2024-04-24T05:45:59.210758+0000","last_active":"2024-04-24T05:46:06.061465+0000","last_peered":"2024-04-24T05:46:06.061465+0000","last_clean":"2024-04-24T05:46:06.061465+0000","last_became_active":"2024-04-24T05:45:59.210533+0000","last_became_peered":"2024-04-24T05:45:59.210533+0000","last_unstale":"2024-04-24T05:46:06.061465+0000","last_undegraded":"2024-04-24T05:46:06.061465+0000","last_fullsized":"2024-04-24T05:46:06.061465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:11.561126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.247851+0000","last_change":"2024-04-24T05:46:00.211972+0000","last_active":"2024-04-24T05:46:05.247851+0000","last_peered":"2024-04-24T05:46:05.247851+0000","last_clean":"2024-04-24T05:46:05.247851+0000","last_became_active":"2024-04-24T05:46:00.210764+0000","last_became_peered":"2024-04-24T05:46:00.210764+0000","last_unstale":"2024-04-24T05:46:05.247851+0000","last_undegraded":"2024-04-24T05:46:05.247851+0000","last_fullsized":"2024-04-24T05:46:05.247851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:32.173672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062871+0000","last_change":"2024-04-24T05:46:04.271064+0000","last_active":"2024-04-24T05:46:06.062871+0000","last_peered":"2024-04-24T05:46:06.062871+0000","last_clean":"2024-04-24T05:46:06.062871+0000","last_became_active":"2024-04-24T05:46:04.270662+0000","last_became_peered":"2024-04-24T05:46:04.270662+0000","last_unstale":"2024-04-24T05:46:06.062871+0000","last_undegraded":"2024-04-24T05:46:06.062871+0000","last_fullsized":"2024-04-24T05:46:06.062871+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:18.795481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058290+0000","last_change":"2024-04-24T05:46:00.221213+0000","last_active":"2024-04-24T05:46:06.058290+0000","last_peered":"2024-04-24T05:46:06.058290+0000","last_clean":"2024-04-24T05:46:06.058290+0000","last_became_active":"2024-04-24T05:46:00.220647+0000","last_became_peered":"2024-04-24T05:46:00.220647+0000","last_unstale":"2024-04-24T05:46:06.058290+0000","last_undegraded":"2024-04-24T05:46:06.058290+0000","last_fullsized":"2024-04-24T05:46:06.058290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058501+0000","last_change":"2024-04-24T05:46:00.199902+0000","last_active":"2024-04-24T05:46:06.058501+0000","last_peered":"2024-04-24T05:46:06.058501+0000","last_clean":"2024-04-24T05:46:06.058501+0000","last_became_active":"2024-04-24T05:46:00.199314+0000","last_became_peered":"2024-04-24T05:46:00.199314+0000","last_unstale":"2024-04-24T05:46:06.058501+0000","last_undegraded":"2024-04-24T05:46:06.058501+0000","last_fullsized":"2024-04-24T05:46:06.058501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:27.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251956+0000","last_change":"2024-04-24T05:46:00.212233+0000","last_active":"2024-04-24T05:46:05.251956+0000","last_peered":"2024-04-24T05:46:05.251956+0000","last_clean":"2024-04-24T05:46:05.251956+0000","last_became_active":"2024-04-24T05:46:00.209914+0000","last_became_peered":"2024-04-24T05:46:00.209914+0000","last_unstale":"2024-04-24T05:46:05.251956+0000","last_undegraded":"2024-04-24T05:46:05.251956+0000","last_fullsized":"2024-04-24T05:46:05.251956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:45.039541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062577+0000","last_change":"2024-04-24T05:46:00.216833+0000","last_active":"2024-04-24T05:46:06.062577+0000","last_peered":"2024-04-24T05:46:06.062577+0000","last_clean":"2024-04-24T05:46:06.062577+0000","last_became_active":"2024-04-24T05:46:00.216494+0000","last_became_peered":"2024-04-24T05:46:00.216494+0000","last_unstale":"2024-04-24T05:46:06.062577+0000","last_undegraded":"2024-04-24T05:46:06.062577+0000","last_fullsized":"2024-04-24T05:46:06.062577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062178+0000","last_change":"2024-04-24T05:46:00.213642+0000","last_active":"2024-04-24T05:46:06.062178+0000","last_peered":"2024-04-24T05:46:06.062178+0000","last_clean":"2024-04-24T05:46:06.062178+0000","last_became_active":"2024-04-24T05:46:00.210164+0000","last_became_peered":"2024-04-24T05:46:00.210164+0000","last_unstale":"2024-04-24T05:46:06.062178+0000","last_undegraded":"2024-04-24T05:46:06.062178+0000","last_fullsized":"2024-04-24T05:46:06.062178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:41.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607788+0000","last_change":"2024-04-24T05:46:00.210082+0000","last_active":"2024-04-24T05:46:05.607788+0000","last_peered":"2024-04-24T05:46:05.607788+0000","last_clean":"2024-04-24T05:46:05.607788+0000","last_became_active":"2024-04-24T05:46:00.209799+0000","last_became_peered":"2024-04-24T05:46:00.209799+0000","last_unstale":"2024-04-24T05:46:05.607788+0000","last_undegraded":"2024-04-24T05:46:05.607788+0000","last_fullsized":"2024-04-24T05:46:05.607788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059611+0000","last_change":"2024-04-24T05:46:00.219304+0000","last_active":"2024-04-24T05:46:06.059611+0000","last_peered":"2024-04-24T05:46:06.059611+0000","last_clean":"2024-04-24T05:46:06.059611+0000","last_became_active":"2024-04-24T05:46:00.218953+0000","last_became_peered":"2024-04-24T05:46:00.218953+0000","last_unstale":"2024-04-24T05:46:06.059611+0000","last_undegraded":"2024-04-24T05:46:06.059611+0000","last_fullsized":"2024-04-24T05:46:06.059611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:41.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063234+0000","last_change":"2024-04-24T05:46:00.217971+0000","last_active":"2024-04-24T05:46:06.063234+0000","last_peered":"2024-04-24T05:46:06.063234+0000","last_clean":"2024-04-24T05:46:06.063234+0000","last_became_active":"2024-04-24T05:46:00.217789+0000","last_became_peered":"2024-04-24T05:46:00.217789+0000","last_unstale":"2024-04-24T05:46:06.063234+0000","last_undegraded":"2024-04-24T05:46:06.063234+0000","last_fullsized":"2024-04-24T05:46:06.063234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:38.107685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251295+0000","last_change":"2024-04-24T05:46:00.212887+0000","last_active":"2024-04-24T05:46:05.251295+0000","last_peered":"2024-04-24T05:46:05.251295+0000","last_clean":"2024-04-24T05:46:05.251295+0000","last_became_active":"2024-04-24T05:46:00.212550+0000","last_became_peered":"2024-04-24T05:46:00.212550+0000","last_unstale":"2024-04-24T05:46:05.251295+0000","last_undegraded":"2024-04-24T05:46:05.251295+0000","last_fullsized":"2024-04-24T05:46:05.251295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:25.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060618+0000","last_change":"2024-04-24T05:46:00.219721+0000","last_active":"2024-04-24T05:46:06.060618+0000","last_peered":"2024-04-24T05:46:06.060618+0000","last_clean":"2024-04-24T05:46:06.060618+0000","last_became_active":"2024-04-24T05:46:00.219429+0000","last_became_peered":"2024-04-24T05:46:00.219429+0000","last_unstale":"2024-04-24T05:46:06.060618+0000","last_undegraded":"2024-04-24T05:46:06.060618+0000","last_fullsized":"2024-04-24T05:46:06.060618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:27.080921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251686+0000","last_change":"2024-04-24T05:46:00.215082+0000","last_active":"2024-04-24T05:46:05.251686+0000","last_peered":"2024-04-24T05:46:05.251686+0000","last_clean":"2024-04-24T05:46:05.251686+0000","last_became_active":"2024-04-24T05:46:00.214719+0000","last_became_peered":"2024-04-24T05:46:00.214719+0000","last_unstale":"2024-04-24T05:46:05.251686+0000","last_undegraded":"2024-04-24T05:46:05.251686+0000","last_fullsized":"2024-04-24T05:46:05.251686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:42.688050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061826+0000","last_change":"2024-04-24T05:46:00.204113+0000","last_active":"2024-04-24T05:46:06.061826+0000","last_peered":"2024-04-24T05:46:06.061826+0000","last_clean":"2024-04-24T05:46:06.061826+0000","last_became_active":"2024-04-24T05:46:00.203764+0000","last_became_peered":"2024-04-24T05:46:00.203764+0000","last_unstale":"2024-04-24T05:46:06.061826+0000","last_undegraded":"2024-04-24T05:46:06.061826+0000","last_fullsized":"2024-04-24T05:46:06.061826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059326+0000","last_change":"2024-04-24T05:46:00.223248+0000","last_active":"2024-04-24T05:46:06.059326+0000","last_peered":"2024-04-24T05:46:06.059326+0000","last_clean":"2024-04-24T05:46:06.059326+0000","last_became_active":"2024-04-24T05:46:00.222366+0000","last_became_peered":"2024-04-24T05:46:00.222366+0000","last_unstale":"2024-04-24T05:46:06.059326+0000","last_undegraded":"2024-04-24T05:46:06.059326+0000","last_fullsized":"2024-04-24T05:46:06.059326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558036+0000","last_change":"2024-04-24T05:46:00.217085+0000","last_active":"2024-04-24T05:46:05.558036+0000","last_peered":"2024-04-24T05:46:05.558036+0000","last_clean":"2024-04-24T05:46:05.558036+0000","last_became_active":"2024-04-24T05:46:00.216799+0000","last_became_peered":"2024-04-24T05:46:00.216799+0000","last_unstale":"2024-04-24T05:46:05.558036+0000","last_undegraded":"2024-04-24T05:46:05.558036+0000","last_fullsized":"2024-04-24T05:46:05.558036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:27.050101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059180+0000","last_change":"2024-04-24T05:46:00.221238+0000","last_active":"2024-04-24T05:46:06.059180+0000","last_peered":"2024-04-24T05:46:06.059180+0000","last_clean":"2024-04-24T05:46:06.059180+0000","last_became_active":"2024-04-24T05:46:00.220945+0000","last_became_peered":"2024-04-24T05:46:00.220945+0000","last_unstale":"2024-04-24T05:46:06.059180+0000","last_undegraded":"2024-04-24T05:46:06.059180+0000","last_fullsized":"2024-04-24T05:46:06.059180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:21:57.974456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.254113+0000","last_change":"2024-04-24T05:46:00.206311+0000","last_active":"2024-04-24T05:46:05.254113+0000","last_peered":"2024-04-24T05:46:05.254113+0000","last_clean":"2024-04-24T05:46:05.254113+0000","last_became_active":"2024-04-24T05:46:00.205935+0000","last_became_peered":"2024-04-24T05:46:00.205935+0000","last_unstale":"2024-04-24T05:46:05.254113+0000","last_undegraded":"2024-04-24T05:46:05.254113+0000","last_fullsized":"2024-04-24T05:46:05.254113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:09.200350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251128+0000","last_change":"2024-04-24T05:46:00.202613+0000","last_active":"2024-04-24T05:46:05.251128+0000","last_peered":"2024-04-24T05:46:05.251128+0000","last_clean":"2024-04-24T05:46:05.251128+0000","last_became_active":"2024-04-24T05:46:00.202286+0000","last_became_peered":"2024-04-24T05:46:00.202286+0000","last_unstale":"2024-04-24T05:46:05.251128+0000","last_undegraded":"2024-04-24T05:46:05.251128+0000","last_fullsized":"2024-04-24T05:46:05.251128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606554+0000","last_change":"2024-04-24T05:46:00.222335+0000","last_active":"2024-04-24T05:46:05.606554+0000","last_peered":"2024-04-24T05:46:05.606554+0000","last_clean":"2024-04-24T05:46:05.606554+0000","last_became_active":"2024-04-24T05:46:00.222066+0000","last_became_peered":"2024-04-24T05:46:00.222066+0000","last_unstale":"2024-04-24T05:46:05.606554+0000","last_undegraded":"2024-04-24T05:46:05.606554+0000","last_fullsized":"2024-04-24T05:46:05.606554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060597+0000","last_change":"2024-04-24T05:46:00.213735+0000","last_active":"2024-04-24T05:46:06.060597+0000","last_peered":"2024-04-24T05:46:06.060597+0000","last_clean":"2024-04-24T05:46:06.060597+0000","last_became_active":"2024-04-24T05:46:00.213390+0000","last_became_peered":"2024-04-24T05:46:00.213390+0000","last_unstale":"2024-04-24T05:46:06.060597+0000","last_undegraded":"2024-04-24T05:46:06.060597+0000","last_fullsized":"2024-04-24T05:46:06.060597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:20.914341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061232+0000","last_change":"2024-04-24T05:46:00.219073+0000","last_active":"2024-04-24T05:46:06.061232+0000","last_peered":"2024-04-24T05:46:06.061232+0000","last_clean":"2024-04-24T05:46:06.061232+0000","last_became_active":"2024-04-24T05:46:00.218577+0000","last_became_peered":"2024-04-24T05:46:00.218577+0000","last_unstale":"2024-04-24T05:46:06.061232+0000","last_undegraded":"2024-04-24T05:46:06.061232+0000","last_fullsized":"2024-04-24T05:46:06.061232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248861+0000","last_change":"2024-04-24T05:46:00.219097+0000","last_active":"2024-04-24T05:46:05.248861+0000","last_peered":"2024-04-24T05:46:05.248861+0000","last_clean":"2024-04-24T05:46:05.248861+0000","last_became_active":"2024-04-24T05:46:00.218772+0000","last_became_peered":"2024-04-24T05:46:00.218772+0000","last_unstale":"2024-04-24T05:46:05.248861+0000","last_undegraded":"2024-04-24T05:46:05.248861+0000","last_fullsized":"2024-04-24T05:46:05.248861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:35.088347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059066+0000","last_change":"2024-04-24T05:46:00.221300+0000","last_active":"2024-04-24T05:46:06.059066+0000","last_peered":"2024-04-24T05:46:06.059066+0000","last_clean":"2024-04-24T05:46:06.059066+0000","last_became_active":"2024-04-24T05:46:00.220921+0000","last_became_peered":"2024-04-24T05:46:00.220921+0000","last_unstale":"2024-04-24T05:46:06.059066+0000","last_undegraded":"2024-04-24T05:46:06.059066+0000","last_fullsized":"2024-04-24T05:46:06.059066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059208+0000","last_change":"2024-04-24T05:46:00.221735+0000","last_active":"2024-04-24T05:46:06.059208+0000","last_peered":"2024-04-24T05:46:06.059208+0000","last_clean":"2024-04-24T05:46:06.059208+0000","last_became_active":"2024-04-24T05:46:00.220859+0000","last_became_peered":"2024-04-24T05:46:00.220859+0000","last_unstale":"2024-04-24T05:46:06.059208+0000","last_undegraded":"2024-04-24T05:46:06.059208+0000","last_fullsized":"2024-04-24T05:46:06.059208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:25.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558492+0000","last_change":"2024-04-24T05:46:00.227925+0000","last_active":"2024-04-24T05:46:05.558492+0000","last_peered":"2024-04-24T05:46:05.558492+0000","last_clean":"2024-04-24T05:46:05.558492+0000","last_became_active":"2024-04-24T05:46:00.227553+0000","last_became_peered":"2024-04-24T05:46:00.227553+0000","last_unstale":"2024-04-24T05:46:05.558492+0000","last_undegraded":"2024-04-24T05:46:05.558492+0000","last_fullsized":"2024-04-24T05:46:05.558492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:09.200350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058874+0000","last_change":"2024-04-24T05:46:00.225244+0000","last_active":"2024-04-24T05:46:06.058874+0000","last_peered":"2024-04-24T05:46:06.058874+0000","last_clean":"2024-04-24T05:46:06.058874+0000","last_became_active":"2024-04-24T05:46:00.224935+0000","last_became_peered":"2024-04-24T05:46:00.224935+0000","last_unstale":"2024-04-24T05:46:06.058874+0000","last_undegraded":"2024-04-24T05:46:06.058874+0000","last_fullsized":"2024-04-24T05:46:06.058874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557220+0000","last_change":"2024-04-24T05:46:00.225238+0000","last_active":"2024-04-24T05:46:05.557220+0000","last_peered":"2024-04-24T05:46:05.557220+0000","last_clean":"2024-04-24T05:46:05.557220+0000","last_became_active":"2024-04-24T05:46:00.224903+0000","last_became_peered":"2024-04-24T05:46:00.224903+0000","last_unstale":"2024-04-24T05:46:05.557220+0000","last_undegraded":"2024-04-24T05:46:05.557220+0000","last_fullsized":"2024-04-24T05:46:05.557220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:42.688050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251674+0000","last_change":"2024-04-24T05:46:00.206725+0000","last_active":"2024-04-24T05:46:05.251674+0000","last_peered":"2024-04-24T05:46:05.251674+0000","last_clean":"2024-04-24T05:46:05.251674+0000","last_became_active":"2024-04-24T05:46:00.206266+0000","last_became_peered":"2024-04-24T05:46:00.206266+0000","last_unstale":"2024-04-24T05:46:05.251674+0000","last_undegraded":"2024-04-24T05:46:05.251674+0000","last_fullsized":"2024-04-24T05:46:05.251674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:41.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557088+0000","last_change":"2024-04-24T05:46:00.219593+0000","last_active":"2024-04-24T05:46:05.557088+0000","last_peered":"2024-04-24T05:46:05.557088+0000","last_clean":"2024-04-24T05:46:05.557088+0000","last_became_active":"2024-04-24T05:46:00.219250+0000","last_became_peered":"2024-04-24T05:46:00.219250+0000","last_unstale":"2024-04-24T05:46:05.557088+0000","last_undegraded":"2024-04-24T05:46:05.557088+0000","last_fullsized":"2024-04-24T05:46:05.557088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:45.039541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252270+0000","last_change":"2024-04-24T05:46:00.213476+0000","last_active":"2024-04-24T05:46:05.252270+0000","last_peered":"2024-04-24T05:46:05.252270+0000","last_clean":"2024-04-24T05:46:05.252270+0000","last_became_active":"2024-04-24T05:46:00.213065+0000","last_became_peered":"2024-04-24T05:46:00.213065+0000","last_unstale":"2024-04-24T05:46:05.252270+0000","last_undegraded":"2024-04-24T05:46:05.252270+0000","last_fullsized":"2024-04-24T05:46:05.252270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:38.107685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063065+0000","last_change":"2024-04-24T05:46:00.221236+0000","last_active":"2024-04-24T05:46:06.063065+0000","last_peered":"2024-04-24T05:46:06.063065+0000","last_clean":"2024-04-24T05:46:06.063065+0000","last_became_active":"2024-04-24T05:46:00.220917+0000","last_became_peered":"2024-04-24T05:46:00.220917+0000","last_unstale":"2024-04-24T05:46:06.063065+0000","last_undegraded":"2024-04-24T05:46:06.063065+0000","last_fullsized":"2024-04-24T05:46:06.063065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:38.445039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608230+0000","last_change":"2024-04-24T05:46:00.219494+0000","last_active":"2024-04-24T05:46:05.608230+0000","last_peered":"2024-04-24T05:46:05.608230+0000","last_clean":"2024-04-24T05:46:05.608230+0000","last_became_active":"2024-04-24T05:46:00.219219+0000","last_became_peered":"2024-04-24T05:46:00.219219+0000","last_unstale":"2024-04-24T05:46:05.608230+0000","last_undegraded":"2024-04-24T05:46:05.608230+0000","last_fullsized":"2024-04-24T05:46:05.608230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558194+0000","last_change":"2024-04-24T05:46:00.221425+0000","last_active":"2024-04-24T05:46:05.558194+0000","last_peered":"2024-04-24T05:46:05.558194+0000","last_clean":"2024-04-24T05:46:05.558194+0000","last_became_active":"2024-04-24T05:46:00.219794+0000","last_became_peered":"2024-04-24T05:46:00.219794+0000","last_unstale":"2024-04-24T05:46:05.558194+0000","last_undegraded":"2024-04-24T05:46:05.558194+0000","last_fullsized":"2024-04-24T05:46:05.558194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:55:08.484471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":696263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":109155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":105439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":109155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":105313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":105473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":696319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T05:46:21.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T05:46:21.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.941+0000 7fa2a0e00640 1 Processor -- start 2024-04-24T05:46:21.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.941+0000 7fa2a0e00640 1 -- start start 2024-04-24T05:46:21.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.943+0000 7fa2a0e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa29c06e2f0 0x7fa29c0cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.943+0000 7fa2a0e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa29c0b34b0 0x7fa29c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.943+0000 7fa2a0e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa29c0b3000 con 0x7fa29c15b5c0 2024-04-24T05:46:21.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.943+0000 7fa2a0e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa29c0ce4b0 con 0x7fa29c0b34b0 2024-04-24T05:46:21.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.943+0000 7fa2a0e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa29c06d0c0 con 0x7fa29c06e2f0 2024-04-24T05:46:21.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.943+0000 7fa29b400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa29c15b5c0 0x7fa29c06dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35488/0 (socket says 172.21.15.123:35488) 2024-04-24T05:46:21.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.943+0000 7fa29b400640 1 -- 172.21.15.123:0/1663580938 learned_addr learned my addr 172.21.15.123:0/1663580938 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.943+0000 7fa29aa00640 1 --2- 172.21.15.123:0/1663580938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa29c06e2f0 0x7fa29c0cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 -- 172.21.15.123:0/1663580938 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2669066873 0 0) 0x7fa29c0b3000 con 0x7fa29c15b5c0 2024-04-24T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 -- 172.21.15.123:0/1663580938 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa280003520 con 0x7fa29c15b5c0 2024-04-24T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29be00640 1 --2- 172.21.15.123:0/1663580938 >> v2:172.21.15.155:3300/0 conn(0x7fa29c0b34b0 0x7fa29c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 -- 172.21.15.123:0/1663580938 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2147044796 0 0) 0x7fa280003520 con 0x7fa29c15b5c0 2024-04-24T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 -- 172.21.15.123:0/1663580938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa29c06e2f0 msgr2=0x7fa29c0cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 --2- 172.21.15.123:0/1663580938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa29c06e2f0 0x7fa29c0cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 -- 172.21.15.123:0/1663580938 >> v2:172.21.15.155:3300/0 conn(0x7fa29c0b34b0 msgr2=0x7fa29c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 --2- 172.21.15.123:0/1663580938 >> v2:172.21.15.155:3300/0 conn(0x7fa29c0b34b0 0x7fa29c0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 -- 172.21.15.123:0/1663580938 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29c0ae230 con 0x7fa29c15b5c0 2024-04-24T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 -- 172.21.15.123:0/1663580938 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7fa28c001670 con 0x7fa29c15b5c0 2024-04-24T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 -- 172.21.15.123:0/1663580938 >> v1:172.21.15.123:6789/0 conn(0x7fa29c15b5c0 legacy=0x7fa29c06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 --2- 172.21.15.123:0/1663580938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa29c06e2f0 0x7fa2800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 --2- 172.21.15.123:0/1663580938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa280003f30 0x7fa280004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 --2- 172.21.15.123:0/1663580938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa280004d40 0x7fa2800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 -- 172.21.15.123:0/1663580938 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c0ce4b0 con 0x7fa280004d40 2024-04-24T05:46:21.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 -- 172.21.15.123:0/1663580938 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c06d0c0 con 0x7fa280003f30 2024-04-24T05:46:21.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29a000640 1 -- 172.21.15.123:0/1663580938 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa28000dc20 con 0x7fa29c06e2f0 2024-04-24T05:46:21.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.944+0000 7fa29be00640 1 --2- 172.21.15.123:0/1663580938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa280004d40 0x7fa2800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.945+0000 7fa29b400640 1 --2- 172.21.15.123:0/1663580938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa29c06e2f0 0x7fa2800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.945+0000 7fa29aa00640 1 --2- 172.21.15.123:0/1663580938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa280003f30 0x7fa280004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.945+0000 7fa29be00640 1 -- 172.21.15.123:0/1663580938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa29c06e2f0 msgr2=0x7fa2800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:21.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.945+0000 7fa29be00640 1 --2- 172.21.15.123:0/1663580938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa29c06e2f0 0x7fa2800039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.945+0000 7fa29be00640 1 -- 172.21.15.123:0/1663580938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa280003f30 msgr2=0x7fa280004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:21.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.945+0000 7fa29be00640 1 --2- 172.21.15.123:0/1663580938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa280003f30 0x7fa280004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.945+0000 7fa29be00640 1 -- 172.21.15.123:0/1663580938 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa280003520 con 0x7fa280004d40 2024-04-24T05:46:21.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.946+0000 7fa29be00640 1 --2- 172.21.15.123:0/1663580938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa280004d40 0x7fa2800051c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa294007360 tx=0x7fa280005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c72091913c3353 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:21.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.946+0000 7fa29a000640 1 -- 172.21.15.123:0/1663580938 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294009be0 con 0x7fa280004d40 2024-04-24T05:46:21.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.946+0000 7fa29a000640 1 -- 172.21.15.123:0/1663580938 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa29402e620 con 0x7fa280004d40 2024-04-24T05:46:21.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.946+0000 7fa29a000640 1 -- 172.21.15.123:0/1663580938 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29402e9f0 con 0x7fa280004d40 2024-04-24T05:46:21.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.947+0000 7fa2a0e00640 1 -- 172.21.15.123:0/1663580938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa280004d40 msgr2=0x7fa2800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.947+0000 7fa2a0e00640 1 --2- 172.21.15.123:0/1663580938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa280004d40 0x7fa2800051c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa294007360 tx=0x7fa280005830 comp rx=0 tx=0).stop 2024-04-24T05:46:21.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.947+0000 7fa29be00640 1 -- 172.21.15.123:0/1663580938 reap_dead start 2024-04-24T05:46:21.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.947+0000 7fa2a0e00640 1 -- 172.21.15.123:0/1663580938 shutdown_connections 2024-04-24T05:46:21.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.947+0000 7fa2a0e00640 1 -- 172.21.15.123:0/1663580938 >> 172.21.15.123:0/1663580938 conn(0x7fa29c0d0070 msgr2=0x7fa29c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:21.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.947+0000 7fa2a0e00640 1 -- 172.21.15.123:0/1663580938 shutdown_connections 2024-04-24T05:46:21.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.947+0000 7fa2a0e00640 1 -- 172.21.15.123:0/1663580938 wait complete. 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.949+0000 7fa2a0e00640 1 Processor -- start 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.949+0000 7fa2a0e00640 1 -- start start 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa2a0e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa280004d40 0x7fa29c14ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa2a0e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c14f200 0x7fa29c09b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa2a0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c09bd40 0x7fa29c0ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa2a0e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c06cd20 con 0x7fa29c09bd40 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa2a0e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c0ae510 con 0x7fa29c14f200 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa2a0e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa29c06cea0 con 0x7fa280004d40 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa29be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c09bd40 0x7fa29c0ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa29be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c09bd40 0x7fa29c0ca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56372/0 (socket says 172.21.15.123:56372) 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa29be00640 1 -- 172.21.15.123:0/308797306 learned_addr learned my addr 172.21.15.123:0/308797306 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa29b400640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa280004d40 0x7fa29c14ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa29aa00640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c14f200 0x7fa29c09b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa29be00640 1 -- 172.21.15.123:0/308797306 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa280004d40 msgr2=0x7fa29c14ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa29be00640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa280004d40 0x7fa29c14ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa29be00640 1 -- 172.21.15.123:0/308797306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c14f200 msgr2=0x7fa29c09b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:21.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa29be00640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c14f200 0x7fa29c09b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:21.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa29be00640 1 -- 172.21.15.123:0/308797306 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29c14f860 con 0x7fa29c09bd40 2024-04-24T05:46:21.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.950+0000 7fa29aa00640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c14f200 0x7fa29c09b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:21.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.951+0000 7fa29be00640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c09bd40 0x7fa29c0ca8e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa29402b0a0 tx=0x7fa294004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:21.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.951+0000 7fa293400640 1 -- 172.21.15.123:0/308797306 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294009be0 con 0x7fa29c09bd40 2024-04-24T05:46:21.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.951+0000 7fa293400640 1 -- 172.21.15.123:0/308797306 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa294003990 con 0x7fa29c09bd40 2024-04-24T05:46:21.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.951+0000 7fa293400640 1 -- 172.21.15.123:0/308797306 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294003d60 con 0x7fa29c09bd40 2024-04-24T05:46:21.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.951+0000 7fa2a0e00640 1 -- 172.21.15.123:0/308797306 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa29c14fb40 con 0x7fa29c09bd40 2024-04-24T05:46:21.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.951+0000 7fa2a0e00640 1 -- 172.21.15.123:0/308797306 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa29c150100 con 0x7fa29c09bd40 2024-04-24T05:46:21.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.952+0000 7fa2a0e00640 1 -- 172.21.15.123:0/308797306 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa29c14f5e0 con 0x7fa29c09bd40 2024-04-24T05:46:21.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.953+0000 7fa293400640 1 -- 172.21.15.123:0/308797306 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa294048020 con 0x7fa29c09bd40 2024-04-24T05:46:21.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.954+0000 7fa293400640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa27c06c8f0 0x7fa27c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:21.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.954+0000 7fa293400640 1 -- 172.21.15.123:0/308797306 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa2940ac8a0 con 0x7fa29c09bd40 2024-04-24T05:46:21.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.955+0000 7fa29b400640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa27c06c8f0 0x7fa27c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:21.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.955+0000 7fa29b400640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa27c06c8f0 0x7fa27c06edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa28c000fb0 tx=0x7fa28c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:21.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:21.959+0000 7fa293400640 1 -- 172.21.15.123:0/308797306 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa29c14f5e0 con 0x7fa29c09bd40 2024-04-24T05:46:22.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.137+0000 7fa2a0e00640 1 -- 172.21.15.123:0/308797306 --> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa29c075b70 con 0x7fa27c06c8f0 2024-04-24T05:46:22.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.160+0000 7fa293400640 1 -- 172.21.15.123:0/308797306 <== mgr.4101 v2:172.21.15.123:6832/1503718323 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332574 (secure 0 0 0) 0x7fa29c075b70 con 0x7fa27c06c8f0 2024-04-24T05:46:22.164 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:22.169 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-24T05:46:22.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.168+0000 7fa2a0e00640 1 -- 172.21.15.123:0/308797306 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa27c06c8f0 msgr2=0x7fa27c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:22.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.168+0000 7fa2a0e00640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa27c06c8f0 0x7fa27c06edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa28c000fb0 tx=0x7fa28c009040 comp rx=0 tx=0).stop 2024-04-24T05:46:22.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.168+0000 7fa2a0e00640 1 -- 172.21.15.123:0/308797306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c09bd40 msgr2=0x7fa29c0ca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:22.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.168+0000 7fa2a0e00640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c09bd40 0x7fa29c0ca8e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa29402b0a0 tx=0x7fa294004ab0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.168+0000 7fa2a0e00640 1 -- 172.21.15.123:0/308797306 shutdown_connections 2024-04-24T05:46:22.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.168+0000 7fa2a0e00640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa27c06c8f0 0x7fa27c06edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.168+0000 7fa2a0e00640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa29c09bd40 0x7fa29c0ca8e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.168+0000 7fa2a0e00640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c14f200 0x7fa29c09b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.168+0000 7fa2a0e00640 1 --2- 172.21.15.123:0/308797306 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa280004d40 0x7fa29c14ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.168+0000 7fa2a0e00640 1 -- 172.21.15.123:0/308797306 >> 172.21.15.123:0/308797306 conn(0x7fa29c0d0070 msgr2=0x7fa29c06ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:22.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.168+0000 7fa2a0e00640 1 -- 172.21.15.123:0/308797306 shutdown_connections 2024-04-24T05:46:22.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.168+0000 7fa2a0e00640 1 -- 172.21.15.123:0/308797306 wait complete. 2024-04-24T05:46:22.187 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-24T05:46:20.517929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219476,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755244,"statfs":{"total":773094113280,"available":772869369856,"internally_reserved":0,"allocated":3383296,"data_stored":2032520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006734"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063974+0000","last_change":"2024-04-24T05:45:59.220218+0000","last_active":"2024-04-24T05:46:06.063974+0000","last_peered":"2024-04-24T05:46:06.063974+0000","last_clean":"2024-04-24T05:46:06.063974+0000","last_became_active":"2024-04-24T05:45:59.219888+0000","last_became_peered":"2024-04-24T05:45:59.219888+0000","last_unstale":"2024-04-24T05:46:06.063974+0000","last_undegraded":"2024-04-24T05:46:06.063974+0000","last_fullsized":"2024-04-24T05:46:06.063974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062489+0000","last_change":"2024-04-24T05:46:00.218964+0000","last_active":"2024-04-24T05:46:06.062489+0000","last_peered":"2024-04-24T05:46:06.062489+0000","last_clean":"2024-04-24T05:46:06.062489+0000","last_became_active":"2024-04-24T05:46:00.218589+0000","last_became_peered":"2024-04-24T05:46:00.218589+0000","last_unstale":"2024-04-24T05:46:06.062489+0000","last_undegraded":"2024-04-24T05:46:06.062489+0000","last_fullsized":"2024-04-24T05:46:06.062489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:42.688050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059820+0000","last_change":"2024-04-24T05:46:04.525402+0000","last_active":"2024-04-24T05:46:06.059820+0000","last_peered":"2024-04-24T05:46:06.059820+0000","last_clean":"2024-04-24T05:46:06.059820+0000","last_became_active":"2024-04-24T05:46:04.524690+0000","last_became_peered":"2024-04-24T05:46:04.524690+0000","last_unstale":"2024-04-24T05:46:06.059820+0000","last_undegraded":"2024-04-24T05:46:06.059820+0000","last_fullsized":"2024-04-24T05:46:06.059820+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:15:19.979319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062738+0000","last_change":"2024-04-24T05:45:59.203824+0000","last_active":"2024-04-24T05:46:06.062738+0000","last_peered":"2024-04-24T05:46:06.062738+0000","last_clean":"2024-04-24T05:46:06.062738+0000","last_became_active":"2024-04-24T05:45:59.203396+0000","last_became_peered":"2024-04-24T05:45:59.203396+0000","last_unstale":"2024-04-24T05:46:06.062738+0000","last_undegraded":"2024-04-24T05:46:06.062738+0000","last_fullsized":"2024-04-24T05:46:06.062738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:53:31.596793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063701+0000","last_change":"2024-04-24T05:46:00.221369+0000","last_active":"2024-04-24T05:46:06.063701+0000","last_peered":"2024-04-24T05:46:06.063701+0000","last_clean":"2024-04-24T05:46:06.063701+0000","last_became_active":"2024-04-24T05:46:00.220890+0000","last_became_peered":"2024-04-24T05:46:00.220890+0000","last_unstale":"2024-04-24T05:46:06.063701+0000","last_undegraded":"2024-04-24T05:46:06.063701+0000","last_fullsized":"2024-04-24T05:46:06.063701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:27.080921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.246725+0000","last_change":"2024-04-24T05:46:04.274351+0000","last_active":"2024-04-24T05:46:05.246725+0000","last_peered":"2024-04-24T05:46:05.246725+0000","last_clean":"2024-04-24T05:46:05.246725+0000","last_became_active":"2024-04-24T05:46:04.273892+0000","last_became_peered":"2024-04-24T05:46:04.273892+0000","last_unstale":"2024-04-24T05:46:05.246725+0000","last_undegraded":"2024-04-24T05:46:05.246725+0000","last_fullsized":"2024-04-24T05:46:05.246725+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:39.015857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557980+0000","last_change":"2024-04-24T05:45:59.221936+0000","last_active":"2024-04-24T05:46:05.557980+0000","last_peered":"2024-04-24T05:46:05.557980+0000","last_clean":"2024-04-24T05:46:05.557980+0000","last_became_active":"2024-04-24T05:45:59.221563+0000","last_became_peered":"2024-04-24T05:45:59.221563+0000","last_unstale":"2024-04-24T05:46:05.557980+0000","last_undegraded":"2024-04-24T05:46:05.557980+0000","last_fullsized":"2024-04-24T05:46:05.557980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:34.072620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252559+0000","last_change":"2024-04-24T05:46:00.204200+0000","last_active":"2024-04-24T05:46:05.252559+0000","last_peered":"2024-04-24T05:46:05.252559+0000","last_clean":"2024-04-24T05:46:05.252559+0000","last_became_active":"2024-04-24T05:46:00.203884+0000","last_became_peered":"2024-04-24T05:46:00.203884+0000","last_unstale":"2024-04-24T05:46:05.252559+0000","last_undegraded":"2024-04-24T05:46:05.252559+0000","last_fullsized":"2024-04-24T05:46:05.252559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:05:06.286589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248046+0000","last_change":"2024-04-24T05:46:04.279449+0000","last_active":"2024-04-24T05:46:05.248046+0000","last_peered":"2024-04-24T05:46:05.248046+0000","last_clean":"2024-04-24T05:46:05.248046+0000","last_became_active":"2024-04-24T05:46:04.278915+0000","last_became_peered":"2024-04-24T05:46:04.278915+0000","last_unstale":"2024-04-24T05:46:05.248046+0000","last_undegraded":"2024-04-24T05:46:05.248046+0000","last_fullsized":"2024-04-24T05:46:05.248046+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:42.160187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557287+0000","last_change":"2024-04-24T05:45:59.220243+0000","last_active":"2024-04-24T05:46:05.557287+0000","last_peered":"2024-04-24T05:46:05.557287+0000","last_clean":"2024-04-24T05:46:05.557287+0000","last_became_active":"2024-04-24T05:45:59.219288+0000","last_became_peered":"2024-04-24T05:45:59.219288+0000","last_unstale":"2024-04-24T05:46:05.557287+0000","last_undegraded":"2024-04-24T05:46:05.557287+0000","last_fullsized":"2024-04-24T05:46:05.557287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253052+0000","last_change":"2024-04-24T05:46:00.208516+0000","last_active":"2024-04-24T05:46:05.253052+0000","last_peered":"2024-04-24T05:46:05.253052+0000","last_clean":"2024-04-24T05:46:05.253052+0000","last_became_active":"2024-04-24T05:46:00.208217+0000","last_became_peered":"2024-04-24T05:46:00.208217+0000","last_unstale":"2024-04-24T05:46:05.253052+0000","last_undegraded":"2024-04-24T05:46:05.253052+0000","last_fullsized":"2024-04-24T05:46:05.253052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:13.907547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.247497+0000","last_change":"2024-04-24T05:46:04.274319+0000","last_active":"2024-04-24T05:46:05.247497+0000","last_peered":"2024-04-24T05:46:05.247497+0000","last_clean":"2024-04-24T05:46:05.247497+0000","last_became_active":"2024-04-24T05:46:04.273687+0000","last_became_peered":"2024-04-24T05:46:04.273687+0000","last_unstale":"2024-04-24T05:46:05.247497+0000","last_undegraded":"2024-04-24T05:46:05.247497+0000","last_fullsized":"2024-04-24T05:46:05.247497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:46.740551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058089+0000","last_change":"2024-04-24T05:46:00.222735+0000","last_active":"2024-04-24T05:46:06.058089+0000","last_peered":"2024-04-24T05:46:06.058089+0000","last_clean":"2024-04-24T05:46:06.058089+0000","last_became_active":"2024-04-24T05:46:00.222428+0000","last_became_peered":"2024-04-24T05:46:00.222428+0000","last_unstale":"2024-04-24T05:46:06.058089+0000","last_undegraded":"2024-04-24T05:46:06.058089+0000","last_fullsized":"2024-04-24T05:46:06.058089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557728+0000","last_change":"2024-04-24T05:45:59.220847+0000","last_active":"2024-04-24T05:46:05.557728+0000","last_peered":"2024-04-24T05:46:05.557728+0000","last_clean":"2024-04-24T05:46:05.557728+0000","last_became_active":"2024-04-24T05:45:59.220187+0000","last_became_peered":"2024-04-24T05:45:59.220187+0000","last_unstale":"2024-04-24T05:46:05.557728+0000","last_undegraded":"2024-04-24T05:46:05.557728+0000","last_fullsized":"2024-04-24T05:46:05.557728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:58:40.227841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.249739+0000","last_change":"2024-04-24T05:46:04.271803+0000","last_active":"2024-04-24T05:46:05.249739+0000","last_peered":"2024-04-24T05:46:05.249739+0000","last_clean":"2024-04-24T05:46:05.249739+0000","last_became_active":"2024-04-24T05:46:04.270863+0000","last_became_peered":"2024-04-24T05:46:04.270863+0000","last_unstale":"2024-04-24T05:46:05.249739+0000","last_undegraded":"2024-04-24T05:46:05.249739+0000","last_fullsized":"2024-04-24T05:46:05.249739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:34.349539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061349+0000","last_change":"2024-04-24T05:46:00.216572+0000","last_active":"2024-04-24T05:46:06.061349+0000","last_peered":"2024-04-24T05:46:06.061349+0000","last_clean":"2024-04-24T05:46:06.061349+0000","last_became_active":"2024-04-24T05:46:00.216226+0000","last_became_peered":"2024-04-24T05:46:00.216226+0000","last_unstale":"2024-04-24T05:46:06.061349+0000","last_undegraded":"2024-04-24T05:46:06.061349+0000","last_fullsized":"2024-04-24T05:46:06.061349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607418+0000","last_change":"2024-04-24T05:45:59.201895+0000","last_active":"2024-04-24T05:46:05.607418+0000","last_peered":"2024-04-24T05:46:05.607418+0000","last_clean":"2024-04-24T05:46:05.607418+0000","last_became_active":"2024-04-24T05:45:59.201064+0000","last_became_peered":"2024-04-24T05:45:59.201064+0000","last_unstale":"2024-04-24T05:46:05.607418+0000","last_undegraded":"2024-04-24T05:46:05.607418+0000","last_fullsized":"2024-04-24T05:46:05.607418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248655+0000","last_change":"2024-04-24T05:46:04.278089+0000","last_active":"2024-04-24T05:46:05.248655+0000","last_peered":"2024-04-24T05:46:05.248655+0000","last_clean":"2024-04-24T05:46:05.248655+0000","last_became_active":"2024-04-24T05:46:04.277685+0000","last_became_peered":"2024-04-24T05:46:04.277685+0000","last_unstale":"2024-04-24T05:46:05.248655+0000","last_undegraded":"2024-04-24T05:46:05.248655+0000","last_fullsized":"2024-04-24T05:46:05.248655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:05:10.339090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059639+0000","last_change":"2024-04-24T05:46:00.210339+0000","last_active":"2024-04-24T05:46:06.059639+0000","last_peered":"2024-04-24T05:46:06.059639+0000","last_clean":"2024-04-24T05:46:06.059639+0000","last_became_active":"2024-04-24T05:46:00.209637+0000","last_became_peered":"2024-04-24T05:46:00.209637+0000","last_unstale":"2024-04-24T05:46:06.059639+0000","last_undegraded":"2024-04-24T05:46:06.059639+0000","last_fullsized":"2024-04-24T05:46:06.059639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606639+0000","last_change":"2024-04-24T05:45:59.204528+0000","last_active":"2024-04-24T05:46:05.606639+0000","last_peered":"2024-04-24T05:46:05.606639+0000","last_clean":"2024-04-24T05:46:05.606639+0000","last_became_active":"2024-04-24T05:45:59.204200+0000","last_became_peered":"2024-04-24T05:45:59.204200+0000","last_unstale":"2024-04-24T05:46:05.606639+0000","last_undegraded":"2024-04-24T05:46:05.606639+0000","last_fullsized":"2024-04-24T05:46:05.606639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:26.453566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608562+0000","last_change":"2024-04-24T05:46:04.521949+0000","last_active":"2024-04-24T05:46:05.608562+0000","last_peered":"2024-04-24T05:46:05.608562+0000","last_clean":"2024-04-24T05:46:05.608562+0000","last_became_active":"2024-04-24T05:46:04.521423+0000","last_became_peered":"2024-04-24T05:46:04.521423+0000","last_unstale":"2024-04-24T05:46:05.608562+0000","last_undegraded":"2024-04-24T05:46:05.608562+0000","last_fullsized":"2024-04-24T05:46:05.608562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:44.521141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248230+0000","last_change":"2024-04-24T05:46:00.209888+0000","last_active":"2024-04-24T05:46:05.248230+0000","last_peered":"2024-04-24T05:46:05.248230+0000","last_clean":"2024-04-24T05:46:05.248230+0000","last_became_active":"2024-04-24T05:46:00.209568+0000","last_became_peered":"2024-04-24T05:46:00.209568+0000","last_unstale":"2024-04-24T05:46:05.248230+0000","last_undegraded":"2024-04-24T05:46:05.248230+0000","last_fullsized":"2024-04-24T05:46:05.248230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:21.844840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557732+0000","last_change":"2024-04-24T05:45:59.218584+0000","last_active":"2024-04-24T05:46:05.557732+0000","last_peered":"2024-04-24T05:46:05.557732+0000","last_clean":"2024-04-24T05:46:05.557732+0000","last_became_active":"2024-04-24T05:45:59.218316+0000","last_became_peered":"2024-04-24T05:45:59.218316+0000","last_unstale":"2024-04-24T05:46:05.557732+0000","last_undegraded":"2024-04-24T05:46:05.557732+0000","last_fullsized":"2024-04-24T05:46:05.557732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.249754+0000","last_change":"2024-04-24T05:46:04.281339+0000","last_active":"2024-04-24T05:46:05.249754+0000","last_peered":"2024-04-24T05:46:05.249754+0000","last_clean":"2024-04-24T05:46:05.249754+0000","last_became_active":"2024-04-24T05:46:04.280972+0000","last_became_peered":"2024-04-24T05:46:04.280972+0000","last_unstale":"2024-04-24T05:46:05.249754+0000","last_undegraded":"2024-04-24T05:46:05.249754+0000","last_fullsized":"2024-04-24T05:46:05.249754+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:17.960048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.064250+0000","last_change":"2024-04-24T05:46:00.210893+0000","last_active":"2024-04-24T05:46:06.064250+0000","last_peered":"2024-04-24T05:46:06.064250+0000","last_clean":"2024-04-24T05:46:06.064250+0000","last_became_active":"2024-04-24T05:46:00.210566+0000","last_became_peered":"2024-04-24T05:46:00.210566+0000","last_unstale":"2024-04-24T05:46:06.064250+0000","last_undegraded":"2024-04-24T05:46:06.064250+0000","last_fullsized":"2024-04-24T05:46:06.064250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059489+0000","last_change":"2024-04-24T05:45:59.213341+0000","last_active":"2024-04-24T05:46:06.059489+0000","last_peered":"2024-04-24T05:46:06.059489+0000","last_clean":"2024-04-24T05:46:06.059489+0000","last_became_active":"2024-04-24T05:45:59.212756+0000","last_became_peered":"2024-04-24T05:45:59.212756+0000","last_unstale":"2024-04-24T05:46:06.059489+0000","last_undegraded":"2024-04-24T05:46:06.059489+0000","last_fullsized":"2024-04-24T05:46:06.059489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:34.072620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.247506+0000","last_change":"2024-04-24T05:46:04.524925+0000","last_active":"2024-04-24T05:46:05.247506+0000","last_peered":"2024-04-24T05:46:05.247506+0000","last_clean":"2024-04-24T05:46:05.247506+0000","last_became_active":"2024-04-24T05:46:04.523828+0000","last_became_peered":"2024-04-24T05:46:04.523828+0000","last_unstale":"2024-04-24T05:46:05.247506+0000","last_undegraded":"2024-04-24T05:46:05.247506+0000","last_fullsized":"2024-04-24T05:46:05.247506+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:45.904568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063326+0000","last_change":"2024-04-24T05:46:00.227429+0000","last_active":"2024-04-24T05:46:06.063326+0000","last_peered":"2024-04-24T05:46:06.063326+0000","last_clean":"2024-04-24T05:46:06.063326+0000","last_became_active":"2024-04-24T05:46:00.226909+0000","last_became_peered":"2024-04-24T05:46:00.226909+0000","last_unstale":"2024-04-24T05:46:06.063326+0000","last_undegraded":"2024-04-24T05:46:06.063326+0000","last_fullsized":"2024-04-24T05:46:06.063326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:45.039541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608337+0000","last_change":"2024-04-24T05:45:59.203340+0000","last_active":"2024-04-24T05:46:05.608337+0000","last_peered":"2024-04-24T05:46:05.608337+0000","last_clean":"2024-04-24T05:46:05.608337+0000","last_became_active":"2024-04-24T05:45:59.203004+0000","last_became_peered":"2024-04-24T05:45:59.203004+0000","last_unstale":"2024-04-24T05:46:05.608337+0000","last_undegraded":"2024-04-24T05:46:05.608337+0000","last_fullsized":"2024-04-24T05:46:05.608337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:34.072620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059109+0000","last_change":"2024-04-24T05:46:04.528328+0000","last_active":"2024-04-24T05:46:06.059109+0000","last_peered":"2024-04-24T05:46:06.059109+0000","last_clean":"2024-04-24T05:46:06.059109+0000","last_became_active":"2024-04-24T05:46:04.527760+0000","last_became_peered":"2024-04-24T05:46:04.527760+0000","last_unstale":"2024-04-24T05:46:06.059109+0000","last_undegraded":"2024-04-24T05:46:06.059109+0000","last_fullsized":"2024-04-24T05:46:06.059109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:16:08.677684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607804+0000","last_change":"2024-04-24T05:46:00.215704+0000","last_active":"2024-04-24T05:46:05.607804+0000","last_peered":"2024-04-24T05:46:05.607804+0000","last_clean":"2024-04-24T05:46:05.607804+0000","last_became_active":"2024-04-24T05:46:00.214751+0000","last_became_peered":"2024-04-24T05:46:00.214751+0000","last_unstale":"2024-04-24T05:46:05.607804+0000","last_undegraded":"2024-04-24T05:46:05.607804+0000","last_fullsized":"2024-04-24T05:46:05.607804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558293+0000","last_change":"2024-04-24T05:45:59.219396+0000","last_active":"2024-04-24T05:46:05.558293+0000","last_peered":"2024-04-24T05:46:05.558293+0000","last_clean":"2024-04-24T05:46:05.558293+0000","last_became_active":"2024-04-24T05:45:59.217765+0000","last_became_peered":"2024-04-24T05:45:59.217765+0000","last_unstale":"2024-04-24T05:46:05.558293+0000","last_undegraded":"2024-04-24T05:46:05.558293+0000","last_fullsized":"2024-04-24T05:46:05.558293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:21:56.958729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061548+0000","last_change":"2024-04-24T05:46:04.518477+0000","last_active":"2024-04-24T05:46:06.061548+0000","last_peered":"2024-04-24T05:46:06.061548+0000","last_clean":"2024-04-24T05:46:06.061548+0000","last_became_active":"2024-04-24T05:46:04.515078+0000","last_became_peered":"2024-04-24T05:46:04.515078+0000","last_unstale":"2024-04-24T05:46:06.061548+0000","last_undegraded":"2024-04-24T05:46:06.061548+0000","last_fullsized":"2024-04-24T05:46:06.061548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:31.102602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253112+0000","last_change":"2024-04-24T05:46:00.212998+0000","last_active":"2024-04-24T05:46:05.253112+0000","last_peered":"2024-04-24T05:46:05.253112+0000","last_clean":"2024-04-24T05:46:05.253112+0000","last_became_active":"2024-04-24T05:46:00.212446+0000","last_became_peered":"2024-04-24T05:46:00.212446+0000","last_unstale":"2024-04-24T05:46:05.253112+0000","last_undegraded":"2024-04-24T05:46:05.253112+0000","last_fullsized":"2024-04-24T05:46:05.253112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:27.050101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607823+0000","last_change":"2024-04-24T05:45:59.197842+0000","last_active":"2024-04-24T05:46:05.607823+0000","last_peered":"2024-04-24T05:46:05.607823+0000","last_clean":"2024-04-24T05:46:05.607823+0000","last_became_active":"2024-04-24T05:45:59.197510+0000","last_became_peered":"2024-04-24T05:45:59.197510+0000","last_unstale":"2024-04-24T05:46:05.607823+0000","last_undegraded":"2024-04-24T05:46:05.607823+0000","last_fullsized":"2024-04-24T05:46:05.607823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606699+0000","last_change":"2024-04-24T05:46:04.270957+0000","last_active":"2024-04-24T05:46:05.606699+0000","last_peered":"2024-04-24T05:46:05.606699+0000","last_clean":"2024-04-24T05:46:05.606699+0000","last_became_active":"2024-04-24T05:46:04.268656+0000","last_became_peered":"2024-04-24T05:46:04.268656+0000","last_unstale":"2024-04-24T05:46:05.606699+0000","last_undegraded":"2024-04-24T05:46:05.606699+0000","last_fullsized":"2024-04-24T05:46:05.606699+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:31.102602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248701+0000","last_change":"2024-04-24T05:46:00.205128+0000","last_active":"2024-04-24T05:46:05.248701+0000","last_peered":"2024-04-24T05:46:05.248701+0000","last_clean":"2024-04-24T05:46:05.248701+0000","last_became_active":"2024-04-24T05:46:00.202044+0000","last_became_peered":"2024-04-24T05:46:00.202044+0000","last_unstale":"2024-04-24T05:46:05.248701+0000","last_undegraded":"2024-04-24T05:46:05.248701+0000","last_fullsized":"2024-04-24T05:46:05.248701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:18.772951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252226+0000","last_change":"2024-04-24T05:45:59.199320+0000","last_active":"2024-04-24T05:46:05.252226+0000","last_peered":"2024-04-24T05:46:05.252226+0000","last_clean":"2024-04-24T05:46:05.252226+0000","last_became_active":"2024-04-24T05:45:59.198382+0000","last_became_peered":"2024-04-24T05:45:59.198382+0000","last_unstale":"2024-04-24T05:46:05.252226+0000","last_undegraded":"2024-04-24T05:46:05.252226+0000","last_fullsized":"2024-04-24T05:46:05.252226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607713+0000","last_change":"2024-04-24T05:46:04.273947+0000","last_active":"2024-04-24T05:46:05.607713+0000","last_peered":"2024-04-24T05:46:05.607713+0000","last_clean":"2024-04-24T05:46:05.607713+0000","last_became_active":"2024-04-24T05:46:04.271583+0000","last_became_peered":"2024-04-24T05:46:04.271583+0000","last_unstale":"2024-04-24T05:46:05.607713+0000","last_undegraded":"2024-04-24T05:46:05.607713+0000","last_fullsized":"2024-04-24T05:46:05.607713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:45.991179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606401+0000","last_change":"2024-04-24T05:46:00.211997+0000","last_active":"2024-04-24T05:46:05.606401+0000","last_peered":"2024-04-24T05:46:05.606401+0000","last_clean":"2024-04-24T05:46:05.606401+0000","last_became_active":"2024-04-24T05:46:00.209526+0000","last_became_peered":"2024-04-24T05:46:00.209526+0000","last_unstale":"2024-04-24T05:46:05.606401+0000","last_undegraded":"2024-04-24T05:46:05.606401+0000","last_fullsized":"2024-04-24T05:46:05.606401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:27.080921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058600+0000","last_change":"2024-04-24T05:45:59.198738+0000","last_active":"2024-04-24T05:46:06.058600+0000","last_peered":"2024-04-24T05:46:06.058600+0000","last_clean":"2024-04-24T05:46:06.058600+0000","last_became_active":"2024-04-24T05:45:59.198458+0000","last_became_peered":"2024-04-24T05:45:59.198458+0000","last_unstale":"2024-04-24T05:46:06.058600+0000","last_undegraded":"2024-04-24T05:46:06.058600+0000","last_fullsized":"2024-04-24T05:46:06.058600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:11.561126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608670+0000","last_change":"2024-04-24T05:46:04.266757+0000","last_active":"2024-04-24T05:46:05.608670+0000","last_peered":"2024-04-24T05:46:05.608670+0000","last_clean":"2024-04-24T05:46:05.608670+0000","last_became_active":"2024-04-24T05:46:04.266279+0000","last_became_peered":"2024-04-24T05:46:04.266279+0000","last_unstale":"2024-04-24T05:46:05.608670+0000","last_undegraded":"2024-04-24T05:46:05.608670+0000","last_fullsized":"2024-04-24T05:46:05.608670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:06:00.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058554+0000","last_change":"2024-04-24T05:46:00.216303+0000","last_active":"2024-04-24T05:46:06.058554+0000","last_peered":"2024-04-24T05:46:06.058554+0000","last_clean":"2024-04-24T05:46:06.058554+0000","last_became_active":"2024-04-24T05:46:00.215719+0000","last_became_peered":"2024-04-24T05:46:00.215719+0000","last_unstale":"2024-04-24T05:46:06.058554+0000","last_undegraded":"2024-04-24T05:46:06.058554+0000","last_fullsized":"2024-04-24T05:46:06.058554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:47:58.509148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558034+0000","last_change":"2024-04-24T05:45:59.218643+0000","last_active":"2024-04-24T05:46:05.558034+0000","last_peered":"2024-04-24T05:46:05.558034+0000","last_clean":"2024-04-24T05:46:05.558034+0000","last_became_active":"2024-04-24T05:45:59.218346+0000","last_became_peered":"2024-04-24T05:45:59.218346+0000","last_unstale":"2024-04-24T05:46:05.558034+0000","last_undegraded":"2024-04-24T05:46:05.558034+0000","last_fullsized":"2024-04-24T05:46:05.558034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:26.453566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063814+0000","last_change":"2024-04-24T05:46:04.527188+0000","last_active":"2024-04-24T05:46:06.063814+0000","last_peered":"2024-04-24T05:46:06.063814+0000","last_clean":"2024-04-24T05:46:06.063814+0000","last_became_active":"2024-04-24T05:46:04.526002+0000","last_became_peered":"2024-04-24T05:46:04.526002+0000","last_unstale":"2024-04-24T05:46:06.063814+0000","last_undegraded":"2024-04-24T05:46:06.063814+0000","last_fullsized":"2024-04-24T05:46:06.063814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:06:00.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060620+0000","last_change":"2024-04-24T05:46:00.224034+0000","last_active":"2024-04-24T05:46:06.060620+0000","last_peered":"2024-04-24T05:46:06.060620+0000","last_clean":"2024-04-24T05:46:06.060620+0000","last_became_active":"2024-04-24T05:46:00.223714+0000","last_became_peered":"2024-04-24T05:46:00.223714+0000","last_unstale":"2024-04-24T05:46:06.060620+0000","last_undegraded":"2024-04-24T05:46:06.060620+0000","last_fullsized":"2024-04-24T05:46:06.060620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:25.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252259+0000","last_change":"2024-04-24T05:45:59.199095+0000","last_active":"2024-04-24T05:46:05.252259+0000","last_peered":"2024-04-24T05:46:05.252259+0000","last_clean":"2024-04-24T05:46:05.252259+0000","last_became_active":"2024-04-24T05:45:59.198772+0000","last_became_peered":"2024-04-24T05:45:59.198772+0000","last_unstale":"2024-04-24T05:46:05.252259+0000","last_undegraded":"2024-04-24T05:46:05.252259+0000","last_fullsized":"2024-04-24T05:46:05.252259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:11.561126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558345+0000","last_change":"2024-04-24T05:45:51.019903+0000","last_active":"2024-04-24T05:46:05.558345+0000","last_peered":"2024-04-24T05:46:05.558345+0000","last_clean":"2024-04-24T05:46:05.558345+0000","last_became_active":"2024-04-24T05:45:51.017323+0000","last_became_peered":"2024-04-24T05:45:51.017323+0000","last_unstale":"2024-04-24T05:46:05.558345+0000","last_undegraded":"2024-04-24T05:46:05.558345+0000","last_fullsized":"2024-04-24T05:46:05.558345+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:48.855871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:48.855871+0000","last_clean_scrub_stamp":"2024-04-24T05:45:48.855871+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:10:41.045203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058781+0000","last_change":"2024-04-24T05:46:04.525962+0000","last_active":"2024-04-24T05:46:06.058781+0000","last_peered":"2024-04-24T05:46:06.058781+0000","last_clean":"2024-04-24T05:46:06.058781+0000","last_became_active":"2024-04-24T05:46:04.524947+0000","last_became_peered":"2024-04-24T05:46:04.524947+0000","last_unstale":"2024-04-24T05:46:06.058781+0000","last_undegraded":"2024-04-24T05:46:06.058781+0000","last_fullsized":"2024-04-24T05:46:06.058781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:17.960048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557690+0000","last_change":"2024-04-24T05:46:00.226016+0000","last_active":"2024-04-24T05:46:05.557690+0000","last_peered":"2024-04-24T05:46:05.557690+0000","last_clean":"2024-04-24T05:46:05.557690+0000","last_became_active":"2024-04-24T05:46:00.225693+0000","last_became_peered":"2024-04-24T05:46:00.225693+0000","last_unstale":"2024-04-24T05:46:05.557690+0000","last_undegraded":"2024-04-24T05:46:05.557690+0000","last_fullsized":"2024-04-24T05:46:05.557690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:13.907547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059847+0000","last_change":"2024-04-24T05:45:59.203021+0000","last_active":"2024-04-24T05:46:06.059847+0000","last_peered":"2024-04-24T05:46:06.059847+0000","last_clean":"2024-04-24T05:46:06.059847+0000","last_became_active":"2024-04-24T05:45:59.202733+0000","last_became_peered":"2024-04-24T05:45:59.202733+0000","last_unstale":"2024-04-24T05:46:06.059847+0000","last_undegraded":"2024-04-24T05:46:06.059847+0000","last_fullsized":"2024-04-24T05:46:06.059847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058297+0000","last_change":"2024-04-24T05:46:04.521125+0000","last_active":"2024-04-24T05:46:06.058297+0000","last_peered":"2024-04-24T05:46:06.058297+0000","last_clean":"2024-04-24T05:46:06.058297+0000","last_became_active":"2024-04-24T05:46:04.519878+0000","last_became_peered":"2024-04-24T05:46:04.519878+0000","last_unstale":"2024-04-24T05:46:06.058297+0000","last_undegraded":"2024-04-24T05:46:06.058297+0000","last_fullsized":"2024-04-24T05:46:06.058297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:34.349539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252701+0000","last_change":"2024-04-24T05:46:00.214353+0000","last_active":"2024-04-24T05:46:05.252701+0000","last_peered":"2024-04-24T05:46:05.252701+0000","last_clean":"2024-04-24T05:46:05.252701+0000","last_became_active":"2024-04-24T05:46:00.213791+0000","last_became_peered":"2024-04-24T05:46:00.213791+0000","last_unstale":"2024-04-24T05:46:05.252701+0000","last_undegraded":"2024-04-24T05:46:05.252701+0000","last_fullsized":"2024-04-24T05:46:05.252701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:30.297037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608191+0000","last_change":"2024-04-24T05:45:59.216779+0000","last_active":"2024-04-24T05:46:05.608191+0000","last_peered":"2024-04-24T05:46:05.608191+0000","last_clean":"2024-04-24T05:46:05.608191+0000","last_became_active":"2024-04-24T05:45:59.216507+0000","last_became_peered":"2024-04-24T05:45:59.216507+0000","last_unstale":"2024-04-24T05:46:05.608191+0000","last_undegraded":"2024-04-24T05:46:05.608191+0000","last_fullsized":"2024-04-24T05:46:05.608191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062332+0000","last_change":"2024-04-24T05:46:04.268777+0000","last_active":"2024-04-24T05:46:06.062332+0000","last_peered":"2024-04-24T05:46:06.062332+0000","last_clean":"2024-04-24T05:46:06.062332+0000","last_became_active":"2024-04-24T05:46:04.268416+0000","last_became_peered":"2024-04-24T05:46:04.268416+0000","last_unstale":"2024-04-24T05:46:06.062332+0000","last_undegraded":"2024-04-24T05:46:06.062332+0000","last_fullsized":"2024-04-24T05:46:06.062332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:13.252851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558634+0000","last_change":"2024-04-24T05:46:00.227851+0000","last_active":"2024-04-24T05:46:05.558634+0000","last_peered":"2024-04-24T05:46:05.558634+0000","last_clean":"2024-04-24T05:46:05.558634+0000","last_became_active":"2024-04-24T05:46:00.220088+0000","last_became_peered":"2024-04-24T05:46:00.220088+0000","last_unstale":"2024-04-24T05:46:05.558634+0000","last_undegraded":"2024-04-24T05:46:05.558634+0000","last_fullsized":"2024-04-24T05:46:05.558634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:40.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059276+0000","last_change":"2024-04-24T05:45:59.196319+0000","last_active":"2024-04-24T05:46:06.059276+0000","last_peered":"2024-04-24T05:46:06.059276+0000","last_clean":"2024-04-24T05:46:06.059276+0000","last_became_active":"2024-04-24T05:45:59.196029+0000","last_became_peered":"2024-04-24T05:45:59.196029+0000","last_unstale":"2024-04-24T05:46:06.059276+0000","last_undegraded":"2024-04-24T05:46:06.059276+0000","last_fullsized":"2024-04-24T05:46:06.059276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.250996+0000","last_change":"2024-04-24T05:46:04.527040+0000","last_active":"2024-04-24T05:46:05.250996+0000","last_peered":"2024-04-24T05:46:05.250996+0000","last_clean":"2024-04-24T05:46:05.250996+0000","last_became_active":"2024-04-24T05:46:04.522889+0000","last_became_peered":"2024-04-24T05:46:04.522889+0000","last_unstale":"2024-04-24T05:46:05.250996+0000","last_undegraded":"2024-04-24T05:46:05.250996+0000","last_fullsized":"2024-04-24T05:46:05.250996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:06:00.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558335+0000","last_change":"2024-04-24T05:46:00.225225+0000","last_active":"2024-04-24T05:46:05.558335+0000","last_peered":"2024-04-24T05:46:05.558335+0000","last_clean":"2024-04-24T05:46:05.558335+0000","last_became_active":"2024-04-24T05:46:00.224932+0000","last_became_peered":"2024-04-24T05:46:00.224932+0000","last_unstale":"2024-04-24T05:46:05.558335+0000","last_undegraded":"2024-04-24T05:46:05.558335+0000","last_fullsized":"2024-04-24T05:46:05.558335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:41.938677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248435+0000","last_change":"2024-04-24T05:45:59.214909+0000","last_active":"2024-04-24T05:46:05.248435+0000","last_peered":"2024-04-24T05:46:05.248435+0000","last_clean":"2024-04-24T05:46:05.248435+0000","last_became_active":"2024-04-24T05:45:59.213540+0000","last_became_peered":"2024-04-24T05:45:59.213540+0000","last_unstale":"2024-04-24T05:46:05.248435+0000","last_undegraded":"2024-04-24T05:46:05.248435+0000","last_fullsized":"2024-04-24T05:46:05.248435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:00.340470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.246897+0000","last_change":"2024-04-24T05:46:04.275887+0000","last_active":"2024-04-24T05:46:05.246897+0000","last_peered":"2024-04-24T05:46:05.246897+0000","last_clean":"2024-04-24T05:46:05.246897+0000","last_became_active":"2024-04-24T05:46:04.275455+0000","last_became_peered":"2024-04-24T05:46:04.275455+0000","last_unstale":"2024-04-24T05:46:05.246897+0000","last_undegraded":"2024-04-24T05:46:05.246897+0000","last_fullsized":"2024-04-24T05:46:05.246897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:29.075099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557429+0000","last_change":"2024-04-24T05:46:00.217599+0000","last_active":"2024-04-24T05:46:05.557429+0000","last_peered":"2024-04-24T05:46:05.557429+0000","last_clean":"2024-04-24T05:46:05.557429+0000","last_became_active":"2024-04-24T05:46:00.217383+0000","last_became_peered":"2024-04-24T05:46:00.217383+0000","last_unstale":"2024-04-24T05:46:05.557429+0000","last_undegraded":"2024-04-24T05:46:05.557429+0000","last_fullsized":"2024-04-24T05:46:05.557429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:38.107685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.057867+0000","last_change":"2024-04-24T05:45:59.196826+0000","last_active":"2024-04-24T05:46:06.057867+0000","last_peered":"2024-04-24T05:46:06.057867+0000","last_clean":"2024-04-24T05:46:06.057867+0000","last_became_active":"2024-04-24T05:45:59.195023+0000","last_became_peered":"2024-04-24T05:45:59.195023+0000","last_unstale":"2024-04-24T05:46:06.057867+0000","last_undegraded":"2024-04-24T05:46:06.057867+0000","last_fullsized":"2024-04-24T05:46:06.057867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063777+0000","last_change":"2024-04-24T05:46:04.265949+0000","last_active":"2024-04-24T05:46:06.063777+0000","last_peered":"2024-04-24T05:46:06.063777+0000","last_clean":"2024-04-24T05:46:06.063777+0000","last_became_active":"2024-04-24T05:46:04.265546+0000","last_became_peered":"2024-04-24T05:46:04.265546+0000","last_unstale":"2024-04-24T05:46:06.063777+0000","last_undegraded":"2024-04-24T05:46:06.063777+0000","last_fullsized":"2024-04-24T05:46:06.063777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:44.521141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058813+0000","last_change":"2024-04-24T05:46:00.221203+0000","last_active":"2024-04-24T05:46:06.058813+0000","last_peered":"2024-04-24T05:46:06.058813+0000","last_clean":"2024-04-24T05:46:06.058813+0000","last_became_active":"2024-04-24T05:46:00.220709+0000","last_became_peered":"2024-04-24T05:46:00.220709+0000","last_unstale":"2024-04-24T05:46:06.058813+0000","last_undegraded":"2024-04-24T05:46:06.058813+0000","last_fullsized":"2024-04-24T05:46:06.058813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059170+0000","last_change":"2024-04-24T05:45:59.217694+0000","last_active":"2024-04-24T05:46:06.059170+0000","last_peered":"2024-04-24T05:46:06.059170+0000","last_clean":"2024-04-24T05:46:06.059170+0000","last_became_active":"2024-04-24T05:45:59.217385+0000","last_became_peered":"2024-04-24T05:45:59.217385+0000","last_unstale":"2024-04-24T05:46:06.059170+0000","last_undegraded":"2024-04-24T05:46:06.059170+0000","last_fullsized":"2024-04-24T05:46:06.059170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063100+0000","last_change":"2024-04-24T05:46:04.529839+0000","last_active":"2024-04-24T05:46:06.063100+0000","last_peered":"2024-04-24T05:46:06.063100+0000","last_clean":"2024-04-24T05:46:06.063100+0000","last_became_active":"2024-04-24T05:46:04.524578+0000","last_became_peered":"2024-04-24T05:46:04.524578+0000","last_unstale":"2024-04-24T05:46:06.063100+0000","last_undegraded":"2024-04-24T05:46:06.063100+0000","last_fullsized":"2024-04-24T05:46:06.063100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:26.692990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557529+0000","last_change":"2024-04-24T05:46:00.226661+0000","last_active":"2024-04-24T05:46:05.557529+0000","last_peered":"2024-04-24T05:46:05.557529+0000","last_clean":"2024-04-24T05:46:05.557529+0000","last_became_active":"2024-04-24T05:46:00.226357+0000","last_became_peered":"2024-04-24T05:46:00.226357+0000","last_unstale":"2024-04-24T05:46:05.557529+0000","last_undegraded":"2024-04-24T05:46:05.557529+0000","last_fullsized":"2024-04-24T05:46:05.557529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:05:06.286589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252051+0000","last_change":"2024-04-24T05:45:59.206276+0000","last_active":"2024-04-24T05:46:05.252051+0000","last_peered":"2024-04-24T05:46:05.252051+0000","last_clean":"2024-04-24T05:46:05.252051+0000","last_became_active":"2024-04-24T05:45:59.205926+0000","last_became_peered":"2024-04-24T05:45:59.205926+0000","last_unstale":"2024-04-24T05:46:05.252051+0000","last_undegraded":"2024-04-24T05:46:05.252051+0000","last_fullsized":"2024-04-24T05:46:05.252051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062045+0000","last_change":"2024-04-24T05:46:04.515144+0000","last_active":"2024-04-24T05:46:06.062045+0000","last_peered":"2024-04-24T05:46:06.062045+0000","last_clean":"2024-04-24T05:46:06.062045+0000","last_became_active":"2024-04-24T05:46:04.514326+0000","last_became_peered":"2024-04-24T05:46:04.514326+0000","last_unstale":"2024-04-24T05:46:06.062045+0000","last_undegraded":"2024-04-24T05:46:06.062045+0000","last_fullsized":"2024-04-24T05:46:06.062045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:45.847497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060827+0000","last_change":"2024-04-24T05:46:00.214864+0000","last_active":"2024-04-24T05:46:06.060827+0000","last_peered":"2024-04-24T05:46:06.060827+0000","last_clean":"2024-04-24T05:46:06.060827+0000","last_became_active":"2024-04-24T05:46:00.214051+0000","last_became_peered":"2024-04-24T05:46:00.214051+0000","last_unstale":"2024-04-24T05:46:06.060827+0000","last_undegraded":"2024-04-24T05:46:06.060827+0000","last_fullsized":"2024-04-24T05:46:06.060827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:41:43.988057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062825+0000","last_change":"2024-04-24T05:45:59.213057+0000","last_active":"2024-04-24T05:46:06.062825+0000","last_peered":"2024-04-24T05:46:06.062825+0000","last_clean":"2024-04-24T05:46:06.062825+0000","last_became_active":"2024-04-24T05:45:59.212716+0000","last_became_peered":"2024-04-24T05:45:59.212716+0000","last_unstale":"2024-04-24T05:46:06.062825+0000","last_undegraded":"2024-04-24T05:46:06.062825+0000","last_fullsized":"2024-04-24T05:46:06.062825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:53.435411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252861+0000","last_change":"2024-04-24T05:46:04.516510+0000","last_active":"2024-04-24T05:46:05.252861+0000","last_peered":"2024-04-24T05:46:05.252861+0000","last_clean":"2024-04-24T05:46:05.252861+0000","last_became_active":"2024-04-24T05:46:04.515458+0000","last_became_peered":"2024-04-24T05:46:04.515458+0000","last_unstale":"2024-04-24T05:46:05.252861+0000","last_undegraded":"2024-04-24T05:46:05.252861+0000","last_fullsized":"2024-04-24T05:46:05.252861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:16:08.677684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557591+0000","last_change":"2024-04-24T05:46:00.215087+0000","last_active":"2024-04-24T05:46:05.557591+0000","last_peered":"2024-04-24T05:46:05.557591+0000","last_clean":"2024-04-24T05:46:05.557591+0000","last_became_active":"2024-04-24T05:46:00.213783+0000","last_became_peered":"2024-04-24T05:46:00.213783+0000","last_unstale":"2024-04-24T05:46:05.557591+0000","last_undegraded":"2024-04-24T05:46:05.557591+0000","last_fullsized":"2024-04-24T05:46:05.557591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:30.297037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:17.734750+0000","last_change":"2024-04-24T05:45:59.211881+0000","last_active":"2024-04-24T05:46:17.734750+0000","last_peered":"2024-04-24T05:46:17.734750+0000","last_clean":"2024-04-24T05:46:17.734750+0000","last_became_active":"2024-04-24T05:45:59.211566+0000","last_became_peered":"2024-04-24T05:45:59.211566+0000","last_unstale":"2024-04-24T05:46:17.734750+0000","last_undegraded":"2024-04-24T05:46:17.734750+0000","last_fullsized":"2024-04-24T05:46:17.734750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607263+0000","last_change":"2024-04-24T05:46:04.264433+0000","last_active":"2024-04-24T05:46:05.607263+0000","last_peered":"2024-04-24T05:46:05.607263+0000","last_clean":"2024-04-24T05:46:05.607263+0000","last_became_active":"2024-04-24T05:46:04.263926+0000","last_became_peered":"2024-04-24T05:46:04.263926+0000","last_unstale":"2024-04-24T05:46:05.607263+0000","last_undegraded":"2024-04-24T05:46:05.607263+0000","last_fullsized":"2024-04-24T05:46:05.607263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:13.252851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248914+0000","last_change":"2024-04-24T05:46:00.218334+0000","last_active":"2024-04-24T05:46:05.248914+0000","last_peered":"2024-04-24T05:46:05.248914+0000","last_clean":"2024-04-24T05:46:05.248914+0000","last_became_active":"2024-04-24T05:46:00.213738+0000","last_became_peered":"2024-04-24T05:46:00.213738+0000","last_unstale":"2024-04-24T05:46:05.248914+0000","last_undegraded":"2024-04-24T05:46:05.248914+0000","last_fullsized":"2024-04-24T05:46:05.248914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:27.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608102+0000","last_change":"2024-04-24T05:45:59.206057+0000","last_active":"2024-04-24T05:46:05.608102+0000","last_peered":"2024-04-24T05:46:05.608102+0000","last_clean":"2024-04-24T05:46:05.608102+0000","last_became_active":"2024-04-24T05:45:59.205560+0000","last_became_peered":"2024-04-24T05:45:59.205560+0000","last_unstale":"2024-04-24T05:46:05.608102+0000","last_undegraded":"2024-04-24T05:46:05.608102+0000","last_fullsized":"2024-04-24T05:46:05.608102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248223+0000","last_change":"2024-04-24T05:46:04.274265+0000","last_active":"2024-04-24T05:46:05.248223+0000","last_peered":"2024-04-24T05:46:05.248223+0000","last_clean":"2024-04-24T05:46:05.248223+0000","last_became_active":"2024-04-24T05:46:04.273775+0000","last_became_peered":"2024-04-24T05:46:04.273775+0000","last_unstale":"2024-04-24T05:46:05.248223+0000","last_undegraded":"2024-04-24T05:46:05.248223+0000","last_fullsized":"2024-04-24T05:46:05.248223+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:49.092042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063970+0000","last_change":"2024-04-24T05:46:00.222245+0000","last_active":"2024-04-24T05:46:06.063970+0000","last_peered":"2024-04-24T05:46:06.063970+0000","last_clean":"2024-04-24T05:46:06.063970+0000","last_became_active":"2024-04-24T05:46:00.220540+0000","last_became_peered":"2024-04-24T05:46:00.220540+0000","last_unstale":"2024-04-24T05:46:06.063970+0000","last_undegraded":"2024-04-24T05:46:06.063970+0000","last_fullsized":"2024-04-24T05:46:06.063970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253552+0000","last_change":"2024-04-24T05:45:59.203808+0000","last_active":"2024-04-24T05:46:05.253552+0000","last_peered":"2024-04-24T05:46:05.253552+0000","last_clean":"2024-04-24T05:46:05.253552+0000","last_became_active":"2024-04-24T05:45:59.203023+0000","last_became_peered":"2024-04-24T05:45:59.203023+0000","last_unstale":"2024-04-24T05:46:05.253552+0000","last_undegraded":"2024-04-24T05:46:05.253552+0000","last_fullsized":"2024-04-24T05:46:05.253552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059474+0000","last_change":"2024-04-24T05:46:04.274610+0000","last_active":"2024-04-24T05:46:06.059474+0000","last_peered":"2024-04-24T05:46:06.059474+0000","last_clean":"2024-04-24T05:46:06.059474+0000","last_became_active":"2024-04-24T05:46:04.274246+0000","last_became_peered":"2024-04-24T05:46:04.274246+0000","last_unstale":"2024-04-24T05:46:06.059474+0000","last_undegraded":"2024-04-24T05:46:06.059474+0000","last_fullsized":"2024-04-24T05:46:06.059474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:31:17.536498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253481+0000","last_change":"2024-04-24T05:46:00.207959+0000","last_active":"2024-04-24T05:46:05.253481+0000","last_peered":"2024-04-24T05:46:05.253481+0000","last_clean":"2024-04-24T05:46:05.253481+0000","last_became_active":"2024-04-24T05:46:00.205225+0000","last_became_peered":"2024-04-24T05:46:00.205225+0000","last_unstale":"2024-04-24T05:46:05.253481+0000","last_undegraded":"2024-04-24T05:46:05.253481+0000","last_fullsized":"2024-04-24T05:46:05.253481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:55:08.484471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059232+0000","last_change":"2024-04-24T05:45:59.212223+0000","last_active":"2024-04-24T05:46:06.059232+0000","last_peered":"2024-04-24T05:46:06.059232+0000","last_clean":"2024-04-24T05:46:06.059232+0000","last_became_active":"2024-04-24T05:45:59.211908+0000","last_became_peered":"2024-04-24T05:45:59.211908+0000","last_unstale":"2024-04-24T05:46:06.059232+0000","last_undegraded":"2024-04-24T05:46:06.059232+0000","last_fullsized":"2024-04-24T05:46:06.059232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060243+0000","last_change":"2024-04-24T05:46:04.529148+0000","last_active":"2024-04-24T05:46:06.060243+0000","last_peered":"2024-04-24T05:46:06.060243+0000","last_clean":"2024-04-24T05:46:06.060243+0000","last_became_active":"2024-04-24T05:46:04.524775+0000","last_became_peered":"2024-04-24T05:46:04.524775+0000","last_unstale":"2024-04-24T05:46:06.060243+0000","last_undegraded":"2024-04-24T05:46:06.060243+0000","last_fullsized":"2024-04-24T05:46:06.060243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:58:44.607716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063332+0000","last_change":"2024-04-24T05:46:04.524596+0000","last_active":"2024-04-24T05:46:06.063332+0000","last_peered":"2024-04-24T05:46:06.063332+0000","last_clean":"2024-04-24T05:46:06.063332+0000","last_became_active":"2024-04-24T05:46:04.524013+0000","last_became_peered":"2024-04-24T05:46:04.524013+0000","last_unstale":"2024-04-24T05:46:06.063332+0000","last_undegraded":"2024-04-24T05:46:06.063332+0000","last_fullsized":"2024-04-24T05:46:06.063332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:45.991179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606337+0000","last_change":"2024-04-24T05:46:00.222945+0000","last_active":"2024-04-24T05:46:05.606337+0000","last_peered":"2024-04-24T05:46:05.606337+0000","last_clean":"2024-04-24T05:46:05.606337+0000","last_became_active":"2024-04-24T05:46:00.222698+0000","last_became_peered":"2024-04-24T05:46:00.222698+0000","last_unstale":"2024-04-24T05:46:05.606337+0000","last_undegraded":"2024-04-24T05:46:05.606337+0000","last_fullsized":"2024-04-24T05:46:05.606337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251487+0000","last_change":"2024-04-24T05:45:59.211291+0000","last_active":"2024-04-24T05:46:05.251487+0000","last_peered":"2024-04-24T05:46:05.251487+0000","last_clean":"2024-04-24T05:46:05.251487+0000","last_became_active":"2024-04-24T05:45:59.209448+0000","last_became_peered":"2024-04-24T05:45:59.209448+0000","last_unstale":"2024-04-24T05:46:05.251487+0000","last_undegraded":"2024-04-24T05:46:05.251487+0000","last_fullsized":"2024-04-24T05:46:05.251487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058917+0000","last_change":"2024-04-24T05:46:04.274043+0000","last_active":"2024-04-24T05:46:06.058917+0000","last_peered":"2024-04-24T05:46:06.058917+0000","last_clean":"2024-04-24T05:46:06.058917+0000","last_became_active":"2024-04-24T05:46:04.273504+0000","last_became_peered":"2024-04-24T05:46:04.273504+0000","last_unstale":"2024-04-24T05:46:06.058917+0000","last_undegraded":"2024-04-24T05:46:06.058917+0000","last_fullsized":"2024-04-24T05:46:06.058917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:55:12.536973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059394+0000","last_change":"2024-04-24T05:46:00.216386+0000","last_active":"2024-04-24T05:46:06.059394+0000","last_peered":"2024-04-24T05:46:06.059394+0000","last_clean":"2024-04-24T05:46:06.059394+0000","last_became_active":"2024-04-24T05:46:00.216078+0000","last_became_peered":"2024-04-24T05:46:00.216078+0000","last_unstale":"2024-04-24T05:46:06.059394+0000","last_undegraded":"2024-04-24T05:46:06.059394+0000","last_fullsized":"2024-04-24T05:46:06.059394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060995+0000","last_change":"2024-04-24T05:45:59.206266+0000","last_active":"2024-04-24T05:46:06.060995+0000","last_peered":"2024-04-24T05:46:06.060995+0000","last_clean":"2024-04-24T05:46:06.060995+0000","last_became_active":"2024-04-24T05:45:59.205881+0000","last_became_peered":"2024-04-24T05:45:59.205881+0000","last_unstale":"2024-04-24T05:46:06.060995+0000","last_undegraded":"2024-04-24T05:46:06.060995+0000","last_fullsized":"2024-04-24T05:46:06.060995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:53.435411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058603+0000","last_change":"2024-04-24T05:45:59.217059+0000","last_active":"2024-04-24T05:46:06.058603+0000","last_peered":"2024-04-24T05:46:06.058603+0000","last_clean":"2024-04-24T05:46:06.058603+0000","last_became_active":"2024-04-24T05:45:59.216742+0000","last_became_peered":"2024-04-24T05:45:59.216742+0000","last_unstale":"2024-04-24T05:46:06.058603+0000","last_undegraded":"2024-04-24T05:46:06.058603+0000","last_fullsized":"2024-04-24T05:46:06.058603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062704+0000","last_change":"2024-04-24T05:46:00.215564+0000","last_active":"2024-04-24T05:46:06.062704+0000","last_peered":"2024-04-24T05:46:06.062704+0000","last_clean":"2024-04-24T05:46:06.062704+0000","last_became_active":"2024-04-24T05:46:00.214442+0000","last_became_peered":"2024-04-24T05:46:00.214442+0000","last_unstale":"2024-04-24T05:46:06.062704+0000","last_undegraded":"2024-04-24T05:46:06.062704+0000","last_fullsized":"2024-04-24T05:46:06.062704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059540+0000","last_change":"2024-04-24T05:46:04.272081+0000","last_active":"2024-04-24T05:46:06.059540+0000","last_peered":"2024-04-24T05:46:06.059540+0000","last_clean":"2024-04-24T05:46:06.059540+0000","last_became_active":"2024-04-24T05:46:04.269391+0000","last_became_peered":"2024-04-24T05:46:04.269391+0000","last_unstale":"2024-04-24T05:46:06.059540+0000","last_undegraded":"2024-04-24T05:46:06.059540+0000","last_fullsized":"2024-04-24T05:46:06.059540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:24.966843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061465+0000","last_change":"2024-04-24T05:45:59.210758+0000","last_active":"2024-04-24T05:46:06.061465+0000","last_peered":"2024-04-24T05:46:06.061465+0000","last_clean":"2024-04-24T05:46:06.061465+0000","last_became_active":"2024-04-24T05:45:59.210533+0000","last_became_peered":"2024-04-24T05:45:59.210533+0000","last_unstale":"2024-04-24T05:46:06.061465+0000","last_undegraded":"2024-04-24T05:46:06.061465+0000","last_fullsized":"2024-04-24T05:46:06.061465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:11.561126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.247851+0000","last_change":"2024-04-24T05:46:00.211972+0000","last_active":"2024-04-24T05:46:05.247851+0000","last_peered":"2024-04-24T05:46:05.247851+0000","last_clean":"2024-04-24T05:46:05.247851+0000","last_became_active":"2024-04-24T05:46:00.210764+0000","last_became_peered":"2024-04-24T05:46:00.210764+0000","last_unstale":"2024-04-24T05:46:05.247851+0000","last_undegraded":"2024-04-24T05:46:05.247851+0000","last_fullsized":"2024-04-24T05:46:05.247851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:32.173672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062871+0000","last_change":"2024-04-24T05:46:04.271064+0000","last_active":"2024-04-24T05:46:06.062871+0000","last_peered":"2024-04-24T05:46:06.062871+0000","last_clean":"2024-04-24T05:46:06.062871+0000","last_became_active":"2024-04-24T05:46:04.270662+0000","last_became_peered":"2024-04-24T05:46:04.270662+0000","last_unstale":"2024-04-24T05:46:06.062871+0000","last_undegraded":"2024-04-24T05:46:06.062871+0000","last_fullsized":"2024-04-24T05:46:06.062871+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:18.795481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058290+0000","last_change":"2024-04-24T05:46:00.221213+0000","last_active":"2024-04-24T05:46:06.058290+0000","last_peered":"2024-04-24T05:46:06.058290+0000","last_clean":"2024-04-24T05:46:06.058290+0000","last_became_active":"2024-04-24T05:46:00.220647+0000","last_became_peered":"2024-04-24T05:46:00.220647+0000","last_unstale":"2024-04-24T05:46:06.058290+0000","last_undegraded":"2024-04-24T05:46:06.058290+0000","last_fullsized":"2024-04-24T05:46:06.058290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058501+0000","last_change":"2024-04-24T05:46:00.199902+0000","last_active":"2024-04-24T05:46:06.058501+0000","last_peered":"2024-04-24T05:46:06.058501+0000","last_clean":"2024-04-24T05:46:06.058501+0000","last_became_active":"2024-04-24T05:46:00.199314+0000","last_became_peered":"2024-04-24T05:46:00.199314+0000","last_unstale":"2024-04-24T05:46:06.058501+0000","last_undegraded":"2024-04-24T05:46:06.058501+0000","last_fullsized":"2024-04-24T05:46:06.058501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:27.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251956+0000","last_change":"2024-04-24T05:46:00.212233+0000","last_active":"2024-04-24T05:46:05.251956+0000","last_peered":"2024-04-24T05:46:05.251956+0000","last_clean":"2024-04-24T05:46:05.251956+0000","last_became_active":"2024-04-24T05:46:00.209914+0000","last_became_peered":"2024-04-24T05:46:00.209914+0000","last_unstale":"2024-04-24T05:46:05.251956+0000","last_undegraded":"2024-04-24T05:46:05.251956+0000","last_fullsized":"2024-04-24T05:46:05.251956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:45.039541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062577+0000","last_change":"2024-04-24T05:46:00.216833+0000","last_active":"2024-04-24T05:46:06.062577+0000","last_peered":"2024-04-24T05:46:06.062577+0000","last_clean":"2024-04-24T05:46:06.062577+0000","last_became_active":"2024-04-24T05:46:00.216494+0000","last_became_peered":"2024-04-24T05:46:00.216494+0000","last_unstale":"2024-04-24T05:46:06.062577+0000","last_undegraded":"2024-04-24T05:46:06.062577+0000","last_fullsized":"2024-04-24T05:46:06.062577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062178+0000","last_change":"2024-04-24T05:46:00.213642+0000","last_active":"2024-04-24T05:46:06.062178+0000","last_peered":"2024-04-24T05:46:06.062178+0000","last_clean":"2024-04-24T05:46:06.062178+0000","last_became_active":"2024-04-24T05:46:00.210164+0000","last_became_peered":"2024-04-24T05:46:00.210164+0000","last_unstale":"2024-04-24T05:46:06.062178+0000","last_undegraded":"2024-04-24T05:46:06.062178+0000","last_fullsized":"2024-04-24T05:46:06.062178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:41.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607788+0000","last_change":"2024-04-24T05:46:00.210082+0000","last_active":"2024-04-24T05:46:05.607788+0000","last_peered":"2024-04-24T05:46:05.607788+0000","last_clean":"2024-04-24T05:46:05.607788+0000","last_became_active":"2024-04-24T05:46:00.209799+0000","last_became_peered":"2024-04-24T05:46:00.209799+0000","last_unstale":"2024-04-24T05:46:05.607788+0000","last_undegraded":"2024-04-24T05:46:05.607788+0000","last_fullsized":"2024-04-24T05:46:05.607788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059611+0000","last_change":"2024-04-24T05:46:00.219304+0000","last_active":"2024-04-24T05:46:06.059611+0000","last_peered":"2024-04-24T05:46:06.059611+0000","last_clean":"2024-04-24T05:46:06.059611+0000","last_became_active":"2024-04-24T05:46:00.218953+0000","last_became_peered":"2024-04-24T05:46:00.218953+0000","last_unstale":"2024-04-24T05:46:06.059611+0000","last_undegraded":"2024-04-24T05:46:06.059611+0000","last_fullsized":"2024-04-24T05:46:06.059611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:41.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063234+0000","last_change":"2024-04-24T05:46:00.217971+0000","last_active":"2024-04-24T05:46:06.063234+0000","last_peered":"2024-04-24T05:46:06.063234+0000","last_clean":"2024-04-24T05:46:06.063234+0000","last_became_active":"2024-04-24T05:46:00.217789+0000","last_became_peered":"2024-04-24T05:46:00.217789+0000","last_unstale":"2024-04-24T05:46:06.063234+0000","last_undegraded":"2024-04-24T05:46:06.063234+0000","last_fullsized":"2024-04-24T05:46:06.063234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:38.107685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251295+0000","last_change":"2024-04-24T05:46:00.212887+0000","last_active":"2024-04-24T05:46:05.251295+0000","last_peered":"2024-04-24T05:46:05.251295+0000","last_clean":"2024-04-24T05:46:05.251295+0000","last_became_active":"2024-04-24T05:46:00.212550+0000","last_became_peered":"2024-04-24T05:46:00.212550+0000","last_unstale":"2024-04-24T05:46:05.251295+0000","last_undegraded":"2024-04-24T05:46:05.251295+0000","last_fullsized":"2024-04-24T05:46:05.251295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:25.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060618+0000","last_change":"2024-04-24T05:46:00.219721+0000","last_active":"2024-04-24T05:46:06.060618+0000","last_peered":"2024-04-24T05:46:06.060618+0000","last_clean":"2024-04-24T05:46:06.060618+0000","last_became_active":"2024-04-24T05:46:00.219429+0000","last_became_peered":"2024-04-24T05:46:00.219429+0000","last_unstale":"2024-04-24T05:46:06.060618+0000","last_undegraded":"2024-04-24T05:46:06.060618+0000","last_fullsized":"2024-04-24T05:46:06.060618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:27.080921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251686+0000","last_change":"2024-04-24T05:46:00.215082+0000","last_active":"2024-04-24T05:46:05.251686+0000","last_peered":"2024-04-24T05:46:05.251686+0000","last_clean":"2024-04-24T05:46:05.251686+0000","last_became_active":"2024-04-24T05:46:00.214719+0000","last_became_peered":"2024-04-24T05:46:00.214719+0000","last_unstale":"2024-04-24T05:46:05.251686+0000","last_undegraded":"2024-04-24T05:46:05.251686+0000","last_fullsized":"2024-04-24T05:46:05.251686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:42.688050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061826+0000","last_change":"2024-04-24T05:46:00.204113+0000","last_active":"2024-04-24T05:46:06.061826+0000","last_peered":"2024-04-24T05:46:06.061826+0000","last_clean":"2024-04-24T05:46:06.061826+0000","last_became_active":"2024-04-24T05:46:00.203764+0000","last_became_peered":"2024-04-24T05:46:00.203764+0000","last_unstale":"2024-04-24T05:46:06.061826+0000","last_undegraded":"2024-04-24T05:46:06.061826+0000","last_fullsized":"2024-04-24T05:46:06.061826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059326+0000","last_change":"2024-04-24T05:46:00.223248+0000","last_active":"2024-04-24T05:46:06.059326+0000","last_peered":"2024-04-24T05:46:06.059326+0000","last_clean":"2024-04-24T05:46:06.059326+0000","last_became_active":"2024-04-24T05:46:00.222366+0000","last_became_peered":"2024-04-24T05:46:00.222366+0000","last_unstale":"2024-04-24T05:46:06.059326+0000","last_undegraded":"2024-04-24T05:46:06.059326+0000","last_fullsized":"2024-04-24T05:46:06.059326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558036+0000","last_change":"2024-04-24T05:46:00.217085+0000","last_active":"2024-04-24T05:46:05.558036+0000","last_peered":"2024-04-24T05:46:05.558036+0000","last_clean":"2024-04-24T05:46:05.558036+0000","last_became_active":"2024-04-24T05:46:00.216799+0000","last_became_peered":"2024-04-24T05:46:00.216799+0000","last_unstale":"2024-04-24T05:46:05.558036+0000","last_undegraded":"2024-04-24T05:46:05.558036+0000","last_fullsized":"2024-04-24T05:46:05.558036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:27.050101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059180+0000","last_change":"2024-04-24T05:46:00.221238+0000","last_active":"2024-04-24T05:46:06.059180+0000","last_peered":"2024-04-24T05:46:06.059180+0000","last_clean":"2024-04-24T05:46:06.059180+0000","last_became_active":"2024-04-24T05:46:00.220945+0000","last_became_peered":"2024-04-24T05:46:00.220945+0000","last_unstale":"2024-04-24T05:46:06.059180+0000","last_undegraded":"2024-04-24T05:46:06.059180+0000","last_fullsized":"2024-04-24T05:46:06.059180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:21:57.974456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.254113+0000","last_change":"2024-04-24T05:46:00.206311+0000","last_active":"2024-04-24T05:46:05.254113+0000","last_peered":"2024-04-24T05:46:05.254113+0000","last_clean":"2024-04-24T05:46:05.254113+0000","last_became_active":"2024-04-24T05:46:00.205935+0000","last_became_peered":"2024-04-24T05:46:00.205935+0000","last_unstale":"2024-04-24T05:46:05.254113+0000","last_undegraded":"2024-04-24T05:46:05.254113+0000","last_fullsized":"2024-04-24T05:46:05.254113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:09.200350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251128+0000","last_change":"2024-04-24T05:46:00.202613+0000","last_active":"2024-04-24T05:46:05.251128+0000","last_peered":"2024-04-24T05:46:05.251128+0000","last_clean":"2024-04-24T05:46:05.251128+0000","last_became_active":"2024-04-24T05:46:00.202286+0000","last_became_peered":"2024-04-24T05:46:00.202286+0000","last_unstale":"2024-04-24T05:46:05.251128+0000","last_undegraded":"2024-04-24T05:46:05.251128+0000","last_fullsized":"2024-04-24T05:46:05.251128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606554+0000","last_change":"2024-04-24T05:46:00.222335+0000","last_active":"2024-04-24T05:46:05.606554+0000","last_peered":"2024-04-24T05:46:05.606554+0000","last_clean":"2024-04-24T05:46:05.606554+0000","last_became_active":"2024-04-24T05:46:00.222066+0000","last_became_peered":"2024-04-24T05:46:00.222066+0000","last_unstale":"2024-04-24T05:46:05.606554+0000","last_undegraded":"2024-04-24T05:46:05.606554+0000","last_fullsized":"2024-04-24T05:46:05.606554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060597+0000","last_change":"2024-04-24T05:46:00.213735+0000","last_active":"2024-04-24T05:46:06.060597+0000","last_peered":"2024-04-24T05:46:06.060597+0000","last_clean":"2024-04-24T05:46:06.060597+0000","last_became_active":"2024-04-24T05:46:00.213390+0000","last_became_peered":"2024-04-24T05:46:00.213390+0000","last_unstale":"2024-04-24T05:46:06.060597+0000","last_undegraded":"2024-04-24T05:46:06.060597+0000","last_fullsized":"2024-04-24T05:46:06.060597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:20.914341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061232+0000","last_change":"2024-04-24T05:46:00.219073+0000","last_active":"2024-04-24T05:46:06.061232+0000","last_peered":"2024-04-24T05:46:06.061232+0000","last_clean":"2024-04-24T05:46:06.061232+0000","last_became_active":"2024-04-24T05:46:00.218577+0000","last_became_peered":"2024-04-24T05:46:00.218577+0000","last_unstale":"2024-04-24T05:46:06.061232+0000","last_undegraded":"2024-04-24T05:46:06.061232+0000","last_fullsized":"2024-04-24T05:46:06.061232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248861+0000","last_change":"2024-04-24T05:46:00.219097+0000","last_active":"2024-04-24T05:46:05.248861+0000","last_peered":"2024-04-24T05:46:05.248861+0000","last_clean":"2024-04-24T05:46:05.248861+0000","last_became_active":"2024-04-24T05:46:00.218772+0000","last_became_peered":"2024-04-24T05:46:00.218772+0000","last_unstale":"2024-04-24T05:46:05.248861+0000","last_undegraded":"2024-04-24T05:46:05.248861+0000","last_fullsized":"2024-04-24T05:46:05.248861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:35.088347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059066+0000","last_change":"2024-04-24T05:46:00.221300+0000","last_active":"2024-04-24T05:46:06.059066+0000","last_peered":"2024-04-24T05:46:06.059066+0000","last_clean":"2024-04-24T05:46:06.059066+0000","last_became_active":"2024-04-24T05:46:00.220921+0000","last_became_peered":"2024-04-24T05:46:00.220921+0000","last_unstale":"2024-04-24T05:46:06.059066+0000","last_undegraded":"2024-04-24T05:46:06.059066+0000","last_fullsized":"2024-04-24T05:46:06.059066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059208+0000","last_change":"2024-04-24T05:46:00.221735+0000","last_active":"2024-04-24T05:46:06.059208+0000","last_peered":"2024-04-24T05:46:06.059208+0000","last_clean":"2024-04-24T05:46:06.059208+0000","last_became_active":"2024-04-24T05:46:00.220859+0000","last_became_peered":"2024-04-24T05:46:00.220859+0000","last_unstale":"2024-04-24T05:46:06.059208+0000","last_undegraded":"2024-04-24T05:46:06.059208+0000","last_fullsized":"2024-04-24T05:46:06.059208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:25.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558492+0000","last_change":"2024-04-24T05:46:00.227925+0000","last_active":"2024-04-24T05:46:05.558492+0000","last_peered":"2024-04-24T05:46:05.558492+0000","last_clean":"2024-04-24T05:46:05.558492+0000","last_became_active":"2024-04-24T05:46:00.227553+0000","last_became_peered":"2024-04-24T05:46:00.227553+0000","last_unstale":"2024-04-24T05:46:05.558492+0000","last_undegraded":"2024-04-24T05:46:05.558492+0000","last_fullsized":"2024-04-24T05:46:05.558492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:09.200350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058874+0000","last_change":"2024-04-24T05:46:00.225244+0000","last_active":"2024-04-24T05:46:06.058874+0000","last_peered":"2024-04-24T05:46:06.058874+0000","last_clean":"2024-04-24T05:46:06.058874+0000","last_became_active":"2024-04-24T05:46:00.224935+0000","last_became_peered":"2024-04-24T05:46:00.224935+0000","last_unstale":"2024-04-24T05:46:06.058874+0000","last_undegraded":"2024-04-24T05:46:06.058874+0000","last_fullsized":"2024-04-24T05:46:06.058874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557220+0000","last_change":"2024-04-24T05:46:00.225238+0000","last_active":"2024-04-24T05:46:05.557220+0000","last_peered":"2024-04-24T05:46:05.557220+0000","last_clean":"2024-04-24T05:46:05.557220+0000","last_became_active":"2024-04-24T05:46:00.224903+0000","last_became_peered":"2024-04-24T05:46:00.224903+0000","last_unstale":"2024-04-24T05:46:05.557220+0000","last_undegraded":"2024-04-24T05:46:05.557220+0000","last_fullsized":"2024-04-24T05:46:05.557220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:42.688050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251674+0000","last_change":"2024-04-24T05:46:00.206725+0000","last_active":"2024-04-24T05:46:05.251674+0000","last_peered":"2024-04-24T05:46:05.251674+0000","last_clean":"2024-04-24T05:46:05.251674+0000","last_became_active":"2024-04-24T05:46:00.206266+0000","last_became_peered":"2024-04-24T05:46:00.206266+0000","last_unstale":"2024-04-24T05:46:05.251674+0000","last_undegraded":"2024-04-24T05:46:05.251674+0000","last_fullsized":"2024-04-24T05:46:05.251674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:41.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557088+0000","last_change":"2024-04-24T05:46:00.219593+0000","last_active":"2024-04-24T05:46:05.557088+0000","last_peered":"2024-04-24T05:46:05.557088+0000","last_clean":"2024-04-24T05:46:05.557088+0000","last_became_active":"2024-04-24T05:46:00.219250+0000","last_became_peered":"2024-04-24T05:46:00.219250+0000","last_unstale":"2024-04-24T05:46:05.557088+0000","last_undegraded":"2024-04-24T05:46:05.557088+0000","last_fullsized":"2024-04-24T05:46:05.557088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:45.039541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252270+0000","last_change":"2024-04-24T05:46:00.213476+0000","last_active":"2024-04-24T05:46:05.252270+0000","last_peered":"2024-04-24T05:46:05.252270+0000","last_clean":"2024-04-24T05:46:05.252270+0000","last_became_active":"2024-04-24T05:46:00.213065+0000","last_became_peered":"2024-04-24T05:46:00.213065+0000","last_unstale":"2024-04-24T05:46:05.252270+0000","last_undegraded":"2024-04-24T05:46:05.252270+0000","last_fullsized":"2024-04-24T05:46:05.252270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:38.107685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063065+0000","last_change":"2024-04-24T05:46:00.221236+0000","last_active":"2024-04-24T05:46:06.063065+0000","last_peered":"2024-04-24T05:46:06.063065+0000","last_clean":"2024-04-24T05:46:06.063065+0000","last_became_active":"2024-04-24T05:46:00.220917+0000","last_became_peered":"2024-04-24T05:46:00.220917+0000","last_unstale":"2024-04-24T05:46:06.063065+0000","last_undegraded":"2024-04-24T05:46:06.063065+0000","last_fullsized":"2024-04-24T05:46:06.063065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:38.445039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608230+0000","last_change":"2024-04-24T05:46:00.219494+0000","last_active":"2024-04-24T05:46:05.608230+0000","last_peered":"2024-04-24T05:46:05.608230+0000","last_clean":"2024-04-24T05:46:05.608230+0000","last_became_active":"2024-04-24T05:46:00.219219+0000","last_became_peered":"2024-04-24T05:46:00.219219+0000","last_unstale":"2024-04-24T05:46:05.608230+0000","last_undegraded":"2024-04-24T05:46:05.608230+0000","last_fullsized":"2024-04-24T05:46:05.608230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558194+0000","last_change":"2024-04-24T05:46:00.221425+0000","last_active":"2024-04-24T05:46:05.558194+0000","last_peered":"2024-04-24T05:46:05.558194+0000","last_clean":"2024-04-24T05:46:05.558194+0000","last_became_active":"2024-04-24T05:46:00.219794+0000","last_became_peered":"2024-04-24T05:46:00.219794+0000","last_unstale":"2024-04-24T05:46:05.558194+0000","last_undegraded":"2024-04-24T05:46:05.558194+0000","last_fullsized":"2024-04-24T05:46:05.558194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:55:08.484471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":696263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":109155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":105439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":109155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":105313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":105473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":696319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T05:46:22.192 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T05:46:22.192 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T05:46:22.192 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-24T05:46:22.192 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-24T05:46:22.193 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-24T05:46:22.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.426+0000 7fc987200640 1 Processor -- start 2024-04-24T05:46:22.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.426+0000 7fc987200640 1 -- start start 2024-04-24T05:46:22.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.428+0000 7fc987200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9880cdf70 0x7fc9880b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:22.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.428+0000 7fc987200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9880b3000 con 0x7fc98806e2f0 2024-04-24T05:46:22.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.428+0000 7fc987200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc98806d0c0 con 0x7fc98815b5c0 2024-04-24T05:46:22.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.428+0000 7fc987200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc9880ae230 con 0x7fc9880cdf70 2024-04-24T05:46:22.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.428+0000 7fc985400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc98806e2f0 0x7fc9880b34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35496/0 (socket says 172.21.15.123:35496) 2024-04-24T05:46:22.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.428+0000 7fc985400640 1 -- 172.21.15.123:0/1132222879 learned_addr learned my addr 172.21.15.123:0/1132222879 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:22.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc986800640 1 --2- 172.21.15.123:0/1132222879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9880cdf70 0x7fc9880b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:22.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 -- 172.21.15.123:0/1132222879 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2939807255 0 0) 0x7fc9880b3000 con 0x7fc98806e2f0 2024-04-24T05:46:22.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 -- 172.21.15.123:0/1132222879 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc96c005730 con 0x7fc98806e2f0 2024-04-24T05:46:22.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 -- 172.21.15.123:0/1132222879 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1751901755 0 0) 0x7fc96c005730 con 0x7fc98806e2f0 2024-04-24T05:46:22.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 -- 172.21.15.123:0/1132222879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9880cdf70 msgr2=0x7fc9880b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:22.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 --2- 172.21.15.123:0/1132222879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9880cdf70 0x7fc9880b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 -- 172.21.15.123:0/1132222879 >> v1:172.21.15.155:6789/0 conn(0x7fc98815b5c0 legacy=0x7fc98806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:22.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 -- 172.21.15.123:0/1132222879 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9880adf50 con 0x7fc98806e2f0 2024-04-24T05:46:22.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 -- 172.21.15.123:0/1132222879 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7fc974001670 con 0x7fc98806e2f0 2024-04-24T05:46:22.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 -- 172.21.15.123:0/1132222879 >> v1:172.21.15.123:6789/0 conn(0x7fc98806e2f0 legacy=0x7fc9880b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:22.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 --2- 172.21.15.123:0/1132222879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc96c005bf0 0x7fc96c006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:22.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 --2- 172.21.15.123:0/1132222879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9880cdf70 0x7fc96c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:22.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 --2- 172.21.15.123:0/1132222879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c006f50 0x7fc96c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:22.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 -- 172.21.15.123:0/1132222879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc9880ae230 con 0x7fc96c005bf0 2024-04-24T05:46:22.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 -- 172.21.15.123:0/1132222879 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c00dc80 con 0x7fc96c006f50 2024-04-24T05:46:22.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.429+0000 7fc984a00640 1 -- 172.21.15.123:0/1132222879 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc96c00de80 con 0x7fc9880cdf70 2024-04-24T05:46:22.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.430+0000 7fc985e00640 1 --2- 172.21.15.123:0/1132222879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc96c005bf0 0x7fc96c006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:22.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.430+0000 7fc985400640 1 --2- 172.21.15.123:0/1132222879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9880cdf70 0x7fc96c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:22.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.430+0000 7fc986800640 1 --2- 172.21.15.123:0/1132222879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c006f50 0x7fc96c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:22.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.430+0000 7fc985e00640 1 -- 172.21.15.123:0/1132222879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9880cdf70 msgr2=0x7fc96c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:22.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.430+0000 7fc985e00640 1 --2- 172.21.15.123:0/1132222879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9880cdf70 0x7fc96c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.430+0000 7fc985e00640 1 -- 172.21.15.123:0/1132222879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c006f50 msgr2=0x7fc96c0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:22.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.430+0000 7fc985e00640 1 --2- 172.21.15.123:0/1132222879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c006f50 0x7fc96c0073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.430+0000 7fc985e00640 1 -- 172.21.15.123:0/1132222879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c005730 con 0x7fc96c005bf0 2024-04-24T05:46:22.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.430+0000 7fc986800640 1 --2- 172.21.15.123:0/1132222879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c006f50 0x7fc96c0073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:22.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.431+0000 7fc985e00640 1 --2- 172.21.15.123:0/1132222879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc96c005bf0 0x7fc96c006010 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc978001720 tx=0x7fc96c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e353ea96a7508e52 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:22.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.431+0000 7fc984a00640 1 -- 172.21.15.123:0/1132222879 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978001370 con 0x7fc96c005bf0 2024-04-24T05:46:22.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.431+0000 7fc984a00640 1 -- 172.21.15.123:0/1132222879 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9780052d0 con 0x7fc96c005bf0 2024-04-24T05:46:22.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.431+0000 7fc984a00640 1 -- 172.21.15.123:0/1132222879 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97802abc0 con 0x7fc96c005bf0 2024-04-24T05:46:22.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.432+0000 7fc987200640 1 -- 172.21.15.123:0/1132222879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc96c005bf0 msgr2=0x7fc96c006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:22.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.432+0000 7fc987200640 1 --2- 172.21.15.123:0/1132222879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc96c005bf0 0x7fc96c006010 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc978001720 tx=0x7fc96c007a40 comp rx=0 tx=0).stop 2024-04-24T05:46:22.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.432+0000 7fc986800640 1 -- 172.21.15.123:0/1132222879 reap_dead start 2024-04-24T05:46:22.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.432+0000 7fc987200640 1 -- 172.21.15.123:0/1132222879 shutdown_connections 2024-04-24T05:46:22.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.432+0000 7fc987200640 1 -- 172.21.15.123:0/1132222879 >> 172.21.15.123:0/1132222879 conn(0x7fc9880d0070 msgr2=0x7fc9880d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:22.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.432+0000 7fc987200640 1 -- 172.21.15.123:0/1132222879 shutdown_connections 2024-04-24T05:46:22.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.432+0000 7fc987200640 1 -- 172.21.15.123:0/1132222879 wait complete. 2024-04-24T05:46:22.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.434+0000 7fc987200640 1 Processor -- start 2024-04-24T05:46:22.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.434+0000 7fc987200640 1 -- start start 2024-04-24T05:46:22.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.435+0000 7fc987200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc96c005bf0 0x7fc98814b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:22.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.435+0000 7fc987200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc98809d140 0x7fc98809d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:22.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.435+0000 7fc987200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9880b33b0 0x7fc9880b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:22.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.435+0000 7fc987200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc98806cd20 con 0x7fc98809d140 2024-04-24T05:46:22.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.435+0000 7fc987200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc9880ae510 con 0x7fc9880b33b0 2024-04-24T05:46:22.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.435+0000 7fc987200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc98806cea0 con 0x7fc96c005bf0 2024-04-24T05:46:22.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.435+0000 7fc985400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc98809d140 0x7fc98809d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:22.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.435+0000 7fc985400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc98809d140 0x7fc98809d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56376/0 (socket says 172.21.15.123:56376) 2024-04-24T05:46:22.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.435+0000 7fc985400640 1 -- 172.21.15.123:0/4282395645 learned_addr learned my addr 172.21.15.123:0/4282395645 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:22.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.435+0000 7fc985e00640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc96c005bf0 0x7fc98814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:22.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.435+0000 7fc985400640 1 -- 172.21.15.123:0/4282395645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc96c005bf0 msgr2=0x7fc98814b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:22.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.435+0000 7fc986800640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9880b33b0 0x7fc9880b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:22.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.436+0000 7fc985400640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc96c005bf0 0x7fc98814b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.436+0000 7fc985400640 1 -- 172.21.15.123:0/4282395645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9880b33b0 msgr2=0x7fc9880b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:22.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.436+0000 7fc985400640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9880b33b0 0x7fc9880b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.436+0000 7fc985400640 1 -- 172.21.15.123:0/4282395645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98814ba60 con 0x7fc98809d140 2024-04-24T05:46:22.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.436+0000 7fc985e00640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc96c005bf0 0x7fc98814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:22.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.436+0000 7fc985400640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc98809d140 0x7fc98809d5c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc97400e830 tx=0x7fc97400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:22.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.436+0000 7fc97e000640 1 -- 172.21.15.123:0/4282395645 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97400ee20 con 0x7fc98809d140 2024-04-24T05:46:22.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.436+0000 7fc97e000640 1 -- 172.21.15.123:0/4282395645 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc974018070 con 0x7fc98809d140 2024-04-24T05:46:22.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.436+0000 7fc987200640 1 -- 172.21.15.123:0/4282395645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9881f3660 con 0x7fc98809d140 2024-04-24T05:46:22.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.436+0000 7fc97e000640 1 -- 172.21.15.123:0/4282395645 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97400caf0 con 0x7fc98809d140 2024-04-24T05:46:22.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.436+0000 7fc987200640 1 -- 172.21.15.123:0/4282395645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9881f3c00 con 0x7fc98809d140 2024-04-24T05:46:22.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.438+0000 7fc987200640 1 -- 172.21.15.123:0/4282395645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc988075b70 con 0x7fc98809d140 2024-04-24T05:46:22.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.438+0000 7fc97e000640 1 -- 172.21.15.123:0/4282395645 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc97400cc90 con 0x7fc98809d140 2024-04-24T05:46:22.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.439+0000 7fc97e000640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc96806c920 0x7fc96806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:22.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.440+0000 7fc97e000640 1 -- 172.21.15.123:0/4282395645 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc97408b8e0 con 0x7fc98809d140 2024-04-24T05:46:22.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.440+0000 7fc985e00640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc96806c920 0x7fc96806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:22.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.440+0000 7fc985e00640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc96806c920 0x7fc96806ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc978032000 tx=0x7fc97802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:22.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.443+0000 7fc97e000640 1 -- 172.21.15.123:0/4282395645 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc974059a60 con 0x7fc98809d140 2024-04-24T05:46:22.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.693+0000 7fc987200640 1 -- 172.21.15.123:0/4282395645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc98806f2e0 con 0x7fc98809d140 2024-04-24T05:46:22.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.699+0000 7fc97e000640 1 -- 172.21.15.123:0/4282395645 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174143 (secure 0 0 0) 0x7fc974059360 con 0x7fc98809d140 2024-04-24T05:46:22.703 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:22.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.704+0000 7fc987200640 1 -- 172.21.15.123:0/4282395645 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc96806c920 msgr2=0x7fc96806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:22.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.704+0000 7fc987200640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc96806c920 0x7fc96806ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc978032000 tx=0x7fc97802e000 comp rx=0 tx=0).stop 2024-04-24T05:46:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.704+0000 7fc987200640 1 -- 172.21.15.123:0/4282395645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc98809d140 msgr2=0x7fc98809d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.704+0000 7fc987200640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc98809d140 0x7fc98809d5c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc97400e830 tx=0x7fc97400ecf0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.704+0000 7fc987200640 1 -- 172.21.15.123:0/4282395645 shutdown_connections 2024-04-24T05:46:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.704+0000 7fc987200640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc96806c920 0x7fc96806ede0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.704+0000 7fc987200640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9880b33b0 0x7fc9880b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.704+0000 7fc987200640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc98809d140 0x7fc98809d5c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.704+0000 7fc987200640 1 --2- 172.21.15.123:0/4282395645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc96c005bf0 0x7fc98814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.704+0000 7fc987200640 1 -- 172.21.15.123:0/4282395645 >> 172.21.15.123:0/4282395645 conn(0x7fc9880d0070 msgr2=0x7fc98806ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.704+0000 7fc987200640 1 -- 172.21.15.123:0/4282395645 shutdown_connections 2024-04-24T05:46:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.704+0000 7fc987200640 1 -- 172.21.15.123:0/4282395645 wait complete. 2024-04-24T05:46:22.725 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1503718323},{"type":"v1","addr":"172.21.15.123:6833","nonce":1503718323}]},"active_addr":"172.21.15.123:6833/1503718323","active_change":"2024-04-24T05:45:40.486872+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":472819266}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2997881687}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3464705502}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":1311560028}]}]} 2024-04-24T05:46:22.728 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-24T05:46:22.728 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-24T05:46:22.728 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:22.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.925+0000 7fa6fec00640 1 Processor -- start 2024-04-24T05:46:22.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.925+0000 7fa6fec00640 1 -- start start 2024-04-24T05:46:22.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.926+0000 7fa6fec00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6f806f090 0x7fa6f8072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:22.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.926+0000 7fa6fec00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa6f806f470 0x7fa6f8073500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:22.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.926+0000 7fa6fec00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6f8071cd0 con 0x7fa6f80b2990 2024-04-24T05:46:22.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.926+0000 7fa6fec00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f8071fb0 con 0x7fa6f806f470 2024-04-24T05:46:22.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.926+0000 7fa6fec00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f8072290 con 0x7fa6f806f090 2024-04-24T05:46:22.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.927+0000 7fa6ef400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa6f806f470 0x7fa6f8073500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:22.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.927+0000 7fa6ef400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa6f806f470 0x7fa6f8073500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56382/0 (socket says 172.21.15.123:56382) 2024-04-24T05:46:22.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.927+0000 7fa6fcc00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa6f80b2990 0x7fa6f8151e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35510/0 (socket says 172.21.15.123:35510) 2024-04-24T05:46:22.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.927+0000 7fa6ef400640 1 -- 172.21.15.123:0/3831989542 learned_addr learned my addr 172.21.15.123:0/3831989542 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:22.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.927+0000 7fa6f7e00640 1 --2- 172.21.15.123:0/3831989542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6f806f090 0x7fa6f8072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.927+0000 7fa6ef400640 1 -- 172.21.15.123:0/3831989542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6f806f090 msgr2=0x7fa6f8072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.927+0000 7fa6ef400640 1 --2- 172.21.15.123:0/3831989542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6f806f090 0x7fa6f8072fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.927+0000 7fa6ef400640 1 -- 172.21.15.123:0/3831989542 >> v1:172.21.15.123:6789/0 conn(0x7fa6f80b2990 legacy=0x7fa6f8151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.927+0000 7fa6ef400640 1 -- 172.21.15.123:0/3831989542 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f8072570 con 0x7fa6f806f470 2024-04-24T05:46:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.927+0000 7fa6f7400640 1 -- 172.21.15.123:0/3831989542 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3983906562 0 0) 0x7fa6f8071cd0 con 0x7fa6f80b2990 2024-04-24T05:46:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.927+0000 7fa6f7e00640 1 --2- 172.21.15.123:0/3831989542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6f806f090 0x7fa6f8072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:46:22.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.928+0000 7fa6ef400640 1 --2- 172.21.15.123:0/3831989542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f806f470 0x7fa6f8073500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009870 tx=0x7fa6e002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cb68d7a175f33f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:22.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.929+0000 7fa6f7400640 1 -- 172.21.15.123:0/3831989542 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0037070 con 0x7fa6f806f470 2024-04-24T05:46:22.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.929+0000 7fa6f7400640 1 -- 172.21.15.123:0/3831989542 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e0032440 con 0x7fa6f806f470 2024-04-24T05:46:22.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.929+0000 7fa6f7400640 1 -- 172.21.15.123:0/3831989542 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0032810 con 0x7fa6f806f470 2024-04-24T05:46:22.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.929+0000 7fa6fec00640 1 -- 172.21.15.123:0/3831989542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f806f470 msgr2=0x7fa6f8073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:22.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.929+0000 7fa6fec00640 1 --2- 172.21.15.123:0/3831989542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f806f470 0x7fa6f8073500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009870 tx=0x7fa6e002a1c0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.929+0000 7fa6fec00640 1 -- 172.21.15.123:0/3831989542 shutdown_connections 2024-04-24T05:46:22.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.929+0000 7fa6fec00640 1 --2- 172.21.15.123:0/3831989542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f806f470 0x7fa6f8073500 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.929+0000 7fa6fec00640 1 --2- 172.21.15.123:0/3831989542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6f806f090 0x7fa6f8072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.929+0000 7fa6fec00640 1 -- 172.21.15.123:0/3831989542 >> 172.21.15.123:0/3831989542 conn(0x7fa6f80b1e70 msgr2=0x7fa6f80b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:22.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.929+0000 7fa6fec00640 1 -- 172.21.15.123:0/3831989542 shutdown_connections 2024-04-24T05:46:22.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.929+0000 7fa6fec00640 1 -- 172.21.15.123:0/3831989542 wait complete. 2024-04-24T05:46:22.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.931+0000 7fa6fec00640 1 Processor -- start 2024-04-24T05:46:22.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.931+0000 7fa6fec00640 1 -- start start 2024-04-24T05:46:22.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.932+0000 7fa6fec00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6f806f090 0x7fa6f81f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:22.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.932+0000 7fa6fec00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f806f470 0x7fa6f814b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:22.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.932+0000 7fa6fec00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6f80b2990 0x7fa6f8149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:22.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.932+0000 7fa6fec00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f815c690 con 0x7fa6f806f470 2024-04-24T05:46:22.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.932+0000 7fa6fec00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f815c510 con 0x7fa6f806f090 2024-04-24T05:46:22.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.932+0000 7fa6fec00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f8072a90 con 0x7fa6f80b2990 2024-04-24T05:46:22.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.932+0000 7fa6ef400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f806f470 0x7fa6f814b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:22.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.932+0000 7fa6ef400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f806f470 0x7fa6f814b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56396/0 (socket says 172.21.15.123:56396) 2024-04-24T05:46:22.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.932+0000 7fa6ef400640 1 -- 172.21.15.123:0/3189790363 learned_addr learned my addr 172.21.15.123:0/3189790363 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:22.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.933+0000 7fa6fcc00640 1 --2- 172.21.15.123:0/3189790363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6f80b2990 0x7fa6f8149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:22.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.933+0000 7fa6ef400640 1 -- 172.21.15.123:0/3189790363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6f80b2990 msgr2=0x7fa6f8149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:22.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.933+0000 7fa6ef400640 1 --2- 172.21.15.123:0/3189790363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6f80b2990 0x7fa6f8149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.933+0000 7fa6ef400640 1 -- 172.21.15.123:0/3189790363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6f806f090 msgr2=0x7fa6f81f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:22.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.933+0000 7fa6ef400640 1 --2- 172.21.15.123:0/3189790363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6f806f090 0x7fa6f81f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:22.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.933+0000 7fa6ef400640 1 -- 172.21.15.123:0/3189790363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f814b9f0 con 0x7fa6f806f470 2024-04-24T05:46:22.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.933+0000 7fa6ef400640 1 --2- 172.21.15.123:0/3189790363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f806f470 0x7fa6f814b1a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0035040 tx=0x7fa6e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:22.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.934+0000 7fa6f4c00640 1 -- 172.21.15.123:0/3189790363 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0037040 con 0x7fa6f806f470 2024-04-24T05:46:22.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.934+0000 7fa6f4c00640 1 -- 172.21.15.123:0/3189790363 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e0004230 con 0x7fa6f806f470 2024-04-24T05:46:22.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.934+0000 7fa6f4c00640 1 -- 172.21.15.123:0/3189790363 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0003610 con 0x7fa6f806f470 2024-04-24T05:46:22.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.934+0000 7fa6fec00640 1 -- 172.21.15.123:0/3189790363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f814bcd0 con 0x7fa6f806f470 2024-04-24T05:46:22.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.934+0000 7fa6fec00640 1 -- 172.21.15.123:0/3189790363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f814c290 con 0x7fa6f806f470 2024-04-24T05:46:22.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.935+0000 7fa6fec00640 1 -- 172.21.15.123:0/3189790363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f8068a20 con 0x7fa6f806f470 2024-04-24T05:46:22.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.936+0000 7fa6f4c00640 1 -- 172.21.15.123:0/3189790363 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6e002ad10 con 0x7fa6f806f470 2024-04-24T05:46:22.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.937+0000 7fa6f4c00640 1 --2- 172.21.15.123:0/3189790363 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa6d406c8f0 0x7fa6d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:22.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.937+0000 7fa6f4c00640 1 -- 172.21.15.123:0/3189790363 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa6e00aca20 con 0x7fa6f806f470 2024-04-24T05:46:22.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.937+0000 7fa6f7e00640 1 --2- 172.21.15.123:0/3189790363 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa6d406c8f0 0x7fa6d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:22.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.938+0000 7fa6f7e00640 1 --2- 172.21.15.123:0/3189790363 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa6d406c8f0 0x7fa6d406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8006fd0 tx=0x7fa6e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:22.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:22.942+0000 7fa6f4c00640 1 -- 172.21.15.123:0/3189790363 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6e007aa70 con 0x7fa6f806f470 2024-04-24T05:46:23.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.141+0000 7fa6fec00640 1 -- 172.21.15.123:0/3189790363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6f8155bf0 con 0x7fa6f806f470 2024-04-24T05:46:23.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.144+0000 7fa6f4c00640 1 -- 172.21.15.123:0/3189790363 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7fa6e007a370 con 0x7fa6f806f470 2024-04-24T05:46:23.146 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:23.146 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:04.715857+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:23.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.146+0000 7fa6fec00640 1 -- 172.21.15.123:0/3189790363 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa6d406c8f0 msgr2=0x7fa6d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.146+0000 7fa6fec00640 1 --2- 172.21.15.123:0/3189790363 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa6d406c8f0 0x7fa6d406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8006fd0 tx=0x7fa6e8008040 comp rx=0 tx=0).stop 2024-04-24T05:46:23.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.146+0000 7fa6fec00640 1 -- 172.21.15.123:0/3189790363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f806f470 msgr2=0x7fa6f814b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.146+0000 7fa6fec00640 1 --2- 172.21.15.123:0/3189790363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f806f470 0x7fa6f814b1a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0035040 tx=0x7fa6e00046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.147+0000 7fa6fec00640 1 -- 172.21.15.123:0/3189790363 shutdown_connections 2024-04-24T05:46:23.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.147+0000 7fa6fec00640 1 --2- 172.21.15.123:0/3189790363 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa6d406c8f0 0x7fa6d406edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.147+0000 7fa6fec00640 1 --2- 172.21.15.123:0/3189790363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6f80b2990 0x7fa6f8149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.147+0000 7fa6fec00640 1 --2- 172.21.15.123:0/3189790363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f806f470 0x7fa6f814b1a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.147+0000 7fa6fec00640 1 --2- 172.21.15.123:0/3189790363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6f806f090 0x7fa6f81f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.147+0000 7fa6fec00640 1 -- 172.21.15.123:0/3189790363 >> 172.21.15.123:0/3189790363 conn(0x7fa6f80b1e70 msgr2=0x7fa6f80b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:23.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.147+0000 7fa6fec00640 1 -- 172.21.15.123:0/3189790363 shutdown_connections 2024-04-24T05:46:23.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.147+0000 7fa6fec00640 1 -- 172.21.15.123:0/3189790363 wait complete. 2024-04-24T05:46:23.167 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-24T05:46:23.167 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:23.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.367+0000 7efd06c00640 1 Processor -- start 2024-04-24T05:46:23.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.367+0000 7efd06c00640 1 -- start start 2024-04-24T05:46:23.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.369+0000 7efd06c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efd0015a2a0 0x7efd000ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.369+0000 7efd06c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd0015cdf0 con 0x7efd000ad690 2024-04-24T05:46:23.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.369+0000 7efd06c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd0015d0d0 con 0x7efd000af950 2024-04-24T05:46:23.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.369+0000 7efd06c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7efd0015a680 con 0x7efd0015a2a0 2024-04-24T05:46:23.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.369+0000 7efcffe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efd0015a2a0 0x7efd000ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.370+0000 7efcffe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efd0015a2a0 0x7efd000ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56412/0 (socket says 172.21.15.123:56412) 2024-04-24T05:46:23.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.370+0000 7efcff400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7efd000ad690 0x7efd000af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35514/0 (socket says 172.21.15.123:35514) 2024-04-24T05:46:23.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.370+0000 7efcffe00640 1 -- 172.21.15.123:0/224449587 learned_addr learned my addr 172.21.15.123:0/224449587 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.370+0000 7efcffe00640 1 -- 172.21.15.123:0/224449587 >> v1:172.21.15.155:6789/0 conn(0x7efd000af950 legacy=0x7efd0006bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.370+0000 7efcffe00640 1 -- 172.21.15.123:0/224449587 >> v1:172.21.15.123:6789/0 conn(0x7efd000ad690 legacy=0x7efd000af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.370+0000 7efcffe00640 1 -- 172.21.15.123:0/224449587 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0015d3b0 con 0x7efd0015a2a0 2024-04-24T05:46:23.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.371+0000 7efcffe00640 1 --2- 172.21.15.123:0/224449587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd0015a2a0 0x7efd000ad150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009e50 tx=0x7efcf002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6618dc6a3ed2a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.371+0000 7efcfea00640 1 -- 172.21.15.123:0/224449587 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0036070 con 0x7efd0015a2a0 2024-04-24T05:46:23.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.372+0000 7efcfea00640 1 -- 172.21.15.123:0/224449587 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf0031600 con 0x7efd0015a2a0 2024-04-24T05:46:23.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.372+0000 7efcfea00640 1 -- 172.21.15.123:0/224449587 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00319d0 con 0x7efd0015a2a0 2024-04-24T05:46:23.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.372+0000 7efd06c00640 1 -- 172.21.15.123:0/224449587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd0015a2a0 msgr2=0x7efd000ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.372+0000 7efd06c00640 1 --2- 172.21.15.123:0/224449587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd0015a2a0 0x7efd000ad150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009e50 tx=0x7efcf002a730 comp rx=0 tx=0).stop 2024-04-24T05:46:23.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.372+0000 7efd06c00640 1 -- 172.21.15.123:0/224449587 shutdown_connections 2024-04-24T05:46:23.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.372+0000 7efd06c00640 1 --2- 172.21.15.123:0/224449587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd0015a2a0 0x7efd000ad150 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.372+0000 7efd06c00640 1 -- 172.21.15.123:0/224449587 >> 172.21.15.123:0/224449587 conn(0x7efd000b1e70 msgr2=0x7efd000b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:23.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.373+0000 7efd06c00640 1 -- 172.21.15.123:0/224449587 shutdown_connections 2024-04-24T05:46:23.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.373+0000 7efd06c00640 1 -- 172.21.15.123:0/224449587 wait complete. 2024-04-24T05:46:23.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.374+0000 7efd06c00640 1 Processor -- start 2024-04-24T05:46:23.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.374+0000 7efd06c00640 1 -- start start 2024-04-24T05:46:23.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.375+0000 7efd06c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd000ad690 0x7efd000738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.375+0000 7efd06c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd000af950 0x7efd0014ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.375+0000 7efd06c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd0015a2a0 0x7efd0006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.375+0000 7efd06c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efd0006c210 con 0x7efd000ad690 2024-04-24T05:46:23.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.375+0000 7efd06c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efd0015da00 con 0x7efd0015a2a0 2024-04-24T05:46:23.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.375+0000 7efd06c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efd0006c390 con 0x7efd000af950 2024-04-24T05:46:23.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.376+0000 7efcffe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd000ad690 0x7efd000738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.376+0000 7efcffe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd000ad690 0x7efd000738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56418/0 (socket says 172.21.15.123:56418) 2024-04-24T05:46:23.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.376+0000 7efcffe00640 1 -- 172.21.15.123:0/3701511330 learned_addr learned my addr 172.21.15.123:0/3701511330 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.376+0000 7efcffe00640 1 -- 172.21.15.123:0/3701511330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd000af950 msgr2=0x7efd0014ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.376+0000 7efcffe00640 1 --2- 172.21.15.123:0/3701511330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd000af950 0x7efd0014ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.376+0000 7efcffe00640 1 -- 172.21.15.123:0/3701511330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd0015a2a0 msgr2=0x7efd0006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.376+0000 7efcffe00640 1 --2- 172.21.15.123:0/3701511330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd0015a2a0 0x7efd0006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.376+0000 7efcffe00640 1 -- 172.21.15.123:0/3701511330 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd000cb7f0 con 0x7efd000ad690 2024-04-24T05:46:23.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.376+0000 7efcffe00640 1 --2- 172.21.15.123:0/3701511330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd000ad690 0x7efd000738b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009f70 tx=0x7efcf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.377+0000 7efcf7e00640 1 -- 172.21.15.123:0/3701511330 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0041020 con 0x7efd000ad690 2024-04-24T05:46:23.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.377+0000 7efcf7e00640 1 -- 172.21.15.123:0/3701511330 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf00087b0 con 0x7efd000ad690 2024-04-24T05:46:23.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.377+0000 7efcf7e00640 1 -- 172.21.15.123:0/3701511330 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0036040 con 0x7efd000ad690 2024-04-24T05:46:23.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.377+0000 7efd06c00640 1 -- 172.21.15.123:0/3701511330 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd000cba70 con 0x7efd000ad690 2024-04-24T05:46:23.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.377+0000 7efd06c00640 1 -- 172.21.15.123:0/3701511330 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd000c9f00 con 0x7efd000ad690 2024-04-24T05:46:23.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.378+0000 7efd06c00640 1 -- 172.21.15.123:0/3701511330 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd00068a20 con 0x7efd000ad690 2024-04-24T05:46:23.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.379+0000 7efcf7e00640 1 -- 172.21.15.123:0/3701511330 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcf0044020 con 0x7efd000ad690 2024-04-24T05:46:23.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.380+0000 7efcf7e00640 1 --2- 172.21.15.123:0/3701511330 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7efce006c8f0 0x7efce006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.380+0000 7efcf7e00640 1 -- 172.21.15.123:0/3701511330 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efcf00acbf0 con 0x7efd000ad690 2024-04-24T05:46:23.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.380+0000 7efcff400640 1 --2- 172.21.15.123:0/3701511330 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7efce006c8f0 0x7efce006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.381+0000 7efcff400640 1 --2- 172.21.15.123:0/3701511330 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7efce006c8f0 0x7efce006edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efcec000f60 tx=0x7efcec007a00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.385+0000 7efcf7e00640 1 -- 172.21.15.123:0/3701511330 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efcf004b030 con 0x7efd000ad690 2024-04-24T05:46:23.531 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:46:23.538 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:46:23.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.587+0000 7efd06c00640 1 -- 172.21.15.123:0/3701511330 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd0006fdf0 con 0x7efd000ad690 2024-04-24T05:46:23.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.589+0000 7efcf7e00640 1 -- 172.21.15.123:0/3701511330 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7efd000cb7f0 con 0x7efd000ad690 2024-04-24T05:46:23.591 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:23.591 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:04.715857+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:23.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.591+0000 7efd06c00640 1 -- 172.21.15.123:0/3701511330 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7efce006c8f0 msgr2=0x7efce006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.591+0000 7efd06c00640 1 --2- 172.21.15.123:0/3701511330 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7efce006c8f0 0x7efce006edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efcec000f60 tx=0x7efcec007a00 comp rx=0 tx=0).stop 2024-04-24T05:46:23.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.591+0000 7efd06c00640 1 -- 172.21.15.123:0/3701511330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd000ad690 msgr2=0x7efd000738b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.591+0000 7efd06c00640 1 --2- 172.21.15.123:0/3701511330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd000ad690 0x7efd000738b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009f70 tx=0x7efcf0004a40 comp rx=0 tx=0).stop 2024-04-24T05:46:23.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.591+0000 7efd06c00640 1 -- 172.21.15.123:0/3701511330 shutdown_connections 2024-04-24T05:46:23.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.591+0000 7efd06c00640 1 --2- 172.21.15.123:0/3701511330 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7efce006c8f0 0x7efce006edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.591+0000 7efd06c00640 1 --2- 172.21.15.123:0/3701511330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd0015a2a0 0x7efd0006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.591+0000 7efd06c00640 1 --2- 172.21.15.123:0/3701511330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd000af950 0x7efd0014ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.591+0000 7efd06c00640 1 --2- 172.21.15.123:0/3701511330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd000ad690 0x7efd000738b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.591+0000 7efd06c00640 1 -- 172.21.15.123:0/3701511330 >> 172.21.15.123:0/3701511330 conn(0x7efd000b1e70 msgr2=0x7efd00155120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:23.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.592+0000 7efd06c00640 1 -- 172.21.15.123:0/3701511330 shutdown_connections 2024-04-24T05:46:23.594 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.592+0000 7efd06c00640 1 -- 172.21.15.123:0/3701511330 wait complete. 2024-04-24T05:46:23.611 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-24T05:46:23.612 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-24T05:46:23.612 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-24T05:46:23.612 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-24T05:46:23.613 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-24T05:46:23.613 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-24T05:46:23.614 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-24T05:46:23.614 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-24T05:46:23.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.898+0000 7f6683e00640 1 Processor -- start 2024-04-24T05:46:23.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.898+0000 7fd207400640 1 Processor -- start 2024-04-24T05:46:23.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.899+0000 7fd207400640 1 -- start start 2024-04-24T05:46:23.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.899+0000 7f6683e00640 1 -- start start 2024-04-24T05:46:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.901+0000 7fd207400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd2000d0b70 0x7fd20006ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.901+0000 7fd207400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd20006bf40 0x7fd20015e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.901+0000 7fd207400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd200162460 con 0x7fd20015e800 2024-04-24T05:46:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.901+0000 7fd207400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd20006c3b0 con 0x7fd20006bf40 2024-04-24T05:46:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.901+0000 7fd207400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd2000cdfc0 con 0x7fd2000d0b70 2024-04-24T05:46:23.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.901+0000 7f6683e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f667c15e530 0x7f667c15e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.902+0000 7f6683e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f667c06ba00 0x7f667c06be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.902+0000 7fd1ffe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd20006bf40 0x7fd20015e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.902+0000 7f6683e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f667c06fa20 con 0x7f667c0ace60 2024-04-24T05:46:23.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.902+0000 7f6683e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f667c06c3b0 con 0x7f667c06ba00 2024-04-24T05:46:23.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.902+0000 7f6683e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f667c0cdfc0 con 0x7f667c15e530 2024-04-24T05:46:23.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.902+0000 7fd1ffe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd20006bf40 0x7fd20015e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56432/0 (socket says 172.21.15.123:56432) 2024-04-24T05:46:23.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.902+0000 7fd1ffe00640 1 -- 172.21.15.123:0/954624962 learned_addr learned my addr 172.21.15.123:0/954624962 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.903+0000 7f6680a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f667c06ba00 0x7f667c06be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.903+0000 7f6680a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f667c06ba00 0x7f667c06be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:49900/0 (socket says 172.21.15.123:49900) 2024-04-24T05:46:23.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.903+0000 7f6680a00640 1 -- 172.21.15.123:0/2468363671 learned_addr learned my addr 172.21.15.123:0/2468363671 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.902+0000 7f6681400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f667c15e530 0x7f667c15e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.903+0000 7fd1ffe00640 1 -- 172.21.15.123:0/954624962 >> v2:172.21.15.155:3300/0 conn(0x7fd2000d0b70 msgr2=0x7fd20006ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:23.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.903+0000 7fd1ffe00640 1 --2- 172.21.15.123:0/954624962 >> v2:172.21.15.155:3300/0 conn(0x7fd2000d0b70 0x7fd20006ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.904+0000 7fd1ffe00640 1 -- 172.21.15.123:0/954624962 >> v1:172.21.15.155:6789/0 conn(0x7fd20015e800 legacy=0x7fd2000ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:23.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.903+0000 7f6680a00640 1 -- 172.21.15.123:0/2468363671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f667c15e530 msgr2=0x7f667c15e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:23.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.903+0000 7f6680a00640 1 --2- 172.21.15.123:0/2468363671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f667c15e530 0x7f667c15e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.904+0000 7f6680a00640 1 -- 172.21.15.123:0/2468363671 >> v1:172.21.15.123:6789/0 conn(0x7f667c0ace60 legacy=0x7f667c0ad2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T05:46:23.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.904+0000 7f6680a00640 1 -- 172.21.15.123:0/2468363671 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c06f460 con 0x7f667c06ba00 2024-04-24T05:46:23.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.905+0000 7f7b34c00640 1 Processor -- start 2024-04-24T05:46:23.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.904+0000 7fd1ffe00640 1 -- 172.21.15.123:0/954624962 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20006f460 con 0x7fd20006bf40 2024-04-24T05:46:23.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.905+0000 7f7b34c00640 1 -- start start 2024-04-24T05:46:23.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.906+0000 7fd1ffe00640 1 --2- 172.21.15.123:0/954624962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd20006bf40 0x7fd20015e2c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd2000cffb0 tx=0x7fd1f802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9dac63f2cbb50c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.906+0000 7f6680a00640 1 --2- 172.21.15.123:0/2468363671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f667c06ba00 0x7f667c06be70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6674009e50 tx=0x7f667402c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e19b65f5bc2fd3a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.906+0000 7fd1ff400640 1 -- 172.21.15.123:0/954624962 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8036070 con 0x7fd20006bf40 2024-04-24T05:46:23.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.906+0000 7fd1ff400640 1 -- 172.21.15.123:0/954624962 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f8031600 con 0x7fd20006bf40 2024-04-24T05:46:23.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.906+0000 7f667be00640 1 -- 172.21.15.123:0/2468363671 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66740333f0 con 0x7f667c06ba00 2024-04-24T05:46:23.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.906+0000 7fd1ff400640 1 -- 172.21.15.123:0/954624962 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80319f0 con 0x7fd20006bf40 2024-04-24T05:46:23.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.907+0000 7f7b34c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b280069d0 0x7f7b28006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.907+0000 7f7b34c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b28053710 con 0x7f7b280eacf0 2024-04-24T05:46:23.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.907+0000 7f7b34c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b280539f0 con 0x7f7b28005e80 2024-04-24T05:46:23.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.907+0000 7f7b34c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7b28054320 con 0x7f7b280069d0 2024-04-24T05:46:23.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.907+0000 7f667be00640 1 -- 172.21.15.123:0/2468363671 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6674033590 con 0x7f667c06ba00 2024-04-24T05:46:23.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.907+0000 7f667be00640 1 -- 172.21.15.123:0/2468363671 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674033960 con 0x7f667c06ba00 2024-04-24T05:46:23.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.907+0000 7f7b2de00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7b280eacf0 0x7f7b2804fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35530/0 (socket says 172.21.15.123:35530) 2024-04-24T05:46:23.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.907+0000 7f7b2de00640 1 -- 172.21.15.123:0/967062442 learned_addr learned my addr 172.21.15.123:0/967062442 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:23.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.907+0000 7fd207400640 1 -- 172.21.15.123:0/954624962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd20006bf40 msgr2=0x7fd20015e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.907+0000 7fd207400640 1 --2- 172.21.15.123:0/954624962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd20006bf40 0x7fd20015e2c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd2000cffb0 tx=0x7fd1f802a7c0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.908+0000 7fd207400640 1 -- 172.21.15.123:0/954624962 shutdown_connections 2024-04-24T05:46:23.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.908+0000 7fd207400640 1 --2- 172.21.15.123:0/954624962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd20006bf40 0x7fd20015e2c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.908+0000 7fd207400640 1 --2- 172.21.15.123:0/954624962 >> v2:172.21.15.155:3300/0 conn(0x7fd2000d0b70 0x7fd20006ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.908+0000 7fd207400640 1 -- 172.21.15.123:0/954624962 >> 172.21.15.123:0/954624962 conn(0x7fd2000d0050 msgr2=0x7fd2000d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:23.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.908+0000 7f7b2e800640 1 --2- 172.21.15.123:0/967062442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b280069d0 0x7f7b28006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.908+0000 7f7b2ca00640 1 -- 172.21.15.123:0/967062442 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1752434913 0 0) 0x7f7b28053710 con 0x7f7b280eacf0 2024-04-24T05:46:23.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.908+0000 7f6683e00640 1 -- 172.21.15.123:0/2468363671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f667c06ba00 msgr2=0x7f667c06be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.908+0000 7f6683e00640 1 --2- 172.21.15.123:0/2468363671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f667c06ba00 0x7f667c06be70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6674009e50 tx=0x7f667402c8e0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.908+0000 7f6683e00640 1 -- 172.21.15.123:0/2468363671 shutdown_connections 2024-04-24T05:46:23.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.908+0000 7f6683e00640 1 --2- 172.21.15.123:0/2468363671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f667c06ba00 0x7f667c06be70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.908+0000 7f6683e00640 1 --2- 172.21.15.123:0/2468363671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f667c15e530 0x7f667c15e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.908+0000 7f6683e00640 1 -- 172.21.15.123:0/2468363671 >> 172.21.15.123:0/2468363671 conn(0x7f667c0d0050 msgr2=0x7f667c0d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:23.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.908+0000 7f7b2ca00640 1 -- 172.21.15.123:0/967062442 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b14003520 con 0x7f7b280eacf0 2024-04-24T05:46:23.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.908+0000 7f6683e00640 1 -- 172.21.15.123:0/2468363671 shutdown_connections 2024-04-24T05:46:23.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.909+0000 7f7b2ca00640 1 -- 172.21.15.123:0/967062442 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1007391740 0 0) 0x7f7b14003520 con 0x7f7b280eacf0 2024-04-24T05:46:23.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.909+0000 7f6683e00640 1 -- 172.21.15.123:0/2468363671 wait complete. 2024-04-24T05:46:23.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.909+0000 7f7b2e800640 1 -- 172.21.15.123:0/967062442 >> v1:172.21.15.155:6789/0 conn(0x7f7b28005e80 legacy=0x7f7b280062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.909+0000 7f7b2e800640 1 -- 172.21.15.123:0/967062442 >> v1:172.21.15.123:6789/0 conn(0x7f7b280eacf0 legacy=0x7f7b2804fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.909+0000 7f7b2e800640 1 -- 172.21.15.123:0/967062442 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b28053cd0 con 0x7f7b280069d0 2024-04-24T05:46:23.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.909+0000 7fd207400640 1 -- 172.21.15.123:0/954624962 shutdown_connections 2024-04-24T05:46:23.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.909+0000 7fd207400640 1 -- 172.21.15.123:0/954624962 wait complete. 2024-04-24T05:46:23.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.911+0000 7f7b2e800640 1 --2- 172.21.15.123:0/967062442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b280069d0 0x7f7b28006e60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7b300502b0 tx=0x7f7b300a5a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faf77add2f6cb018 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.911+0000 7f2191000640 1 Processor -- start 2024-04-24T05:46:23.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.912+0000 7f2191000640 1 -- start start 2024-04-24T05:46:23.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.911+0000 7f46ce800640 1 Processor -- start 2024-04-24T05:46:23.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.911+0000 7f46ce800640 1 -- start start 2024-04-24T05:46:23.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.912+0000 7fd207400640 1 Processor -- start 2024-04-24T05:46:23.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.912+0000 7fd207400640 1 -- start start 2024-04-24T05:46:23.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.912+0000 7f6683e00640 1 Processor -- start 2024-04-24T05:46:23.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.913+0000 7f6683e00640 1 -- start start 2024-04-24T05:46:23.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.913+0000 7f46ce800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c0048040 0x7f46c004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.912+0000 7f0453600640 1 Processor -- start 2024-04-24T05:46:23.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.912+0000 7f0453600640 1 -- start start 2024-04-24T05:46:23.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.913+0000 7fd0db200640 1 Processor -- start 2024-04-24T05:46:23.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.913+0000 7f46ce800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f46c0037bb0 0x7f46c0038010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.913+0000 7f2191000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840e1310 0x7f218403a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.913+0000 7f46ce800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f46c0038550 0x7f46c00389e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7fd207400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd20006bf40 0x7fd2000c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7fd207400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2000d0b70 0x7fd20015cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f0453600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f044c073020 0x7f044c073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f0453600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f044c06bb00 0x7f044c06bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f0453600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f044c099da0 con 0x7f044c0b5da0 2024-04-24T05:46:23.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f0453600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f044c16f400 con 0x7f044c073020 2024-04-24T05:46:23.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f0453600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f044c16f6e0 con 0x7f044c06bb00 2024-04-24T05:46:23.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f7b2ca00640 1 -- 172.21.15.123:0/967062442 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b300ae070 con 0x7f7b280069d0 2024-04-24T05:46:23.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f46ce800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f46c0048420 con 0x7f46c0038550 2024-04-24T05:46:23.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f46ce800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f46c00547a0 con 0x7f46c0037bb0 2024-04-24T05:46:23.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f46ce800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f46c0053700 con 0x7f46c0048040 2024-04-24T05:46:23.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f7b2ca00640 1 -- 172.21.15.123:0/967062442 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b300aa6d0 con 0x7f7b280069d0 2024-04-24T05:46:23.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f7b2ca00640 1 -- 172.21.15.123:0/967062442 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20001670 con 0x7f7b280069d0 2024-04-24T05:46:23.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f6683e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f667c06ba00 0x7f667c15a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f6683e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f667c0ace60 0x7f667c0b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f6683e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f667c15e530 0x7f667c0c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f6683e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f667c0cfd20 con 0x7f667c0ace60 2024-04-24T05:46:23.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f6683e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f667c0cf800 con 0x7f667c06ba00 2024-04-24T05:46:23.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f6683e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f667c0cdc20 con 0x7f667c15e530 2024-04-24T05:46:23.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f46cca00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f46c0037bb0 0x7f46c0038010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f46cd400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c0048040 0x7f46c004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7f218b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840e1310 0x7f218403a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f218b400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840e1310 0x7f218403a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38488/0 (socket says 172.21.15.123:38488) 2024-04-24T05:46:23.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f218b400640 1 -- 172.21.15.123:0/2801285987 learned_addr learned my addr 172.21.15.123:0/2801285987 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f2191000640 1 -- 172.21.15.123:0/2801285987 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218403a6b0 con 0x7f218400a460 2024-04-24T05:46:23.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f2191000640 1 -- 172.21.15.123:0/2801285987 --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2184054d10 con 0x7f218400ab40 2024-04-24T05:46:23.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f2191000640 1 -- 172.21.15.123:0/2801285987 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2184055920 con 0x7f21840e1310 2024-04-24T05:46:23.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7fd207400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd20015e800 0x7fd20015d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7fd207400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2000cfd20 con 0x7fd2000d0b70 2024-04-24T05:46:23.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f044be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f044c073020 0x7f044c073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7fd204a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd20006bf40 0x7fd2000c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7fd204a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd20006bf40 0x7fd2000c7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38494/0 (socket says 172.21.15.123:38494) 2024-04-24T05:46:23.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7fd204a00640 1 -- 172.21.15.123:0/449693257 learned_addr learned my addr 172.21.15.123:0/449693257 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f7b34c00640 1 -- 172.21.15.123:0/967062442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b280069d0 msgr2=0x7f7b28006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f7b34c00640 1 --2- 172.21.15.123:0/967062442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b280069d0 0x7f7b28006e60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7b300502b0 tx=0x7f7b300a5a70 comp rx=0 tx=0).stop 2024-04-24T05:46:23.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7fd205400640 1 --2- 172.21.15.123:0/449693257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd20015e800 0x7fd20015d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.914+0000 7fd0db200640 1 -- start start 2024-04-24T05:46:23.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f46cd400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c0048040 0x7f46c004fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38486/0 (socket says 172.21.15.123:38486) 2024-04-24T05:46:23.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f46cd400640 1 -- 172.21.15.123:0/533812642 learned_addr learned my addr 172.21.15.123:0/533812642 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7fd1ffe00640 1 --2- 172.21.15.123:0/449693257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2000d0b70 0x7fd20015cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f044be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f044c073020 0x7f044c073470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56442/0 (socket says 172.21.15.123:56442) 2024-04-24T05:46:23.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f6681400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f667c06ba00 0x7f667c15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f7b34c00640 1 -- 172.21.15.123:0/967062442 shutdown_connections 2024-04-24T05:46:23.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f044be00640 1 -- 172.21.15.123:0/2781131522 learned_addr learned my addr 172.21.15.123:0/2781131522 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f7b34c00640 1 --2- 172.21.15.123:0/967062442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b280069d0 0x7f7b28006e60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f7b34c00640 1 -- 172.21.15.123:0/967062442 >> 172.21.15.123:0/967062442 conn(0x7f7b28009fa0 msgr2=0x7f7b2800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:23.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f46cde00640 1 --2- 172.21.15.123:0/533812642 >> v2:172.21.15.123:3300/0 conn(0x7f46c0038550 0x7f46c00389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f6681400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f667c06ba00 0x7f667c15a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:49934/0 (socket says 172.21.15.123:49934) 2024-04-24T05:46:23.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.915+0000 7f6681400640 1 -- 172.21.15.123:0/878877777 learned_addr learned my addr 172.21.15.123:0/878877777 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7fd207400640 1 -- 172.21.15.123:0/449693257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd2000cf800 con 0x7fd20015e800 2024-04-24T05:46:23.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7fd207400640 1 -- 172.21.15.123:0/449693257 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd2000cdc20 con 0x7fd20006bf40 2024-04-24T05:46:23.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7f0451600640 1 --2- 172.21.15.123:0/2781131522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f044c06bb00 0x7f044c06bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7f7b34c00640 1 -- 172.21.15.123:0/967062442 shutdown_connections 2024-04-24T05:46:23.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7f7b34c00640 1 -- 172.21.15.123:0/967062442 wait complete. 2024-04-24T05:46:23.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7f46cd400640 1 -- 172.21.15.123:0/533812642 >> v2:172.21.15.155:3300/0 conn(0x7f46c0037bb0 msgr2=0x7f46c0038010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:23.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7f46cd400640 1 --2- 172.21.15.123:0/533812642 >> v2:172.21.15.155:3300/0 conn(0x7f46c0037bb0 0x7f46c0038010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7f46cd400640 1 -- 172.21.15.123:0/533812642 >> v2:172.21.15.123:3300/0 conn(0x7f46c0038550 msgr2=0x7f46c00389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:23.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7f46cd400640 1 --2- 172.21.15.123:0/533812642 >> v2:172.21.15.123:3300/0 conn(0x7f46c0038550 0x7f46c00389e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7f46cd400640 1 -- 172.21.15.123:0/533812642 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c00539e0 con 0x7f46c0048040 2024-04-24T05:46:23.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7f46cde00640 1 --2- 172.21.15.123:0/533812642 >> v2:172.21.15.123:3300/0 conn(0x7f46c0038550 0x7f46c00389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:46:23.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7f6680a00640 1 --2- 172.21.15.123:0/878877777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f667c0ace60 0x7f667c0b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7f218a000640 1 -- 172.21.15.123:0/2801285987 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3070403783 0 0) 0x7f2184054d10 con 0x7f218400ab40 2024-04-24T05:46:23.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7f218a000640 1 -- 172.21.15.123:0/2801285987 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2174003520 con 0x7f218400ab40 2024-04-24T05:46:23.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.916+0000 7f6681e00640 1 --2- 172.21.15.123:0/878877777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f667c15e530 0x7f667c0c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7f218b400640 1 -- 172.21.15.123:0/2801285987 >> v1:172.21.15.155:6789/0 conn(0x7f218400ab40 legacy=0x7f2184009d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7f218b400640 1 -- 172.21.15.123:0/2801285987 >> v1:172.21.15.123:6789/0 conn(0x7f218400a460 legacy=0x7f2184055db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:23.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7f6681400640 1 -- 172.21.15.123:0/878877777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f667c15e530 msgr2=0x7f667c0c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7f6681400640 1 --2- 172.21.15.123:0/878877777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f667c15e530 0x7f667c0c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7f6681400640 1 -- 172.21.15.123:0/878877777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f667c0ace60 msgr2=0x7f667c0b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7f6681400640 1 --2- 172.21.15.123:0/878877777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f667c0ace60 0x7f667c0b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7f6681400640 1 -- 172.21.15.123:0/878877777 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c0c7b00 con 0x7f667c06ba00 2024-04-24T05:46:23.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7fd204a00640 1 -- 172.21.15.123:0/449693257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd20015e800 msgr2=0x7fd20015d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7fd204a00640 1 --2- 172.21.15.123:0/449693257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd20015e800 0x7fd20015d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7fd0db200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc15e680 0x7fd0dc06cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7fd0db200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd0dc0ceac0 0x7fd0dc0d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7fd0db200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0dc0cf060 con 0x7fd0dc0d0640 2024-04-24T05:46:23.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7fd0db200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd0dc0cf340 con 0x7fd0dc0ceac0 2024-04-24T05:46:23.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7fd0db200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd0dc0cf620 con 0x7fd0dc15e680 2024-04-24T05:46:23.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7f218b400640 1 -- 172.21.15.123:0/2801285987 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2184054ff0 con 0x7f21840e1310 2024-04-24T05:46:23.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7fd204a00640 1 -- 172.21.15.123:0/449693257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2000d0b70 msgr2=0x7fd20015cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7fd204a00640 1 --2- 172.21.15.123:0/449693257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2000d0b70 0x7fd20015cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.917+0000 7fd204a00640 1 -- 172.21.15.123:0/449693257 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2000c96f0 con 0x7fd20006bf40 2024-04-24T05:46:23.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.918+0000 7f044b400640 1 -- 172.21.15.123:0/2781131522 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1475082304 0 0) 0x7f044c099da0 con 0x7f044c0b5da0 2024-04-24T05:46:23.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.918+0000 7f044b400640 1 -- 172.21.15.123:0/2781131522 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0434003520 con 0x7f044c0b5da0 2024-04-24T05:46:23.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.918+0000 7fd204a00640 1 --2- 172.21.15.123:0/449693257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd20006bf40 0x7fd2000c7c80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000b4d0 tx=0x7fd1f000b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.918+0000 7fd0d9e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc15e680 0x7fd0dc06cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.918+0000 7fd0d9e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc15e680 0x7fd0dc06cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38536/0 (socket says 172.21.15.123:38536) 2024-04-24T05:46:23.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.918+0000 7fd0d9e00640 1 -- 172.21.15.123:0/1223712608 learned_addr learned my addr 172.21.15.123:0/1223712608 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.918+0000 7f6681400640 1 --2- 172.21.15.123:0/878877777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f667c06ba00 0x7f667c15a670 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f667000e870 tx=0x7f667000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.918+0000 7f46cd400640 1 --2- 172.21.15.123:0/533812642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c0048040 0x7f46c004fa90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f46c00525b0 tx=0x7f46bc02a260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11784dac1c85cb4a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.918+0000 7f7b34c00640 1 Processor -- start 2024-04-24T05:46:23.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.918+0000 7fd1fcc00640 1 -- 172.21.15.123:0/449693257 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0014070 con 0x7fd20006bf40 2024-04-24T05:46:23.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.918+0000 7fd1fcc00640 1 -- 172.21.15.123:0/449693257 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f0010870 con 0x7fd20006bf40 2024-04-24T05:46:23.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.918+0000 7fd1fcc00640 1 -- 172.21.15.123:0/449693257 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0010c60 con 0x7fd20006bf40 2024-04-24T05:46:23.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.918+0000 7fd207400640 1 -- 172.21.15.123:0/449693257 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2000c99d0 con 0x7fd20006bf40 2024-04-24T05:46:23.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.918+0000 7fd0d9400640 1 --2- 172.21.15.123:0/1223712608 >> v2:172.21.15.123:3300/0 conn(0x7fd0dc0ceac0 0x7fd0dc0d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7fd207400640 1 -- 172.21.15.123:0/449693257 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2000c81c0 con 0x7fd20006bf40 2024-04-24T05:46:23.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7f46c7e00640 1 -- 172.21.15.123:0/533812642 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc037070 con 0x7f46c0048040 2024-04-24T05:46:23.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7f46c7e00640 1 -- 172.21.15.123:0/533812642 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46bc032440 con 0x7f46c0048040 2024-04-24T05:46:23.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7f46c7e00640 1 -- 172.21.15.123:0/533812642 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc032810 con 0x7f46c0048040 2024-04-24T05:46:23.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7fd0d9e00640 1 -- 172.21.15.123:0/1223712608 >> v2:172.21.15.123:3300/0 conn(0x7fd0dc0ceac0 msgr2=0x7fd0dc0d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:23.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7fd0d9e00640 1 --2- 172.21.15.123:0/1223712608 >> v2:172.21.15.123:3300/0 conn(0x7fd0dc0ceac0 0x7fd0dc0d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7fd0d9e00640 1 -- 172.21.15.123:0/1223712608 >> v1:172.21.15.155:6789/0 conn(0x7fd0dc0d0640 legacy=0x7fd0dc0b76a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T05:46:23.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7fd0d9e00640 1 -- 172.21.15.123:0/1223712608 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0dc0cf900 con 0x7fd0dc15e680 2024-04-24T05:46:23.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7f7b34c00640 1 -- start start 2024-04-24T05:46:23.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7f6679600640 1 -- 172.21.15.123:0/878877777 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670017070 con 0x7f667c06ba00 2024-04-24T05:46:23.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7f6679600640 1 -- 172.21.15.123:0/878877777 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f667000c940 con 0x7f667c06ba00 2024-04-24T05:46:23.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7f6679600640 1 -- 172.21.15.123:0/878877777 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667000cd30 con 0x7f667c06ba00 2024-04-24T05:46:23.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7f044be00640 1 -- 172.21.15.123:0/2781131522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f044c06bb00 msgr2=0x7f044c06bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:23.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7f044be00640 1 --2- 172.21.15.123:0/2781131522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f044c06bb00 0x7f044c06bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7f044be00640 1 -- 172.21.15.123:0/2781131522 >> v1:172.21.15.123:6789/0 conn(0x7f044c0b5da0 legacy=0x7f044c099690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.919+0000 7f044be00640 1 -- 172.21.15.123:0/2781131522 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044c16f9c0 con 0x7f044c073020 2024-04-24T05:46:23.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.920+0000 7f46ce800640 1 -- 172.21.15.123:0/533812642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c0048040 msgr2=0x7f46c004fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.920+0000 7f46ce800640 1 --2- 172.21.15.123:0/533812642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c0048040 0x7f46c004fa90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f46c00525b0 tx=0x7f46bc02a260 comp rx=0 tx=0).stop 2024-04-24T05:46:23.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.920+0000 7f6683e00640 1 -- 172.21.15.123:0/878877777 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667c0c7de0 con 0x7f667c06ba00 2024-04-24T05:46:23.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.920+0000 7f6683e00640 1 -- 172.21.15.123:0/878877777 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667c0c83a0 con 0x7f667c06ba00 2024-04-24T05:46:23.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.921+0000 7fd207400640 1 -- 172.21.15.123:0/449693257 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd2000b2e40 con 0x7fd20006bf40 2024-04-24T05:46:23.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.921+0000 7f46ce800640 1 -- 172.21.15.123:0/533812642 shutdown_connections 2024-04-24T05:46:23.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.921+0000 7f46ce800640 1 --2- 172.21.15.123:0/533812642 >> v2:172.21.15.123:3300/0 conn(0x7f46c0038550 0x7f46c00389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.921+0000 7f46ce800640 1 --2- 172.21.15.123:0/533812642 >> v2:172.21.15.155:3300/0 conn(0x7f46c0037bb0 0x7f46c0038010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.921+0000 7f46ce800640 1 --2- 172.21.15.123:0/533812642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c0048040 0x7f46c004fa90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.921+0000 7f46ce800640 1 -- 172.21.15.123:0/533812642 >> 172.21.15.123:0/533812642 conn(0x7f46c0009fb0 msgr2=0x7f46c000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:23.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.921+0000 7f218b400640 1 --2- 172.21.15.123:0/2801285987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840e1310 0x7f218403a170 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f21800098a0 tx=0x7f218002a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb7d5691f5c825c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.921+0000 7f46ce800640 1 -- 172.21.15.123:0/533812642 shutdown_connections 2024-04-24T05:46:23.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.921+0000 7f7b34c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b28005e80 0x7f7b28180c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.921+0000 7f044be00640 1 --2- 172.21.15.123:0/2781131522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f044c073020 0x7f044c073470 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f043800bb00 tx=0x7f043802c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b839c6d6d3eeab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.921+0000 7f7b34c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b280069d0 0x7f7b280dc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.921+0000 7f46ce800640 1 -- 172.21.15.123:0/533812642 wait complete. 2024-04-24T05:46:23.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.922+0000 7f7b34c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b280eacf0 0x7f7b280d96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.922+0000 7f7b34c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b280547b0 con 0x7f7b280069d0 2024-04-24T05:46:23.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.922+0000 7f7b34c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b28050190 con 0x7f7b280eacf0 2024-04-24T05:46:23.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.922+0000 7f7b34c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7b28054930 con 0x7f7b28005e80 2024-04-24T05:46:23.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.922+0000 7f218a000640 1 -- 172.21.15.123:0/2801285987 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180037070 con 0x7f21840e1310 2024-04-24T05:46:23.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.922+0000 7f218a000640 1 -- 172.21.15.123:0/2801285987 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2180032440 con 0x7f21840e1310 2024-04-24T05:46:23.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.922+0000 7f218a000640 1 -- 172.21.15.123:0/2801285987 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180032810 con 0x7f21840e1310 2024-04-24T05:46:23.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.922+0000 7f044b400640 1 -- 172.21.15.123:0/2781131522 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043803a070 con 0x7f044c073020 2024-04-24T05:46:23.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.922+0000 7f044b400640 1 -- 172.21.15.123:0/2781131522 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0438032480 con 0x7f044c073020 2024-04-24T05:46:23.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.922+0000 7f044b400640 1 -- 172.21.15.123:0/2781131522 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438032850 con 0x7f044c073020 2024-04-24T05:46:23.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f2191000640 1 -- 172.21.15.123:0/2801285987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840e1310 msgr2=0x7f218403a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.922+0000 7f7b2d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b280069d0 0x7f7b280dc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.922+0000 7f7b2de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b28005e80 0x7f7b28180c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.922+0000 7f7b2de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b28005e80 0x7f7b28180c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38552/0 (socket says 172.21.15.123:38552) 2024-04-24T05:46:23.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f2191000640 1 --2- 172.21.15.123:0/2801285987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840e1310 0x7f218403a170 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f21800098a0 tx=0x7f218002a2c0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.922+0000 7f7b2de00640 1 -- 172.21.15.123:0/3203296399 learned_addr learned my addr 172.21.15.123:0/3203296399 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f666ea00640 1 -- 172.21.15.123:0/878877777 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6650001030 con 0x7f667c06ba00 2024-04-24T05:46:23.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f7b2e800640 1 --2- 172.21.15.123:0/3203296399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b280eacf0 0x7f7b280d96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f7b2de00640 1 -- 172.21.15.123:0/3203296399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b280eacf0 msgr2=0x7f7b280d96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f7b2de00640 1 --2- 172.21.15.123:0/3203296399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b280eacf0 0x7f7b280d96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f7b2de00640 1 -- 172.21.15.123:0/3203296399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b280069d0 msgr2=0x7f7b280dc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f7b2de00640 1 --2- 172.21.15.123:0/3203296399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b280069d0 0x7f7b280dc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f7b2de00640 1 -- 172.21.15.123:0/3203296399 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b280dd1a0 con 0x7f7b28005e80 2024-04-24T05:46:23.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f0453600640 1 -- 172.21.15.123:0/2781131522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f044c073020 msgr2=0x7f044c073470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f0453600640 1 --2- 172.21.15.123:0/2781131522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f044c073020 0x7f044c073470 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f043800bb00 tx=0x7f043802c6e0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f2191000640 1 -- 172.21.15.123:0/2801285987 shutdown_connections 2024-04-24T05:46:23.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f2191000640 1 --2- 172.21.15.123:0/2801285987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840e1310 0x7f218403a170 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f2191000640 1 -- 172.21.15.123:0/2801285987 >> 172.21.15.123:0/2801285987 conn(0x7f2184056cd0 msgr2=0x7f21840570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:23.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7f7b2e800640 1 --2- 172.21.15.123:0/3203296399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b280eacf0 0x7f7b280d96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:23.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.923+0000 7fd0d9e00640 1 --2- 172.21.15.123:0/1223712608 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc15e680 0x7fd0dc06cf80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009e50 tx=0x7fd0d402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf71a5b19ea71f15 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.924+0000 7f2191000640 1 -- 172.21.15.123:0/2801285987 shutdown_connections 2024-04-24T05:46:23.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.924+0000 7f0453600640 1 -- 172.21.15.123:0/2781131522 shutdown_connections 2024-04-24T05:46:23.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.924+0000 7f0453600640 1 --2- 172.21.15.123:0/2781131522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f044c06bb00 0x7f044c06bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.924+0000 7f2191000640 1 -- 172.21.15.123:0/2801285987 wait complete. 2024-04-24T05:46:23.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.924+0000 7f0453600640 1 --2- 172.21.15.123:0/2781131522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f044c073020 0x7f044c073470 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.924+0000 7f0453600640 1 -- 172.21.15.123:0/2781131522 >> 172.21.15.123:0/2781131522 conn(0x7f044c0b4fd0 msgr2=0x7f044c0b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:23.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.924+0000 7f0453600640 1 -- 172.21.15.123:0/2781131522 shutdown_connections 2024-04-24T05:46:23.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.924+0000 7f0453600640 1 -- 172.21.15.123:0/2781131522 wait complete. 2024-04-24T05:46:23.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.924+0000 7fd0d8a00640 1 -- 172.21.15.123:0/1223712608 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4036070 con 0x7fd0dc15e680 2024-04-24T05:46:23.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.924+0000 7fd0d8a00640 1 -- 172.21.15.123:0/1223712608 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d4031600 con 0x7fd0dc15e680 2024-04-24T05:46:23.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.924+0000 7fd0d8a00640 1 -- 172.21.15.123:0/1223712608 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d40319d0 con 0x7fd0dc15e680 2024-04-24T05:46:23.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.924+0000 7fd1fcc00640 1 -- 172.21.15.123:0/449693257 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1f0017070 con 0x7fd20006bf40 2024-04-24T05:46:23.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.925+0000 7f7b2de00640 1 --2- 172.21.15.123:0/3203296399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b28005e80 0x7f7b28180c40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b20001eb0 tx=0x7f7b2000c9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.924+0000 7f46ce800640 1 Processor -- start 2024-04-24T05:46:23.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.925+0000 7fd1fcc00640 1 --2- 172.21.15.123:0/449693257 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd1e006c920 0x7fd1e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.925+0000 7fd1fcc00640 1 -- 172.21.15.123:0/449693257 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd1f008c170 con 0x7fd20006bf40 2024-04-24T05:46:23.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.925+0000 7fd1fcc00640 1 --2- 172.21.15.123:0/449693257 >> [v2:172.21.15.123:6800/1099836695,v1:172.21.15.123:6801/1099836695] conn(0x7fd1e00760a0 0x7fd1e00784e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.925+0000 7fd0db200640 1 -- 172.21.15.123:0/1223712608 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc15e680 msgr2=0x7fd0dc06cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.925+0000 7f46ce800640 1 -- start start 2024-04-24T05:46:23.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.925+0000 7fd0db200640 1 --2- 172.21.15.123:0/1223712608 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc15e680 0x7fd0dc06cf80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009e50 tx=0x7fd0d402a6d0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.925+0000 7fd1fcc00640 1 -- 172.21.15.123:0/449693257 --> [v2:172.21.15.123:6800/1099836695,v1:172.21.15.123:6801/1099836695] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1f0016ec0 con 0x7fd1e00760a0 2024-04-24T05:46:23.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.926+0000 7f6679600640 1 -- 172.21.15.123:0/878877777 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f667001a070 con 0x7f667c06ba00 2024-04-24T05:46:23.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.926+0000 7f2191000640 1 Processor -- start 2024-04-24T05:46:23.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.926+0000 7fd0db200640 1 -- 172.21.15.123:0/1223712608 shutdown_connections 2024-04-24T05:46:23.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.926+0000 7fd0db200640 1 --2- 172.21.15.123:0/1223712608 >> v2:172.21.15.123:3300/0 conn(0x7fd0dc0ceac0 0x7fd0dc0d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.926+0000 7fd0db200640 1 --2- 172.21.15.123:0/1223712608 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc15e680 0x7fd0dc06cf80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.926+0000 7fd0db200640 1 -- 172.21.15.123:0/1223712608 >> 172.21.15.123:0/1223712608 conn(0x7fd0dc0b5f60 msgr2=0x7fd0dc0b6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:23.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.925+0000 7f7b26000640 1 -- 172.21.15.123:0/3203296399 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20017070 con 0x7f7b28005e80 2024-04-24T05:46:23.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.926+0000 7f2191000640 1 -- start start 2024-04-24T05:46:23.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.926+0000 7fd0db200640 1 -- 172.21.15.123:0/1223712608 shutdown_connections 2024-04-24T05:46:23.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.925+0000 7fd1fcc00640 1 -- 172.21.15.123:0/449693257 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd1f008c560 con 0x7fd20006bf40 2024-04-24T05:46:23.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.926+0000 7fd1ffe00640 1 --2- 172.21.15.123:0/449693257 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd1e006c920 0x7fd1e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.926+0000 7fd205400640 1 --2- 172.21.15.123:0/449693257 >> [v2:172.21.15.123:6800/1099836695,v1:172.21.15.123:6801/1099836695] conn(0x7fd1e00760a0 0x7fd1e00784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.927+0000 7f0453600640 1 Processor -- start 2024-04-24T05:46:23.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.927+0000 7f0453600640 1 -- start start 2024-04-24T05:46:23.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.926+0000 7f7b26000640 1 -- 172.21.15.123:0/3203296399 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b20005ce0 con 0x7f7b28005e80 2024-04-24T05:46:23.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.927+0000 7f7b26000640 1 -- 172.21.15.123:0/3203296399 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20017070 con 0x7f7b28005e80 2024-04-24T05:46:23.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.927+0000 7f7b34c00640 1 -- 172.21.15.123:0/3203296399 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b280dcbe0 con 0x7f7b28005e80 2024-04-24T05:46:23.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.927+0000 7fbc2ee00640 1 Processor -- start 2024-04-24T05:46:23.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.927+0000 7f7b34c00640 1 -- 172.21.15.123:0/3203296399 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b28180440 con 0x7f7b28005e80 2024-04-24T05:46:23.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.927+0000 7fbc2ee00640 1 -- start start 2024-04-24T05:46:23.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.927+0000 7f6679600640 1 --2- 172.21.15.123:0/878877777 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f665c06c940 0x7f665c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f6679600640 1 -- 172.21.15.123:0/878877777 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f667008c5b0 con 0x7f667c06ba00 2024-04-24T05:46:23.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f2191000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f218400a460 0x7f21840d3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f2191000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f218400ab40 0x7f2184035990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f46ce800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c0037bb0 0x7f46c004bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f46ce800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46c0038550 0x7f46c00dcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f6679600640 1 --2- 172.21.15.123:0/878877777 >> [v2:172.21.15.123:6808/2060855224,v1:172.21.15.123:6809/2060855224] conn(0x7f665c0760c0 0x7f665c078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f2191000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840e1310 0x7f2184037720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f2191000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21840548a0 con 0x7f218400a460 2024-04-24T05:46:23.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f2191000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f21840555b0 con 0x7f218400ab40 2024-04-24T05:46:23.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f2191000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2184054a20 con 0x7f21840e1310 2024-04-24T05:46:23.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f6679600640 1 -- 172.21.15.123:0/878877777 --> [v2:172.21.15.123:6808/2060855224,v1:172.21.15.123:6809/2060855224] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f665c078bd0 con 0x7f665c0760c0 2024-04-24T05:46:23.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f6679600640 1 -- 172.21.15.123:0/878877777 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f667008c960 con 0x7f667c06ba00 2024-04-24T05:46:23.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f6681e00640 1 --2- 172.21.15.123:0/878877777 >> [v2:172.21.15.123:6808/2060855224,v1:172.21.15.123:6809/2060855224] conn(0x7f665c0760c0 0x7f665c078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f46ce800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c0048040 0x7f46c0036590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f46ce800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46c0054280 con 0x7f46c0038550 2024-04-24T05:46:23.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f46ce800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f46c0053fa0 con 0x7f46c0037bb0 2024-04-24T05:46:23.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7f46ce800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f46c004ffd0 con 0x7f46c0048040 2024-04-24T05:46:23.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7f46cd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c0037bb0 0x7f46c004bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7f46cd400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c0037bb0 0x7f46c004bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:49948/0 (socket says 172.21.15.123:49948) 2024-04-24T05:46:23.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7f46cd400640 1 -- 172.21.15.123:0/1180566313 learned_addr learned my addr 172.21.15.123:0/1180566313 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7fbc2ee00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc2806d120 0x7fbc280ca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7fbc2ee00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbc280cbfa0 0x7fbc280cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7f0453600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f044c06bb00 0x7f044c16a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7f0453600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f044c073020 0x7f044c0af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7f0453600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f044c0b5da0 0x7f044c0973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7f0453600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f044c0739b0 con 0x7f044c06bb00 2024-04-24T05:46:23.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7f0453600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f044c0b6180 con 0x7f044c073020 2024-04-24T05:46:23.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7f0453600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f044c073b30 con 0x7f044c0b5da0 2024-04-24T05:46:23.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7f6681e00640 1 --2- 172.21.15.123:0/878877777 >> [v2:172.21.15.123:6808/2060855224,v1:172.21.15.123:6809/2060855224] conn(0x7f665c0760c0 0x7f665c078500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7f218be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840e1310 0x7f2184037720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7f218be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840e1310 0x7f2184037720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38558/0 (socket says 172.21.15.123:38558) 2024-04-24T05:46:23.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7f218be00640 1 -- 172.21.15.123:0/1075353928 learned_addr learned my addr 172.21.15.123:0/1075353928 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7f218aa00640 1 --2- 172.21.15.123:0/1075353928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f218400ab40 0x7f2184035990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7f218b400640 1 --2- 172.21.15.123:0/1075353928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f218400a460 0x7f21840d3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7fbc2ee00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc2815ef30 con 0x7fbc2815e950 2024-04-24T05:46:23.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7fbc2ee00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbc2806d870 con 0x7fbc280cbfa0 2024-04-24T05:46:23.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.929+0000 7fbc2ee00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbc2815fb00 con 0x7fbc2806d120 2024-04-24T05:46:23.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7fd0db200640 1 -- 172.21.15.123:0/1223712608 wait complete. 2024-04-24T05:46:23.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7fbc2da00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbc2815e950 0x7fbc2806ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35556/0 (socket says 172.21.15.123:35556) 2024-04-24T05:46:23.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7fbc2da00640 1 -- 172.21.15.123:0/4157675027 learned_addr learned my addr 172.21.15.123:0/4157675027 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:23.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7f0450c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f044c06bb00 0x7f044c16a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7f218be00640 1 -- 172.21.15.123:0/1075353928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f218400ab40 msgr2=0x7f2184035990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7f218be00640 1 --2- 172.21.15.123:0/1075353928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f218400ab40 0x7f2184035990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7f218be00640 1 -- 172.21.15.123:0/1075353928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f218400a460 msgr2=0x7f21840d3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7f218be00640 1 --2- 172.21.15.123:0/1075353928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f218400a460 0x7f21840d3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7f218be00640 1 -- 172.21.15.123:0/1075353928 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218404d3b0 con 0x7f21840e1310 2024-04-24T05:46:23.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7f0451600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f044c0b5da0 0x7f044c0973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7f0451600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f044c0b5da0 0x7f044c0973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38568/0 (socket says 172.21.15.123:38568) 2024-04-24T05:46:23.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7f0451600640 1 -- 172.21.15.123:0/411285902 learned_addr learned my addr 172.21.15.123:0/411285902 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7f6680a00640 1 --2- 172.21.15.123:0/878877777 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f665c06c940 0x7f665c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7f46cca00640 1 --2- 172.21.15.123:0/1180566313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46c0038550 0x7f46c00dcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7f46cd400640 1 -- 172.21.15.123:0/1180566313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c0048040 msgr2=0x7f46c0036590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T05:46:23.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7f46cd400640 1 --2- 172.21.15.123:0/1180566313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46c0048040 0x7f46c0036590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7f46cd400640 1 -- 172.21.15.123:0/1180566313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46c0038550 msgr2=0x7f46c00dcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7f46cd400640 1 --2- 172.21.15.123:0/1180566313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46c0038550 0x7f46c00dcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7f46cd400640 1 -- 172.21.15.123:0/1180566313 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c004c300 con 0x7f46c0037bb0 2024-04-24T05:46:23.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7f6679600640 1 -- 172.21.15.123:0/878877777 <== osd.0 v2:172.21.15.123:6808/2060855224 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f665c078bd0 con 0x7f665c0760c0 2024-04-24T05:46:23.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7fd0db200640 1 Processor -- start 2024-04-24T05:46:23.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7fbc2d000640 1 --2- 172.21.15.123:0/4157675027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc2806d120 0x7fbc280ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.930+0000 7fbc2e400640 1 --2- 172.21.15.123:0/4157675027 >> v2:172.21.15.155:3300/0 conn(0x7fbc280cbfa0 0x7fbc280cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.928+0000 7fd205400640 1 --2- 172.21.15.123:0/449693257 >> [v2:172.21.15.123:6800/1099836695,v1:172.21.15.123:6801/1099836695] conn(0x7fd1e00760a0 0x7fd1e00784e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7fd1ffe00640 1 --2- 172.21.15.123:0/449693257 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd1e006c920 0x7fd1e006ede0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8033000 tx=0x7fd1f8009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7f044be00640 1 --2- 172.21.15.123:0/411285902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f044c073020 0x7f044c0af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7fd0db200640 1 -- start start 2024-04-24T05:46:23.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7fd1fcc00640 1 -- 172.21.15.123:0/449693257 <== osd.2 v2:172.21.15.123:6800/1099836695 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1f0016ec0 con 0x7fd1e00760a0 2024-04-24T05:46:23.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7f46cca00640 1 --2- 172.21.15.123:0/1180566313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46c0038550 0x7f46c00dcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:23.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.931+0000 7fbc27e00640 1 -- 172.21.15.123:0/4157675027 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 822710519 0 0) 0x7fbc2815ef30 con 0x7fbc2815e950 2024-04-24T05:46:23.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.932+0000 7fbc27e00640 1 -- 172.21.15.123:0/4157675027 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc10003520 con 0x7fbc2815e950 2024-04-24T05:46:23.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.932+0000 7f6680a00640 1 --2- 172.21.15.123:0/878877777 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f665c06c940 0x7f665c06ee00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6674036000 tx=0x7f6674009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.932+0000 7f0451600640 1 -- 172.21.15.123:0/411285902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f044c073020 msgr2=0x7f044c0af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.932+0000 7f0451600640 1 --2- 172.21.15.123:0/411285902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f044c073020 0x7f044c0af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.932+0000 7f0451600640 1 -- 172.21.15.123:0/411285902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f044c06bb00 msgr2=0x7f044c16a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.932+0000 7f0451600640 1 --2- 172.21.15.123:0/411285902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f044c06bb00 0x7f044c16a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.932+0000 7f0451600640 1 -- 172.21.15.123:0/411285902 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044c16aed0 con 0x7f044c0b5da0 2024-04-24T05:46:23.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.932+0000 7f46cd400640 1 --2- 172.21.15.123:0/1180566313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c0037bb0 0x7f46c004bb90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000c00 tx=0x7f46bc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.932+0000 7f218be00640 1 --2- 172.21.15.123:0/1075353928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840e1310 0x7f2184037720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f218c075760 tx=0x7f218c069500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.932+0000 7f7b34c00640 1 -- 172.21.15.123:0/3203296399 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b28180840 con 0x7f7b28005e80 2024-04-24T05:46:23.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.933+0000 7f0451600640 1 --2- 172.21.15.123:0/411285902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f044c0b5da0 0x7f044c0973c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0444007e70 tx=0x7f044400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.933+0000 7f46c5600640 1 -- 172.21.15.123:0/1180566313 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc041070 con 0x7f46c0037bb0 2024-04-24T05:46:23.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.932+0000 7fbc2d000640 1 -- 172.21.15.123:0/4157675027 >> v2:172.21.15.155:3300/0 conn(0x7fbc280cbfa0 msgr2=0x7fbc280cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:23.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.932+0000 7fbc2d000640 1 --2- 172.21.15.123:0/4157675027 >> v2:172.21.15.155:3300/0 conn(0x7fbc280cbfa0 0x7fbc280cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.932+0000 7fbc2d000640 1 -- 172.21.15.123:0/4157675027 >> v1:172.21.15.123:6789/0 conn(0x7fbc2815e950 legacy=0x7fbc2806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.932+0000 7fbc2d000640 1 -- 172.21.15.123:0/4157675027 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2815f1d0 con 0x7fbc2806d120 2024-04-24T05:46:23.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.933+0000 7fd0db200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc0ceac0 0x7fd0dc0b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.933+0000 7f217f400640 1 -- 172.21.15.123:0/1075353928 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c0748a0 con 0x7f21840e1310 2024-04-24T05:46:23.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.933+0000 7f217f400640 1 -- 172.21.15.123:0/1075353928 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218c074e70 con 0x7f21840e1310 2024-04-24T05:46:23.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.933+0000 7f217f400640 1 -- 172.21.15.123:0/1075353928 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c0752a0 con 0x7f21840e1310 2024-04-24T05:46:23.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.934+0000 7fd0db200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0dc0d0640 0x7fd0dc150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.933+0000 7f46ce800640 1 -- 172.21.15.123:0/1180566313 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c004c5e0 con 0x7f46c0037bb0 2024-04-24T05:46:23.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.934+0000 7f2191000640 1 -- 172.21.15.123:0/1075353928 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218404b6a0 con 0x7f21840e1310 2024-04-24T05:46:23.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.933+0000 7f46ce800640 1 -- 172.21.15.123:0/1180566313 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c004cba0 con 0x7f46c0037bb0 2024-04-24T05:46:23.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.934+0000 7f2191000640 1 -- 172.21.15.123:0/1075353928 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f218404bc20 con 0x7f21840e1310 2024-04-24T05:46:23.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.934+0000 7fd0db200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0dc15e680 0x7fd0dc150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.934+0000 7fbc2d000640 1 --2- 172.21.15.123:0/4157675027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc2806d120 0x7fbc280ca670 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbc140097a0 tx=0x7fbc1402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c2a280296ab338a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.934+0000 7f0448c00640 1 -- 172.21.15.123:0/411285902 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444014620 con 0x7f044c0b5da0 2024-04-24T05:46:23.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.934+0000 7f7b26000640 1 -- 172.21.15.123:0/3203296399 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b2001a070 con 0x7f7b28005e80 2024-04-24T05:46:23.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.935+0000 7fd0db200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc06d4c0 con 0x7fd0dc15e680 2024-04-24T05:46:23.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.934+0000 7fd0d9e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc0ceac0 0x7fd0dc0b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.935+0000 7fd0d9e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc0ceac0 0x7fd0dc0b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38582/0 (socket says 172.21.15.123:38582) 2024-04-24T05:46:23.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.935+0000 7fd0d9e00640 1 -- 172.21.15.123:0/1425795536 learned_addr learned my addr 172.21.15.123:0/1425795536 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.935+0000 7fd0db200640 1 -- 172.21.15.123:0/1425795536 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc15ea60 con 0x7fd0dc0d0640 2024-04-24T05:46:23.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.935+0000 7fd0d9400640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0dc0d0640 0x7fd0dc150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.935+0000 7fd0db200640 1 -- 172.21.15.123:0/1425795536 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd0dc0b7db0 con 0x7fd0dc0ceac0 2024-04-24T05:46:23.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.934+0000 7f0448c00640 1 -- 172.21.15.123:0/411285902 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0444019070 con 0x7f044c0b5da0 2024-04-24T05:46:23.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.934+0000 7f0448c00640 1 -- 172.21.15.123:0/411285902 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444014a20 con 0x7f044c0b5da0 2024-04-24T05:46:23.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.935+0000 7fd0da800640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0dc15e680 0x7fd0dc150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.935+0000 7f2191000640 1 -- 172.21.15.123:0/1075353928 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f21840057e0 con 0x7f21840e1310 2024-04-24T05:46:23.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.935+0000 7f46c5600640 1 -- 172.21.15.123:0/1180566313 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46bc032440 con 0x7f46c0037bb0 2024-04-24T05:46:23.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.935+0000 7f46c5600640 1 -- 172.21.15.123:0/1180566313 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc037040 con 0x7f46c0037bb0 2024-04-24T05:46:23.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.935+0000 7f46ce800640 1 -- 172.21.15.123:0/1180566313 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46c004cf80 con 0x7f46c0037bb0 2024-04-24T05:46:23.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.936+0000 7f0453600640 1 -- 172.21.15.123:0/411285902 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044c0af2c0 con 0x7f044c0b5da0 2024-04-24T05:46:23.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.936+0000 7f0453600640 1 -- 172.21.15.123:0/411285902 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f044c0aea20 con 0x7f044c0b5da0 2024-04-24T05:46:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.936+0000 7fd0da800640 1 -- 172.21.15.123:0/1425795536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc0ceac0 msgr2=0x7fd0dc0b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.937+0000 7fd0da800640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0dc0ceac0 0x7fd0dc0b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.937+0000 7fd0da800640 1 -- 172.21.15.123:0/1425795536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0dc0d0640 msgr2=0x7fd0dc150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.937+0000 7fd0da800640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0dc0d0640 0x7fd0dc150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.937+0000 7fd0da800640 1 -- 172.21.15.123:0/1425795536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0dc0caf30 con 0x7fd0dc15e680 2024-04-24T05:46:23.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.937+0000 7fd0d9400640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0dc0d0640 0x7fd0dc150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:46:23.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.937+0000 7fbc27e00640 1 -- 172.21.15.123:0/4157675027 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14037070 con 0x7fbc2806d120 2024-04-24T05:46:23.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.937+0000 7fbc27e00640 1 -- 172.21.15.123:0/4157675027 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc14032440 con 0x7fbc2806d120 2024-04-24T05:46:23.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.937+0000 7fbc27e00640 1 -- 172.21.15.123:0/4157675027 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1402ae60 con 0x7fbc2806d120 2024-04-24T05:46:23.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.937+0000 7fd0da800640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0dc15e680 0x7fd0dc150770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800b3e0 tx=0x7fd0c800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.938+0000 7fd0d2000640 1 -- 172.21.15.123:0/1425795536 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8015020 con 0x7fd0dc15e680 2024-04-24T05:46:23.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.938+0000 7fbc2ee00640 1 -- 172.21.15.123:0/4157675027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc2806d120 msgr2=0x7fbc280ca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.938+0000 7fbc2ee00640 1 --2- 172.21.15.123:0/4157675027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc2806d120 0x7fbc280ca670 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbc140097a0 tx=0x7fbc1402a110 comp rx=0 tx=0).stop 2024-04-24T05:46:23.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.938+0000 7fd0db200640 1 -- 172.21.15.123:0/1425795536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0dc0cb210 con 0x7fd0dc15e680 2024-04-24T05:46:23.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.938+0000 7fbc2ee00640 1 -- 172.21.15.123:0/4157675027 shutdown_connections 2024-04-24T05:46:23.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.938+0000 7fbc2ee00640 1 --2- 172.21.15.123:0/4157675027 >> v2:172.21.15.155:3300/0 conn(0x7fbc280cbfa0 0x7fbc280cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.938+0000 7fbc2ee00640 1 --2- 172.21.15.123:0/4157675027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc2806d120 0x7fbc280ca670 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.938+0000 7fbc2ee00640 1 -- 172.21.15.123:0/4157675027 >> 172.21.15.123:0/4157675027 conn(0x7fbc280b1b10 msgr2=0x7fbc280b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:23.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.940+0000 7f7b26000640 1 --2- 172.21.15.123:0/3203296399 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7b1006caf0 0x7f7b1006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.939+0000 7f0453600640 1 -- 172.21.15.123:0/411285902 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f044c068a20 con 0x7f044c0b5da0 2024-04-24T05:46:23.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.939+0000 7fd0db200640 1 -- 172.21.15.123:0/1425795536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0dc1f1d50 con 0x7fd0dc15e680 2024-04-24T05:46:23.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.940+0000 7fd0c7400640 1 -- 172.21.15.123:0/1425795536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd0b4001030 con 0x7fd0dc15e680 2024-04-24T05:46:23.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.939+0000 7f46c5600640 1 -- 172.21.15.123:0/1180566313 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46bc044070 con 0x7f46c0037bb0 2024-04-24T05:46:23.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.941+0000 7f46c5600640 1 --2- 172.21.15.123:0/1180566313 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f46a806c8f0 0x7f46a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.940+0000 7f7b26000640 1 -- 172.21.15.123:0/3203296399 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7b2008ae40 con 0x7f7b28005e80 2024-04-24T05:46:23.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.941+0000 7f7b26000640 1 --2- 172.21.15.123:0/3203296399 >> [v2:172.21.15.123:6815/4191632066,v1:172.21.15.123:6817/4191632066] conn(0x7f7b10076270 0x7f7b100786b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.941+0000 7fbc2ee00640 1 -- 172.21.15.123:0/4157675027 shutdown_connections 2024-04-24T05:46:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.941+0000 7f46c5600640 1 -- 172.21.15.123:0/1180566313 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f46bc0ace30 con 0x7f46c0037bb0 2024-04-24T05:46:23.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.941+0000 7fbc2ee00640 1 -- 172.21.15.123:0/4157675027 wait complete. 2024-04-24T05:46:23.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.941+0000 7f7b2d400640 1 --2- 172.21.15.123:0/3203296399 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7b1006caf0 0x7f7b1006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.941+0000 7fd0d2000640 1 -- 172.21.15.123:0/1425795536 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b4001030 con 0x7fd0dc15e680 2024-04-24T05:46:23.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.941+0000 7fd0d2000640 1 -- 172.21.15.123:0/1425795536 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8011a30 con 0x7fd0dc15e680 2024-04-24T05:46:23.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.941+0000 7f46cca00640 1 --2- 172.21.15.123:0/1180566313 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f46a806c8f0 0x7f46a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.941+0000 7f46c5600640 1 --2- 172.21.15.123:0/1180566313 >> [v2:172.21.15.155:6808/570528717,v1:172.21.15.155:6809/570528717] conn(0x7f46a8076060 0x7f46a80784a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.941+0000 7f46c5600640 1 -- 172.21.15.123:0/1180566313 --> [v2:172.21.15.155:6808/570528717,v1:172.21.15.155:6809/570528717] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46bc048e50 con 0x7f46a8076060 2024-04-24T05:46:23.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.941+0000 7f46c5600640 1 -- 172.21.15.123:0/1180566313 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f46bc0ad220 con 0x7f46c0037bb0 2024-04-24T05:46:23.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.938+0000 7f217f400640 1 -- 172.21.15.123:0/1075353928 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f218c093400 con 0x7f21840e1310 2024-04-24T05:46:23.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.940+0000 7f217f400640 1 --2- 172.21.15.123:0/1075353928 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f216c06c940 0x7f216c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.940+0000 7f217f400640 1 -- 172.21.15.123:0/1075353928 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f218c107330 con 0x7f21840e1310 2024-04-24T05:46:23.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.940+0000 7f217f400640 1 --2- 172.21.15.123:0/1075353928 >> [v2:172.21.15.155:6824/2153713758,v1:172.21.15.155:6825/2153713758] conn(0x7f216c0760c0 0x7f216c078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.940+0000 7f217f400640 1 -- 172.21.15.123:0/1075353928 --> [v2:172.21.15.155:6824/2153713758,v1:172.21.15.155:6825/2153713758] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f216c078bd0 con 0x7f216c0760c0 2024-04-24T05:46:23.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.940+0000 7f217f400640 1 -- 172.21.15.123:0/1075353928 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f218c107810 con 0x7f21840e1310 2024-04-24T05:46:23.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.941+0000 7f218b400640 1 --2- 172.21.15.123:0/1075353928 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f216c06c940 0x7f216c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.942+0000 7f7b26000640 1 -- 172.21.15.123:0/3203296399 --> [v2:172.21.15.123:6815/4191632066,v1:172.21.15.123:6817/4191632066] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b10078d80 con 0x7f7b10076270 2024-04-24T05:46:23.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.942+0000 7f7b26000640 1 -- 172.21.15.123:0/3203296399 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7b2008d4a0 con 0x7f7b28005e80 2024-04-24T05:46:23.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.942+0000 7f7b2d400640 1 --2- 172.21.15.123:0/3203296399 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7b1006caf0 0x7f7b1006efb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7b28047e20 tx=0x7f7b1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.942+0000 7f46cde00640 1 --2- 172.21.15.123:0/1180566313 >> [v2:172.21.15.155:6808/570528717,v1:172.21.15.155:6809/570528717] conn(0x7f46a8076060 0x7f46a80784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.943+0000 7f46cca00640 1 --2- 172.21.15.123:0/1180566313 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f46a806c8f0 0x7f46a806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f46c0052d20 tx=0x7f46b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.943+0000 7f218aa00640 1 --2- 172.21.15.123:0/1075353928 >> [v2:172.21.15.155:6824/2153713758,v1:172.21.15.155:6825/2153713758] conn(0x7f216c0760c0 0x7f216c078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.943+0000 7f46cde00640 1 --2- 172.21.15.123:0/1180566313 >> [v2:172.21.15.155:6808/570528717,v1:172.21.15.155:6809/570528717] conn(0x7f46a8076060 0x7f46a80784a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.943+0000 7f7b2e800640 1 --2- 172.21.15.123:0/3203296399 >> [v2:172.21.15.123:6815/4191632066,v1:172.21.15.123:6817/4191632066] conn(0x7f7b10076270 0x7f7b100786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.943+0000 7f218b400640 1 --2- 172.21.15.123:0/1075353928 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f216c06c940 0x7f216c06ee00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2180034000 tx=0x7f218002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.943+0000 7fbc2ee00640 1 Processor -- start 2024-04-24T05:46:23.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.944+0000 7fbc2ee00640 1 -- start start 2024-04-24T05:46:23.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.944+0000 7f7b2e800640 1 --2- 172.21.15.123:0/3203296399 >> [v2:172.21.15.123:6815/4191632066,v1:172.21.15.123:6817/4191632066] conn(0x7f7b10076270 0x7f7b100786b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.944+0000 7f0448c00640 1 -- 172.21.15.123:0/411285902 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0444014bc0 con 0x7f044c0b5da0 2024-04-24T05:46:23.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.944+0000 7f218aa00640 1 --2- 172.21.15.123:0/1075353928 >> [v2:172.21.15.155:6824/2153713758,v1:172.21.15.155:6825/2153713758] conn(0x7f216c0760c0 0x7f216c078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:23.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.944+0000 7fd0d2000640 1 -- 172.21.15.123:0/1425795536 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0c8018020 con 0x7fd0dc15e680 2024-04-24T05:46:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.945+0000 7fbc2ee00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc2806d120 0x7fbc28153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.945+0000 7fbc2ee00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc280cbfa0 0x7fbc280aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.945+0000 7fbc2ee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc2815e950 0x7fbc2814ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.945+0000 7fbc2ee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc280cc680 con 0x7fbc2815e950 2024-04-24T05:46:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.945+0000 7fbc2ee00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2815f790 con 0x7fbc280cbfa0 2024-04-24T05:46:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.945+0000 7fbc2ee00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbc280cc800 con 0x7fbc2806d120 2024-04-24T05:46:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.945+0000 7fbc2d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc280cbfa0 0x7fbc280aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7fbc2d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc280cbfa0 0x7fbc280aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:49998/0 (socket says 172.21.15.123:49998) 2024-04-24T05:46:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7fbc2d000640 1 -- 172.21.15.123:0/197058486 learned_addr learned my addr 172.21.15.123:0/197058486 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:23.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7fbc2da00640 1 --2- 172.21.15.123:0/197058486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc2806d120 0x7fbc28153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7f0448c00640 1 --2- 172.21.15.123:0/411285902 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f042c06c970 0x7f042c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7fd0d2000640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd0bc06c8f0 0x7fd0bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7f0448c00640 1 -- 172.21.15.123:0/411285902 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f044408c5e0 con 0x7f044c0b5da0 2024-04-24T05:46:23.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7f0448c00640 1 --2- 172.21.15.123:0/411285902 >> [v2:172.21.15.123:6824/315591476,v1:172.21.15.123:6825/315591476] conn(0x7f042c0760f0 0x7f042c078530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7f0448c00640 1 -- 172.21.15.123:0/411285902 --> [v2:172.21.15.123:6824/315591476,v1:172.21.15.123:6825/315591476] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f044400ce50 con 0x7f042c0760f0 2024-04-24T05:46:23.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7f0448c00640 1 -- 172.21.15.123:0/411285902 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0444093050 con 0x7f044c0b5da0 2024-04-24T05:46:23.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7fbc2d000640 1 -- 172.21.15.123:0/197058486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc2806d120 msgr2=0x7fbc28153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7fbc2d000640 1 --2- 172.21.15.123:0/197058486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc2806d120 0x7fbc28153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7fd0d2000640 1 -- 172.21.15.123:0/1425795536 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd0c808c100 con 0x7fd0dc15e680 2024-04-24T05:46:23.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7fbc2d000640 1 -- 172.21.15.123:0/197058486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc2815e950 msgr2=0x7fbc2814ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:23.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7fbc2d000640 1 --2- 172.21.15.123:0/197058486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc2815e950 0x7fbc2814ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:23.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.946+0000 7fbc2d000640 1 -- 172.21.15.123:0/197058486 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc28153230 con 0x7fbc280cbfa0 2024-04-24T05:46:23.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.947+0000 7f7b26000640 1 -- 172.21.15.123:0/3203296399 <== osd.1 v2:172.21.15.123:6815/4191632066 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b10078d80 con 0x7f7b10076270 2024-04-24T05:46:23.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.947+0000 7fd0d2000640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.155:6800/1404325606,v1:172.21.15.155:6801/1404325606] conn(0x7fd0bc076070 0x7fd0bc0784b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:23.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.947+0000 7fd0d2000640 1 -- 172.21.15.123:0/1425795536 --> [v2:172.21.15.155:6800/1404325606,v1:172.21.15.155:6801/1404325606] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd0bc078b80 con 0x7fd0bc076070 2024-04-24T05:46:23.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.947+0000 7fd0d2000640 1 -- 172.21.15.123:0/1425795536 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd0c808c4f0 con 0x7fd0dc15e680 2024-04-24T05:46:23.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.947+0000 7f0450c00640 1 --2- 172.21.15.123:0/411285902 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f042c06c970 0x7f042c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.947+0000 7f46c5600640 1 -- 172.21.15.123:0/1180566313 <== osd.6 v2:172.21.15.155:6808/570528717 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46bc048e50 con 0x7f46a8076060 2024-04-24T05:46:23.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.947+0000 7f044be00640 1 --2- 172.21.15.123:0/411285902 >> [v2:172.21.15.123:6824/315591476,v1:172.21.15.123:6825/315591476] conn(0x7f042c0760f0 0x7f042c078530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.948+0000 7fd0d9400640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.155:6800/1404325606,v1:172.21.15.155:6801/1404325606] conn(0x7fd0bc076070 0x7fd0bc0784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.948+0000 7fd0d9e00640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd0bc06c8f0 0x7fd0bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:23.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.948+0000 7f217f400640 1 -- 172.21.15.123:0/1075353928 <== osd.5 v2:172.21.15.155:6824/2153713758 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f216c078bd0 con 0x7f216c0760c0 2024-04-24T05:46:23.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.949+0000 7fbc2d000640 1 --2- 172.21.15.123:0/197058486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc280cbfa0 0x7fbc280aee80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc1402a700 tx=0x7fbc140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.949+0000 7fd0d9e00640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd0bc06c8f0 0x7fd0bc06edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009c90 tx=0x7fd0d4009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.948+0000 7f044be00640 1 --2- 172.21.15.123:0/411285902 >> [v2:172.21.15.123:6824/315591476,v1:172.21.15.123:6825/315591476] conn(0x7f042c0760f0 0x7f042c078530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.949+0000 7f0450c00640 1 --2- 172.21.15.123:0/411285902 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f042c06c970 0x7f042c06ee30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f04400011e0 tx=0x7f0440009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.949+0000 7fbc25600640 1 -- 172.21.15.123:0/197058486 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14037050 con 0x7fbc280cbfa0 2024-04-24T05:46:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.950+0000 7fbc25600640 1 -- 172.21.15.123:0/197058486 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc14043070 con 0x7fbc280cbfa0 2024-04-24T05:46:24.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.950+0000 7fbc25600640 1 -- 172.21.15.123:0/197058486 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1403cb30 con 0x7fbc280cbfa0 2024-04-24T05:46:24.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.950+0000 7fd0d9400640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.155:6800/1404325606,v1:172.21.15.155:6801/1404325606] conn(0x7fd0bc076070 0x7fd0bc0784b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.950+0000 7f0448c00640 1 -- 172.21.15.123:0/411285902 <== osd.3 v2:172.21.15.123:6824/315591476 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f044400ce50 con 0x7f042c0760f0 2024-04-24T05:46:24.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.950+0000 7fbc2ee00640 1 -- 172.21.15.123:0/197058486 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc28153510 con 0x7fbc280cbfa0 2024-04-24T05:46:24.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.951+0000 7fbc2ee00640 1 -- 172.21.15.123:0/197058486 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc280af3c0 con 0x7fbc280cbfa0 2024-04-24T05:46:24.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.953+0000 7fbc2ee00640 1 -- 172.21.15.123:0/197058486 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbc280cfe70 con 0x7fbc280cbfa0 2024-04-24T05:46:24.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.953+0000 7fd0d2000640 1 -- 172.21.15.123:0/1425795536 <== osd.4 v2:172.21.15.155:6800/1404325606 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd0bc078b80 con 0x7fd0bc076070 2024-04-24T05:46:24.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.955+0000 7fbc25600640 1 -- 172.21.15.123:0/197058486 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc1403ccd0 con 0x7fbc280cbfa0 2024-04-24T05:46:24.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.956+0000 7fbc25600640 1 --2- 172.21.15.123:0/197058486 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbc0806c9c0 0x7fbc0806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.956+0000 7fbc25600640 1 -- 172.21.15.123:0/197058486 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbc140acac0 con 0x7fbc280cbfa0 2024-04-24T05:46:24.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.957+0000 7fbc25600640 1 --2- 172.21.15.123:0/197058486 >> [v2:172.21.15.155:6816/1974539526,v1:172.21.15.155:6817/1974539526] conn(0x7fbc080760b0 0x7fbc080784f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.957+0000 7fbc25600640 1 -- 172.21.15.123:0/197058486 --> [v2:172.21.15.155:6816/1974539526,v1:172.21.15.155:6817/1974539526] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbc14046e80 con 0x7fbc080760b0 2024-04-24T05:46:24.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.955+0000 7f666ea00640 1 -- 172.21.15.123:0/878877777 --> [v2:172.21.15.123:6808/2060855224,v1:172.21.15.123:6809/2060855224] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6650002dd0 con 0x7f665c0760c0 2024-04-24T05:46:24.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.956+0000 7f6679600640 1 -- 172.21.15.123:0/878877777 <== osd.0 v2:172.21.15.123:6808/2060855224 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6650002dd0 con 0x7f665c0760c0 2024-04-24T05:46:24.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.957+0000 7f666ea00640 1 -- 172.21.15.123:0/878877777 >> [v2:172.21.15.123:6808/2060855224,v1:172.21.15.123:6809/2060855224] conn(0x7f665c0760c0 msgr2=0x7f665c078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.957+0000 7f666ea00640 1 --2- 172.21.15.123:0/878877777 >> [v2:172.21.15.123:6808/2060855224,v1:172.21.15.123:6809/2060855224] conn(0x7f665c0760c0 0x7f665c078500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.958+0000 7f666ea00640 1 -- 172.21.15.123:0/878877777 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f665c06c940 msgr2=0x7f665c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.958+0000 7f666ea00640 1 --2- 172.21.15.123:0/878877777 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f665c06c940 0x7f665c06ee00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6674036000 tx=0x7f6674009000 comp rx=0 tx=0).stop 2024-04-24T05:46:24.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.958+0000 7f666ea00640 1 -- 172.21.15.123:0/878877777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f667c06ba00 msgr2=0x7f667c15a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.958+0000 7f666ea00640 1 --2- 172.21.15.123:0/878877777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f667c06ba00 0x7f667c15a670 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f667000e870 tx=0x7f667000ed30 comp rx=0 tx=0).stop 2024-04-24T05:46:24.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.958+0000 7f6681e00640 1 -- 172.21.15.123:0/878877777 reap_dead start 2024-04-24T05:46:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.957+0000 7fbc2da00640 1 --2- 172.21.15.123:0/197058486 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbc0806c9c0 0x7fbc0806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.958+0000 7fbc25600640 1 -- 172.21.15.123:0/197058486 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc14076850 con 0x7fbc280cbfa0 2024-04-24T05:46:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.958+0000 7fbc2e400640 1 --2- 172.21.15.123:0/197058486 >> [v2:172.21.15.155:6816/1974539526,v1:172.21.15.155:6817/1974539526] conn(0x7fbc080760b0 0x7fbc080784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.959+0000 7fbc2da00640 1 --2- 172.21.15.123:0/197058486 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbc0806c9c0 0x7fbc0806ee80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c001560 tx=0x7fbc1c0019f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.961+0000 7f666ea00640 1 -- 172.21.15.123:0/878877777 shutdown_connections 2024-04-24T05:46:24.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.961+0000 7f666ea00640 1 -- 172.21.15.123:0/878877777 >> 172.21.15.123:0/878877777 conn(0x7f667c0d0050 msgr2=0x7f667c15b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.961+0000 7f666ea00640 1 -- 172.21.15.123:0/878877777 shutdown_connections 2024-04-24T05:46:24.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.961+0000 7f666ea00640 1 -- 172.21.15.123:0/878877777 wait complete. 2024-04-24T05:46:24.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.961+0000 7fbc2e400640 1 --2- 172.21.15.123:0/197058486 >> [v2:172.21.15.155:6816/1974539526,v1:172.21.15.155:6817/1974539526] conn(0x7fbc080760b0 0x7fbc080784f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.962+0000 7fd207400640 1 -- 172.21.15.123:0/449693257 --> [v2:172.21.15.123:6800/1099836695,v1:172.21.15.123:6801/1099836695] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd200075b70 con 0x7fd1e00760a0 2024-04-24T05:46:24.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.963+0000 7fd1fcc00640 1 -- 172.21.15.123:0/449693257 <== osd.2 v2:172.21.15.123:6800/1099836695 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd200075b70 con 0x7fd1e00760a0 2024-04-24T05:46:24.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.964+0000 7fbc25600640 1 -- 172.21.15.123:0/197058486 <== osd.7 v2:172.21.15.155:6816/1974539526 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbc14046e80 con 0x7fbc080760b0 2024-04-24T05:46:24.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.965+0000 7fd207400640 1 -- 172.21.15.123:0/449693257 >> [v2:172.21.15.123:6800/1099836695,v1:172.21.15.123:6801/1099836695] conn(0x7fd1e00760a0 msgr2=0x7fd1e00784e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.965+0000 7fd207400640 1 --2- 172.21.15.123:0/449693257 >> [v2:172.21.15.123:6800/1099836695,v1:172.21.15.123:6801/1099836695] conn(0x7fd1e00760a0 0x7fd1e00784e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.966+0000 7fd207400640 1 -- 172.21.15.123:0/449693257 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd1e006c920 msgr2=0x7fd1e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.966+0000 7fd207400640 1 --2- 172.21.15.123:0/449693257 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd1e006c920 0x7fd1e006ede0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8033000 tx=0x7fd1f8009000 comp rx=0 tx=0).stop 2024-04-24T05:46:24.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.966+0000 7fd207400640 1 -- 172.21.15.123:0/449693257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd20006bf40 msgr2=0x7fd2000c7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.966+0000 7fd207400640 1 --2- 172.21.15.123:0/449693257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd20006bf40 0x7fd2000c7c80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000b4d0 tx=0x7fd1f000b990 comp rx=0 tx=0).stop 2024-04-24T05:46:24.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.966+0000 7fd205400640 1 -- 172.21.15.123:0/449693257 reap_dead start 2024-04-24T05:46:24.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.966+0000 7fd207400640 1 -- 172.21.15.123:0/449693257 shutdown_connections 2024-04-24T05:46:24.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.966+0000 7fd207400640 1 -- 172.21.15.123:0/449693257 >> 172.21.15.123:0/449693257 conn(0x7fd2000d0050 msgr2=0x7fd20006eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.966+0000 7fd207400640 1 -- 172.21.15.123:0/449693257 shutdown_connections 2024-04-24T05:46:24.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.966+0000 7fd207400640 1 -- 172.21.15.123:0/449693257 wait complete. 2024-04-24T05:46:24.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.977+0000 7f2191000640 1 -- 172.21.15.123:0/1075353928 --> [v2:172.21.15.155:6824/2153713758,v1:172.21.15.155:6825/2153713758] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f21840578f0 con 0x7f216c0760c0 2024-04-24T05:46:24.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.978+0000 7f217f400640 1 -- 172.21.15.123:0/1075353928 <== osd.5 v2:172.21.15.155:6824/2153713758 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f21840578f0 con 0x7f216c0760c0 2024-04-24T05:46:24.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.979+0000 7f2191000640 1 -- 172.21.15.123:0/1075353928 >> [v2:172.21.15.155:6824/2153713758,v1:172.21.15.155:6825/2153713758] conn(0x7f216c0760c0 msgr2=0x7f216c078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.981+0000 7f7b34c00640 1 -- 172.21.15.123:0/3203296399 --> [v2:172.21.15.123:6815/4191632066,v1:172.21.15.123:6817/4191632066] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7b28005380 con 0x7f7b10076270 2024-04-24T05:46:24.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.979+0000 7f2191000640 1 --2- 172.21.15.123:0/1075353928 >> [v2:172.21.15.155:6824/2153713758,v1:172.21.15.155:6825/2153713758] conn(0x7f216c0760c0 0x7f216c078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.979+0000 7f2191000640 1 -- 172.21.15.123:0/1075353928 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f216c06c940 msgr2=0x7f216c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.979+0000 7f2191000640 1 --2- 172.21.15.123:0/1075353928 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f216c06c940 0x7f216c06ee00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2180034000 tx=0x7f218002e000 comp rx=0 tx=0).stop 2024-04-24T05:46:24.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.979+0000 7f2191000640 1 -- 172.21.15.123:0/1075353928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840e1310 msgr2=0x7f2184037720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.979+0000 7f2191000640 1 --2- 172.21.15.123:0/1075353928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840e1310 0x7f2184037720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f218c075760 tx=0x7f218c069500 comp rx=0 tx=0).stop 2024-04-24T05:46:24.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.979+0000 7f218be00640 1 -- 172.21.15.123:0/1075353928 reap_dead start 2024-04-24T05:46:24.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.979+0000 7f2191000640 1 -- 172.21.15.123:0/1075353928 shutdown_connections 2024-04-24T05:46:24.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.979+0000 7f2191000640 1 -- 172.21.15.123:0/1075353928 >> 172.21.15.123:0/1075353928 conn(0x7f2184056cd0 msgr2=0x7f2184050fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.981+0000 7f7b26000640 1 -- 172.21.15.123:0/3203296399 <== osd.1 v2:172.21.15.123:6815/4191632066 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7b28005380 con 0x7f7b10076270 2024-04-24T05:46:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.982+0000 7f2191000640 1 -- 172.21.15.123:0/1075353928 shutdown_connections 2024-04-24T05:46:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.982+0000 7f2191000640 1 -- 172.21.15.123:0/1075353928 wait complete. 2024-04-24T05:46:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.982+0000 7f7b34c00640 1 -- 172.21.15.123:0/3203296399 >> [v2:172.21.15.123:6815/4191632066,v1:172.21.15.123:6817/4191632066] conn(0x7f7b10076270 msgr2=0x7f7b100786b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.982+0000 7f7b34c00640 1 --2- 172.21.15.123:0/3203296399 >> [v2:172.21.15.123:6815/4191632066,v1:172.21.15.123:6817/4191632066] conn(0x7f7b10076270 0x7f7b100786b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.012 INFO:teuthology.orchestra.run.smithi123.stdout:55834574859 2024-04-24T05:46:24.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.984+0000 7f7b34c00640 1 -- 172.21.15.123:0/3203296399 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7b1006caf0 msgr2=0x7f7b1006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.984+0000 7f7b34c00640 1 --2- 172.21.15.123:0/3203296399 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7b1006caf0 0x7f7b1006efb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7b28047e20 tx=0x7f7b1c009040 comp rx=0 tx=0).stop 2024-04-24T05:46:24.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.984+0000 7f7b34c00640 1 -- 172.21.15.123:0/3203296399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b28005e80 msgr2=0x7f7b28180c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.984+0000 7f7b34c00640 1 --2- 172.21.15.123:0/3203296399 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b28005e80 0x7f7b28180c40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b20001eb0 tx=0x7f7b2000c9b0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.013 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T05:46:24.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.986+0000 7f46ce800640 1 -- 172.21.15.123:0/1180566313 --> [v2:172.21.15.155:6808/570528717,v1:172.21.15.155:6809/570528717] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46c004b3d0 con 0x7f46a8076060 2024-04-24T05:46:24.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.983+0000 7f0453600640 1 -- 172.21.15.123:0/411285902 --> [v2:172.21.15.123:6824/315591476,v1:172.21.15.123:6825/315591476] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f044c0b0020 con 0x7f042c0760f0 2024-04-24T05:46:24.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.986+0000 7f7b2e800640 1 -- 172.21.15.123:0/3203296399 reap_dead start 2024-04-24T05:46:24.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.986+0000 7f7b34c00640 1 -- 172.21.15.123:0/3203296399 shutdown_connections 2024-04-24T05:46:24.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.986+0000 7f7b34c00640 1 -- 172.21.15.123:0/3203296399 >> 172.21.15.123:0/3203296399 conn(0x7f7b28009fa0 msgr2=0x7f7b2804f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.986+0000 7f7b34c00640 1 -- 172.21.15.123:0/3203296399 shutdown_connections 2024-04-24T05:46:24.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.987+0000 7f7b34c00640 1 -- 172.21.15.123:0/3203296399 wait complete. 2024-04-24T05:46:24.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.987+0000 7f0448c00640 1 -- 172.21.15.123:0/411285902 <== osd.3 v2:172.21.15.123:6824/315591476 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f044c0b0020 con 0x7f042c0760f0 2024-04-24T05:46:24.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.987+0000 7f46c5600640 1 -- 172.21.15.123:0/1180566313 <== osd.6 v2:172.21.15.155:6808/570528717 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f46c004b3d0 con 0x7f46a8076060 2024-04-24T05:46:24.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.988+0000 7f46ce800640 1 -- 172.21.15.123:0/1180566313 >> [v2:172.21.15.155:6808/570528717,v1:172.21.15.155:6809/570528717] conn(0x7f46a8076060 msgr2=0x7f46a80784a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.989+0000 7fd0c7400640 1 -- 172.21.15.123:0/1425795536 --> [v2:172.21.15.155:6800/1404325606,v1:172.21.15.155:6801/1404325606] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd0b4002dd0 con 0x7fd0bc076070 2024-04-24T05:46:24.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.988+0000 7f46ce800640 1 --2- 172.21.15.123:0/1180566313 >> [v2:172.21.15.155:6808/570528717,v1:172.21.15.155:6809/570528717] conn(0x7f46a8076060 0x7f46a80784a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.989+0000 7f46ce800640 1 -- 172.21.15.123:0/1180566313 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f46a806c8f0 msgr2=0x7f46a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.989+0000 7f46ce800640 1 --2- 172.21.15.123:0/1180566313 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f46a806c8f0 0x7f46a806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f46c0052d20 tx=0x7f46b4008040 comp rx=0 tx=0).stop 2024-04-24T05:46:24.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.989+0000 7f46ce800640 1 -- 172.21.15.123:0/1180566313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c0037bb0 msgr2=0x7f46c004bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.989+0000 7f46ce800640 1 --2- 172.21.15.123:0/1180566313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46c0037bb0 0x7f46c004bb90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000c00 tx=0x7f46bc0046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.989+0000 7f46cde00640 1 -- 172.21.15.123:0/1180566313 reap_dead start 2024-04-24T05:46:24.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.989+0000 7f46ce800640 1 -- 172.21.15.123:0/1180566313 shutdown_connections 2024-04-24T05:46:24.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.989+0000 7f46ce800640 1 -- 172.21.15.123:0/1180566313 >> 172.21.15.123:0/1180566313 conn(0x7f46c0009fb0 msgr2=0x7f46c00057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.990+0000 7f46ce800640 1 -- 172.21.15.123:0/1180566313 shutdown_connections 2024-04-24T05:46:24.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.990+0000 7f46ce800640 1 -- 172.21.15.123:0/1180566313 wait complete. 2024-04-24T05:46:24.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.990+0000 7f0453600640 1 -- 172.21.15.123:0/411285902 >> [v2:172.21.15.123:6824/315591476,v1:172.21.15.123:6825/315591476] conn(0x7f042c0760f0 msgr2=0x7f042c078530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.990+0000 7f0453600640 1 --2- 172.21.15.123:0/411285902 >> [v2:172.21.15.123:6824/315591476,v1:172.21.15.123:6825/315591476] conn(0x7f042c0760f0 0x7f042c078530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.990+0000 7fd0d2000640 1 -- 172.21.15.123:0/1425795536 <== osd.4 v2:172.21.15.155:6800/1404325606 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd0b4002dd0 con 0x7fd0bc076070 2024-04-24T05:46:24.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.990+0000 7f0453600640 1 -- 172.21.15.123:0/411285902 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f042c06c970 msgr2=0x7f042c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.990+0000 7f0453600640 1 --2- 172.21.15.123:0/411285902 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f042c06c970 0x7f042c06ee30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f04400011e0 tx=0x7f0440009040 comp rx=0 tx=0).stop 2024-04-24T05:46:24.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.990+0000 7fd0c7400640 1 -- 172.21.15.123:0/1425795536 >> [v2:172.21.15.155:6800/1404325606,v1:172.21.15.155:6801/1404325606] conn(0x7fd0bc076070 msgr2=0x7fd0bc0784b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.990+0000 7fd0c7400640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.155:6800/1404325606,v1:172.21.15.155:6801/1404325606] conn(0x7fd0bc076070 0x7fd0bc0784b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.991+0000 7f0453600640 1 -- 172.21.15.123:0/411285902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f044c0b5da0 msgr2=0x7f044c0973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.991+0000 7f0453600640 1 --2- 172.21.15.123:0/411285902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f044c0b5da0 0x7f044c0973c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0444007e70 tx=0x7f044400e4b0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.991+0000 7f0451600640 1 -- 172.21.15.123:0/411285902 reap_dead start 2024-04-24T05:46:24.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.990+0000 7fd0c7400640 1 -- 172.21.15.123:0/1425795536 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd0bc06c8f0 msgr2=0x7fd0bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.991+0000 7fd0c7400640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd0bc06c8f0 0x7fd0bc06edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009c90 tx=0x7fd0d4009000 comp rx=0 tx=0).stop 2024-04-24T05:46:24.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.991+0000 7fd0c7400640 1 -- 172.21.15.123:0/1425795536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0dc15e680 msgr2=0x7fd0dc150770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.991+0000 7fd0c7400640 1 --2- 172.21.15.123:0/1425795536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0dc15e680 0x7fd0dc150770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800b3e0 tx=0x7fd0c800b8a0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.991+0000 7fd0da800640 1 -- 172.21.15.123:0/1425795536 reap_dead start 2024-04-24T05:46:24.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.991+0000 7fd0c7400640 1 -- 172.21.15.123:0/1425795536 shutdown_connections 2024-04-24T05:46:24.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.991+0000 7fd0c7400640 1 -- 172.21.15.123:0/1425795536 >> 172.21.15.123:0/1425795536 conn(0x7fd0dc0b5f60 msgr2=0x7fd0dc0b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.992+0000 7fd0c7400640 1 -- 172.21.15.123:0/1425795536 shutdown_connections 2024-04-24T05:46:24.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.992+0000 7fd0c7400640 1 -- 172.21.15.123:0/1425795536 wait complete. 2024-04-24T05:46:24.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.992+0000 7f0453600640 1 -- 172.21.15.123:0/411285902 shutdown_connections 2024-04-24T05:46:24.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.992+0000 7f0453600640 1 -- 172.21.15.123:0/411285902 >> 172.21.15.123:0/411285902 conn(0x7f044c0b4fd0 msgr2=0x7f044c16cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.992+0000 7f0453600640 1 -- 172.21.15.123:0/411285902 shutdown_connections 2024-04-24T05:46:24.022 INFO:teuthology.orchestra.run.smithi123.stdout:55834574858 2024-04-24T05:46:24.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.996+0000 7f0453600640 1 -- 172.21.15.123:0/411285902 wait complete. 2024-04-24T05:46:24.023 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T05:46:24.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.996+0000 7fbc2ee00640 1 -- 172.21.15.123:0/197058486 --> [v2:172.21.15.155:6816/1974539526,v1:172.21.15.155:6817/1974539526] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbc28075b70 con 0x7fbc080760b0 2024-04-24T05:46:24.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.997+0000 7fbc25600640 1 -- 172.21.15.123:0/197058486 <== osd.7 v2:172.21.15.155:6816/1974539526 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbc28075b70 con 0x7fbc080760b0 2024-04-24T05:46:24.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.998+0000 7fbc2ee00640 1 -- 172.21.15.123:0/197058486 >> [v2:172.21.15.155:6816/1974539526,v1:172.21.15.155:6817/1974539526] conn(0x7fbc080760b0 msgr2=0x7fbc080784f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.998+0000 7fbc2ee00640 1 --2- 172.21.15.123:0/197058486 >> [v2:172.21.15.155:6816/1974539526,v1:172.21.15.155:6817/1974539526] conn(0x7fbc080760b0 0x7fbc080784f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.998+0000 7fbc2ee00640 1 -- 172.21.15.123:0/197058486 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbc0806c9c0 msgr2=0x7fbc0806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.998+0000 7fbc2ee00640 1 --2- 172.21.15.123:0/197058486 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbc0806c9c0 0x7fbc0806ee80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c001560 tx=0x7fbc1c0019f0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.998+0000 7fbc2ee00640 1 -- 172.21.15.123:0/197058486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc280cbfa0 msgr2=0x7fbc280aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.998+0000 7fbc2ee00640 1 --2- 172.21.15.123:0/197058486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc280cbfa0 0x7fbc280aee80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc1402a700 tx=0x7fbc140046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.998+0000 7fbc2e400640 1 -- 172.21.15.123:0/197058486 reap_dead start 2024-04-24T05:46:24.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.999+0000 7fbc2ee00640 1 -- 172.21.15.123:0/197058486 shutdown_connections 2024-04-24T05:46:24.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:23.999+0000 7fbc2ee00640 1 -- 172.21.15.123:0/197058486 >> 172.21.15.123:0/197058486 conn(0x7fbc280b1b10 msgr2=0x7fbc2815c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.000+0000 7fbc2ee00640 1 -- 172.21.15.123:0/197058486 shutdown_connections 2024-04-24T05:46:24.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.000+0000 7fbc2ee00640 1 -- 172.21.15.123:0/197058486 wait complete. 2024-04-24T05:46:24.026 INFO:teuthology.orchestra.run.smithi123.stdout:55834574857 2024-04-24T05:46:24.027 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T05:46:24.027 INFO:teuthology.orchestra.run.smithi123.stdout:55834574859 2024-04-24T05:46:24.028 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T05:46:24.028 INFO:teuthology.orchestra.run.smithi123.stdout:55834574857 2024-04-24T05:46:24.029 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T05:46:24.029 INFO:teuthology.orchestra.run.smithi123.stdout:55834574859 2024-04-24T05:46:24.030 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T05:46:24.030 INFO:teuthology.orchestra.run.smithi123.stdout:55834574859 2024-04-24T05:46:24.031 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T05:46:24.031 INFO:teuthology.orchestra.run.smithi123.stdout:55834574858 2024-04-24T05:46:24.032 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T05:46:24.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.341+0000 7fbc15000640 1 Processor -- start 2024-04-24T05:46:24.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.342+0000 7fbc15000640 1 -- start start 2024-04-24T05:46:24.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.344+0000 7fbc15000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbc1015e610 0x7fbc1006cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.344+0000 7fbc15000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbc1015e9f0 0x7fbc100d3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.345+0000 7fbc15000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc100b3340 0x7fbc100d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.345+0000 7fbc15000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbc1006d5a0 con 0x7fbc1015e610 2024-04-24T05:46:24.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.345+0000 7fbc15000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbc100ceae0 con 0x7fbc1015e9f0 2024-04-24T05:46:24.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.345+0000 7fbc15000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1006d020 con 0x7fbc100b3340 2024-04-24T05:46:24.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.345+0000 7fbc0d400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbc1015e9f0 0x7fbc100d3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.345+0000 7fbc0d400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbc1015e9f0 0x7fbc100d3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:50010/0 (socket says 172.21.15.123:50010) 2024-04-24T05:46:24.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.345+0000 7fbc0d400640 1 -- 172.21.15.123:0/2917804901 learned_addr learned my addr 172.21.15.123:0/2917804901 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.345+0000 7fbc0e800640 1 --2- 172.21.15.123:0/2917804901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc100b3340 0x7fbc100d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.346+0000 7fbc0e800640 1 -- 172.21.15.123:0/2917804901 >> v2:172.21.15.155:3300/0 conn(0x7fbc1015e9f0 msgr2=0x7fbc100d3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:24.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.346+0000 7fbc0e800640 1 --2- 172.21.15.123:0/2917804901 >> v2:172.21.15.155:3300/0 conn(0x7fbc1015e9f0 0x7fbc100d3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.346+0000 7fbc0e800640 1 -- 172.21.15.123:0/2917804901 >> v2:172.21.15.123:3300/0 conn(0x7fbc1015e610 msgr2=0x7fbc1006cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:24.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.346+0000 7fbc0e800640 1 --2- 172.21.15.123:0/2917804901 >> v2:172.21.15.123:3300/0 conn(0x7fbc1015e610 0x7fbc1006cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.346+0000 7fbc0e800640 1 -- 172.21.15.123:0/2917804901 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1006d2c0 con 0x7fbc100b3340 2024-04-24T05:46:24.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.348+0000 7fbc0e800640 1 --2- 172.21.15.123:0/2917804901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc100b3340 0x7fbc100d35b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbc100b32d0 tx=0x7fbbfc02a120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cc66592b755afca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.349+0000 7fbc0ca00640 1 -- 172.21.15.123:0/2917804901 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc037070 con 0x7fbc100b3340 2024-04-24T05:46:24.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.349+0000 7f1294a00640 1 Processor -- start 2024-04-24T05:46:24.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.349+0000 7f1294a00640 1 -- start start 2024-04-24T05:46:24.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.350+0000 7fbc0ca00640 1 -- 172.21.15.123:0/2917804901 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbfc032440 con 0x7fbc100b3340 2024-04-24T05:46:24.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.350+0000 7fbc0ca00640 1 -- 172.21.15.123:0/2917804901 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc032850 con 0x7fbc100b3340 2024-04-24T05:46:24.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.351+0000 7fbc15000640 1 -- 172.21.15.123:0/2917804901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc100b3340 msgr2=0x7fbc100d35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.351+0000 7fbc15000640 1 --2- 172.21.15.123:0/2917804901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc100b3340 0x7fbc100d35b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbc100b32d0 tx=0x7fbbfc02a120 comp rx=0 tx=0).stop 2024-04-24T05:46:24.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.352+0000 7fbc15000640 1 -- 172.21.15.123:0/2917804901 shutdown_connections 2024-04-24T05:46:24.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.352+0000 7fbc15000640 1 --2- 172.21.15.123:0/2917804901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc100b3340 0x7fbc100d35b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.352+0000 7fbc15000640 1 --2- 172.21.15.123:0/2917804901 >> v2:172.21.15.155:3300/0 conn(0x7fbc1015e9f0 0x7fbc100d3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.352+0000 7fbc15000640 1 --2- 172.21.15.123:0/2917804901 >> v2:172.21.15.123:3300/0 conn(0x7fbc1015e610 0x7fbc1006cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.352+0000 7fbc15000640 1 -- 172.21.15.123:0/2917804901 >> 172.21.15.123:0/2917804901 conn(0x7fbc100d0f10 msgr2=0x7fbc100d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.352+0000 7f1294a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f128803e840 0x7f1288053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.352+0000 7f1294a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1288054130 0x7f12880068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.352+0000 7f1294a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1288058570 con 0x7f12880e30c0 2024-04-24T05:46:24.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.352+0000 7f1294a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1288058850 con 0x7f1288054130 2024-04-24T05:46:24.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.352+0000 7f1294a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1288058b30 con 0x7f128803e840 2024-04-24T05:46:24.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.353+0000 7fc0a9000640 1 Processor -- start 2024-04-24T05:46:24.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.353+0000 7fc0a9000640 1 -- start start 2024-04-24T05:46:24.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.352+0000 7fbc15000640 1 -- 172.21.15.123:0/2917804901 shutdown_connections 2024-04-24T05:46:24.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.352+0000 7fbc15000640 1 -- 172.21.15.123:0/2917804901 wait complete. 2024-04-24T05:46:24.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.354+0000 7f128de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f128803e840 0x7f1288053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.354+0000 7f128de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f128803e840 0x7f1288053bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38602/0 (socket says 172.21.15.123:38602) 2024-04-24T05:46:24.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.354+0000 7f128de00640 1 -- 172.21.15.123:0/2850004618 learned_addr learned my addr 172.21.15.123:0/2850004618 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.354+0000 7f128f200640 1 --2- 172.21.15.123:0/2850004618 >> v2:172.21.15.123:3300/0 conn(0x7f1288054130 0x7f12880068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.354+0000 7f128f200640 1 -- 172.21.15.123:0/2850004618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f128803e840 msgr2=0x7f1288053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:24.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.355+0000 7fbc15000640 1 Processor -- start 2024-04-24T05:46:24.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.354+0000 7f128f200640 1 --2- 172.21.15.123:0/2850004618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f128803e840 0x7f1288053bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.354+0000 7f128f200640 1 -- 172.21.15.123:0/2850004618 >> v1:172.21.15.123:6789/0 conn(0x7f12880e30c0 legacy=0x7f1288057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:24.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.355+0000 7fbc15000640 1 -- start start 2024-04-24T05:46:24.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.355+0000 7fc0a9000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc09c0e32d0 0x7fc09c052c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.355+0000 7fc0a9000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc09c04ff80 0x7fc09c037bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.355+0000 7fc0a9000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc09c0536a0 con 0x7fc09c053190 2024-04-24T05:46:24.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.355+0000 7fc0a9000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc09c0e36b0 con 0x7fc09c0e32d0 2024-04-24T05:46:24.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.355+0000 7fc0a9000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc09c053980 con 0x7fc09c04ff80 2024-04-24T05:46:24.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.356+0000 7fc0a3400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc09c0e32d0 0x7fc09c052c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.356+0000 7fc0a3400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc09c0e32d0 0x7fc09c052c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56562/0 (socket says 172.21.15.123:56562) 2024-04-24T05:46:24.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.356+0000 7fc0a3400640 1 -- 172.21.15.123:0/3547330399 learned_addr learned my addr 172.21.15.123:0/3547330399 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.356+0000 7fc0a3e00640 1 --2- 172.21.15.123:0/3547330399 >> v2:172.21.15.155:3300/0 conn(0x7fc09c04ff80 0x7fc09c037bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.356+0000 7fbc15000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc100b3340 0x7fbc100b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.356+0000 7fbc15000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc1015e610 0x7fbc1015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.357+0000 7fbc15000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc1015e9f0 0x7fbc1015b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.357+0000 7fbc15000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc100cf2f0 con 0x7fbc100b3340 2024-04-24T05:46:24.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.357+0000 7fbc15000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbc100cf010 con 0x7fbc1015e610 2024-04-24T05:46:24.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.357+0000 7fbc15000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbc10162390 con 0x7fbc1015e9f0 2024-04-24T05:46:24.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.356+0000 7f128f200640 1 -- 172.21.15.123:0/2850004618 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12880063b0 con 0x7f1288054130 2024-04-24T05:46:24.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.357+0000 7fc0a3400640 1 -- 172.21.15.123:0/3547330399 >> v2:172.21.15.155:3300/0 conn(0x7fc09c04ff80 msgr2=0x7fc09c037bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:24.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.357+0000 7fc0a3400640 1 --2- 172.21.15.123:0/3547330399 >> v2:172.21.15.155:3300/0 conn(0x7fc09c04ff80 0x7fc09c037bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.357+0000 7fc0a3400640 1 -- 172.21.15.123:0/3547330399 >> v1:172.21.15.155:6789/0 conn(0x7fc09c053190 legacy=0x7fc09c04f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:24.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.357+0000 7fc0a3400640 1 -- 172.21.15.123:0/3547330399 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09c053c60 con 0x7fc09c0e32d0 2024-04-24T05:46:24.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.357+0000 7fbc0e800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc1015e9f0 0x7fbc1015b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.357+0000 7fbc0d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc1015e610 0x7fbc1015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.357+0000 7fbc0d400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc1015e610 0x7fbc1015adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:50032/0 (socket says 172.21.15.123:50032) 2024-04-24T05:46:24.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.357+0000 7fbc0d400640 1 -- 172.21.15.123:0/1750534612 learned_addr learned my addr 172.21.15.123:0/1750534612 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.357+0000 7f128f200640 1 --2- 172.21.15.123:0/2850004618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1288054130 0x7f12880068d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1290065520 tx=0x7f12900a5b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd5f3eebbac40c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.358+0000 7f128d400640 1 -- 172.21.15.123:0/2850004618 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12900a7930 con 0x7f1288054130 2024-04-24T05:46:24.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.358+0000 7f128d400640 1 -- 172.21.15.123:0/2850004618 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12900a7ad0 con 0x7f1288054130 2024-04-24T05:46:24.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.358+0000 7f128d400640 1 -- 172.21.15.123:0/2850004618 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12900a8440 con 0x7f1288054130 2024-04-24T05:46:24.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.358+0000 7fc0a3400640 1 --2- 172.21.15.123:0/3547330399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c0e32d0 0x7fc09c052c50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc0a406ac00 tx=0x7fc0a40a5b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e3323f2a656c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.359+0000 7fc0a2000640 1 -- 172.21.15.123:0/3547330399 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a40a7980 con 0x7fc09c0e32d0 2024-04-24T05:46:24.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.359+0000 7f1294a00640 1 -- 172.21.15.123:0/2850004618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1288054130 msgr2=0x7f12880068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.359+0000 7f1294a00640 1 --2- 172.21.15.123:0/2850004618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1288054130 0x7f12880068d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1290065520 tx=0x7f12900a5b10 comp rx=0 tx=0).stop 2024-04-24T05:46:24.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.359+0000 7fc0a2000640 1 -- 172.21.15.123:0/3547330399 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a40a7b20 con 0x7fc09c0e32d0 2024-04-24T05:46:24.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.359+0000 7fc0a2000640 1 -- 172.21.15.123:0/3547330399 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a40a7980 con 0x7fc09c0e32d0 2024-04-24T05:46:24.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.359+0000 7f1294a00640 1 -- 172.21.15.123:0/2850004618 shutdown_connections 2024-04-24T05:46:24.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.359+0000 7f1294a00640 1 --2- 172.21.15.123:0/2850004618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1288054130 0x7f12880068d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.359+0000 7f1294a00640 1 --2- 172.21.15.123:0/2850004618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f128803e840 0x7f1288053bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.359+0000 7f1294a00640 1 -- 172.21.15.123:0/2850004618 >> 172.21.15.123:0/2850004618 conn(0x7f12880559e0 msgr2=0x7f1288055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.358+0000 7fbc0d400640 1 -- 172.21.15.123:0/1750534612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc1015e9f0 msgr2=0x7fbc1015b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.358+0000 7fbc0d400640 1 --2- 172.21.15.123:0/1750534612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc1015e9f0 0x7fbc1015b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.358+0000 7fbc0d400640 1 -- 172.21.15.123:0/1750534612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc100b3340 msgr2=0x7fbc100b48f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T05:46:24.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.360+0000 7fbc0de00640 1 --2- 172.21.15.123:0/1750534612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc100b3340 0x7fbc100b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.360+0000 7fbc0d400640 1 --2- 172.21.15.123:0/1750534612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc100b3340 0x7fbc100b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.360+0000 7fbc0d400640 1 -- 172.21.15.123:0/1750534612 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1015b8e0 con 0x7fbc1015e610 2024-04-24T05:46:24.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.360+0000 7fbc0de00640 1 --2- 172.21.15.123:0/1750534612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc100b3340 0x7fbc100b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:24.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.360+0000 7fc0a9000640 1 -- 172.21.15.123:0/3547330399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c0e32d0 msgr2=0x7fc09c052c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.360+0000 7fc0a9000640 1 --2- 172.21.15.123:0/3547330399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c0e32d0 0x7fc09c052c50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc0a406ac00 tx=0x7fc0a40a5b60 comp rx=0 tx=0).stop 2024-04-24T05:46:24.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.360+0000 7fbc0d400640 1 --2- 172.21.15.123:0/1750534612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc1015e610 0x7fbc1015adb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800efc0 tx=0x7fbc0800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.360+0000 7f1294a00640 1 -- 172.21.15.123:0/2850004618 shutdown_connections 2024-04-24T05:46:24.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.361+0000 7fc0a9000640 1 -- 172.21.15.123:0/3547330399 shutdown_connections 2024-04-24T05:46:24.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.361+0000 7fc0a9000640 1 --2- 172.21.15.123:0/3547330399 >> v2:172.21.15.155:3300/0 conn(0x7fc09c04ff80 0x7fc09c037bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.361+0000 7fc0a9000640 1 --2- 172.21.15.123:0/3547330399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c0e32d0 0x7fc09c052c50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.361+0000 7fc0a9000640 1 -- 172.21.15.123:0/3547330399 >> 172.21.15.123:0/3547330399 conn(0x7fc09c009fb0 msgr2=0x7fc09c00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.361+0000 7fc0a9000640 1 -- 172.21.15.123:0/3547330399 shutdown_connections 2024-04-24T05:46:24.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.361+0000 7fc0a9000640 1 -- 172.21.15.123:0/3547330399 wait complete. 2024-04-24T05:46:24.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.360+0000 7f1294a00640 1 -- 172.21.15.123:0/2850004618 wait complete. 2024-04-24T05:46:24.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.362+0000 7fbc06000640 1 -- 172.21.15.123:0/1750534612 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0800cda0 con 0x7fbc1015e610 2024-04-24T05:46:24.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.362+0000 7fbc06000640 1 -- 172.21.15.123:0/1750534612 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc08019070 con 0x7fbc1015e610 2024-04-24T05:46:24.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.362+0000 7fbc06000640 1 -- 172.21.15.123:0/1750534612 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc080146c0 con 0x7fbc1015e610 2024-04-24T05:46:24.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.362+0000 7f3350a00640 1 Processor -- start 2024-04-24T05:46:24.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.362+0000 7fbc15000640 1 -- 172.21.15.123:0/1750534612 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1015bbc0 con 0x7fbc1015e610 2024-04-24T05:46:24.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.362+0000 7fbc15000640 1 -- 172.21.15.123:0/1750534612 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1015c180 con 0x7fbc1015e610 2024-04-24T05:46:24.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.362+0000 7fee0d000640 1 Processor -- start 2024-04-24T05:46:24.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.363+0000 7fee0d000640 1 -- start start 2024-04-24T05:46:24.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.363+0000 7f1294a00640 1 Processor -- start 2024-04-24T05:46:24.369 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.363+0000 7fc0a9000640 1 Processor -- start 2024-04-24T05:46:24.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.363+0000 7f1294a00640 1 -- start start 2024-04-24T05:46:24.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.364+0000 7fbc15000640 1 -- 172.21.15.123:0/1750534612 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc10075b70 con 0x7fbc1015e610 2024-04-24T05:46:24.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.364+0000 7fc0a9000640 1 -- start start 2024-04-24T05:46:24.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.364+0000 7f3da7e00640 1 Processor -- start 2024-04-24T05:46:24.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.363+0000 7f3350a00640 1 -- start start 2024-04-24T05:46:24.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.364+0000 7f3350a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f334c071e40 0x7f334c072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.364+0000 7f3350a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f334c14c1d0 0x7f334c06efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.371 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.364+0000 7f3350a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f334c072760 con 0x7f334c162e90 2024-04-24T05:46:24.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.364+0000 7f3350a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f334c06eb40 con 0x7f334c071e40 2024-04-24T05:46:24.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.364+0000 7f3350a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f334c06df30 con 0x7f334c14c1d0 2024-04-24T05:46:24.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.364+0000 7fee0d000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fee0815ebf0 0x7fee0806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.365+0000 7fee0d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee080d35f0 0x7fee080b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.365+0000 7fee0d000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee080d3af0 con 0x7fee080d1500 2024-04-24T05:46:24.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.365+0000 7fee0d000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fee080b6030 con 0x7fee0815ebf0 2024-04-24T05:46:24.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.365+0000 7fee0d000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee080b6310 con 0x7fee080d35f0 2024-04-24T05:46:24.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.365+0000 7fee05e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fee0815ebf0 0x7fee0806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.365+0000 7fee05400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee080d35f0 0x7fee080b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.365+0000 7fee05400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee080d35f0 0x7fee080b67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38620/0 (socket says 172.21.15.123:38620) 2024-04-24T05:46:24.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.365+0000 7fee05400640 1 -- 172.21.15.123:0/878097232 learned_addr learned my addr 172.21.15.123:0/878097232 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.365+0000 7f1294a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f128803e840 0x7f12880d6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.365+0000 7f1294a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1288054130 0x7f1288051430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7f334a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f334c14c1d0 0x7f334c06efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.365+0000 7f1294a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12880e30c0 0x7f12880e1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.365+0000 7f1294a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1288006e10 con 0x7f128803e840 2024-04-24T05:46:24.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7fc0a9000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc09c04ff80 0x7fc09c0e4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7fc0a9000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc09c053190 0x7fc09c0e6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.365+0000 7f1294a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f12880e34a0 con 0x7f1288054130 2024-04-24T05:46:24.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.365+0000 7f1294a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1288006f90 con 0x7f12880e30c0 2024-04-24T05:46:24.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7f128f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12880e30c0 0x7f12880e1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7f128f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12880e30c0 0x7f12880e1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38634/0 (socket says 172.21.15.123:38634) 2024-04-24T05:46:24.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7f128f200640 1 -- 172.21.15.123:0/2951515772 learned_addr learned my addr 172.21.15.123:0/2951515772 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7f334a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f334c14c1d0 0x7f334c06efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38632/0 (socket says 172.21.15.123:38632) 2024-04-24T05:46:24.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7f3349e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f334c071e40 0x7f334c072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7f3349e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f334c071e40 0x7f334c072220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:50036/0 (socket says 172.21.15.123:50036) 2024-04-24T05:46:24.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7f3349e00640 1 -- 172.21.15.123:0/3961744483 learned_addr learned my addr 172.21.15.123:0/3961744483 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7f3349400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f334c162e90 0x7f334c0ad150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35570/0 (socket says 172.21.15.123:35570) 2024-04-24T05:46:24.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7fee05e00640 1 -- 172.21.15.123:0/878097232 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee080d35f0 msgr2=0x7fee080b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:24.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7fee05e00640 1 --2- 172.21.15.123:0/878097232 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee080d35f0 0x7fee080b67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.364+0000 7f3da7e00640 1 -- start start 2024-04-24T05:46:24.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7f3da7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3da00e30c0 0x7f3da0057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7f3da7e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3da003e840 0x7f3da0053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7f3da7e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3da00582d0 con 0x7f3da0054130 2024-04-24T05:46:24.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.367+0000 7fee05e00640 1 -- 172.21.15.123:0/878097232 >> v1:172.21.15.155:6789/0 conn(0x7fee080d1500 legacy=0x7fee081625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.367+0000 7f128de00640 1 --2- 172.21.15.123:0/2951515772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1288054130 0x7f1288051430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7f3da7e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3da0058570 con 0x7f3da00e30c0 2024-04-24T05:46:24.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.366+0000 7f3da7e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3da0058850 con 0x7f3da003e840 2024-04-24T05:46:24.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.367+0000 7fee05e00640 1 -- 172.21.15.123:0/878097232 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee080d3dd0 con 0x7fee0815ebf0 2024-04-24T05:46:24.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.367+0000 7f128e800640 1 --2- 172.21.15.123:0/2951515772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f128803e840 0x7f12880d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.367+0000 7f3da6000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3da003e840 0x7f3da0053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.367+0000 7f3da6000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3da003e840 0x7f3da0053bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38636/0 (socket says 172.21.15.123:38636) 2024-04-24T05:46:24.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.367+0000 7f3da6000640 1 -- 172.21.15.123:0/46717419 learned_addr learned my addr 172.21.15.123:0/46717419 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.367+0000 7f3da6a00640 1 --2- 172.21.15.123:0/46717419 >> v2:172.21.15.123:3300/0 conn(0x7f3da00e30c0 0x7f3da0057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7f3348a00640 1 -- 172.21.15.123:0/3961744483 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2636118679 0 0) 0x7f334c072760 con 0x7f334c162e90 2024-04-24T05:46:24.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7f128de00640 1 -- 172.21.15.123:0/2951515772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12880e30c0 msgr2=0x7f12880e1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7f128de00640 1 --2- 172.21.15.123:0/2951515772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12880e30c0 0x7f12880e1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7f128de00640 1 -- 172.21.15.123:0/2951515772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f128803e840 msgr2=0x7f12880d6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.367+0000 7fbc06000640 1 -- 172.21.15.123:0/1750534612 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc08005d10 con 0x7fbc1015e610 2024-04-24T05:46:24.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7f3349e00640 1 -- 172.21.15.123:0/3961744483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f334c14c1d0 msgr2=0x7f334c06efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:24.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7f3349e00640 1 --2- 172.21.15.123:0/3961744483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f334c14c1d0 0x7f334c06efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7f3349e00640 1 -- 172.21.15.123:0/3961744483 >> v1:172.21.15.123:6789/0 conn(0x7f334c162e90 legacy=0x7f334c0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7f3349e00640 1 -- 172.21.15.123:0/3961744483 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c06e210 con 0x7f334c071e40 2024-04-24T05:46:24.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7fc0a9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c0e32d0 0x7fc09c04bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7fc0a9000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c054740 con 0x7fc09c0e32d0 2024-04-24T05:46:24.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7fc0a9000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c054220 con 0x7fc09c053190 2024-04-24T05:46:24.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7fc0a9000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc09c03a900 con 0x7fc09c04ff80 2024-04-24T05:46:24.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7f3da6000640 1 -- 172.21.15.123:0/46717419 >> v2:172.21.15.123:3300/0 conn(0x7f3da00e30c0 msgr2=0x7f3da0057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:24.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7f3da6000640 1 --2- 172.21.15.123:0/46717419 >> v2:172.21.15.123:3300/0 conn(0x7f3da00e30c0 0x7f3da0057d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7f3da6000640 1 -- 172.21.15.123:0/46717419 >> v1:172.21.15.155:6789/0 conn(0x7f3da0054130 legacy=0x7f3da00e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:24.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7fee05e00640 1 --2- 172.21.15.123:0/878097232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee0815ebf0 0x7fee0806ed20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fedf80097a0 tx=0x7fedf802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac80aaa934888dfc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.369+0000 7fbc06000640 1 --2- 172.21.15.123:0/1750534612 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbbf006c940 0x7fbbf006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.369+0000 7f3da6000640 1 -- 172.21.15.123:0/46717419 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da0058b30 con 0x7f3da003e840 2024-04-24T05:46:24.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.369+0000 7fbc06000640 1 -- 172.21.15.123:0/1750534612 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbc0808cdd0 con 0x7fbc1015e610 2024-04-24T05:46:24.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.369+0000 7fc0a3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c0e32d0 0x7fc09c04bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.369+0000 7fc0a3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c0e32d0 0x7fc09c04bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56608/0 (socket says 172.21.15.123:56608) 2024-04-24T05:46:24.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.369+0000 7fc0a3e00640 1 -- 172.21.15.123:0/4249753357 learned_addr learned my addr 172.21.15.123:0/4249753357 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.369+0000 7f6222800640 1 Processor -- start 2024-04-24T05:46:24.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.369+0000 7f6222800640 1 -- start start 2024-04-24T05:46:24.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.370+0000 7fbc0de00640 1 --2- 172.21.15.123:0/1750534612 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbbf006c940 0x7fbbf006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.369+0000 7fc0a3400640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc09c04ff80 0x7fc09c0e4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.370+0000 7fc0a2a00640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc09c053190 0x7fc09c0e6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7f128de00640 1 --2- 172.21.15.123:0/2951515772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f128803e840 0x7f12880d6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.368+0000 7f128de00640 1 -- 172.21.15.123:0/2951515772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128817d920 con 0x7f1288054130 2024-04-24T05:46:24.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.370+0000 7f128de00640 1 --2- 172.21.15.123:0/2951515772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1288054130 0x7f1288051430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f127c00e850 tx=0x7f127c00ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.370+0000 7fee04a00640 1 -- 172.21.15.123:0/878097232 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8037070 con 0x7fee0815ebf0 2024-04-24T05:46:24.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.370+0000 7fee04a00640 1 -- 172.21.15.123:0/878097232 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf8032440 con 0x7fee0815ebf0 2024-04-24T05:46:24.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.370+0000 7fee04a00640 1 -- 172.21.15.123:0/878097232 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8032810 con 0x7fee0815ebf0 2024-04-24T05:46:24.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.371+0000 7fc0a3e00640 1 -- 172.21.15.123:0/4249753357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc09c04ff80 msgr2=0x7fc09c0e4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.371+0000 7fc0a3e00640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc09c04ff80 0x7fc09c0e4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.371+0000 7fc0a3e00640 1 -- 172.21.15.123:0/4249753357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc09c053190 msgr2=0x7fc09c0e6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.371+0000 7fc0a3e00640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc09c053190 0x7fc09c0e6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.371+0000 7fc0a3e00640 1 -- 172.21.15.123:0/4249753357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09c0d95c0 con 0x7fc09c0e32d0 2024-04-24T05:46:24.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.371+0000 7fc0a2a00640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc09c053190 0x7fc09c0e6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:46:24.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.371+0000 7f3349e00640 1 --2- 172.21.15.123:0/3961744483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f334c071e40 0x7f334c072220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f333c009870 tx=0x7f333c02a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c03159d2cf6439a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.371+0000 7fee0d000640 1 -- 172.21.15.123:0/878097232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee0815ebf0 msgr2=0x7fee0806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.371+0000 7fee0d000640 1 --2- 172.21.15.123:0/878097232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee0815ebf0 0x7fee0806ed20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fedf80097a0 tx=0x7fedf802a180 comp rx=0 tx=0).stop 2024-04-24T05:46:24.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.371+0000 7fc0a3e00640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c0e32d0 0x7fc09c04bb90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc09800d490 tx=0x7fc09800d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7fbc0de00640 1 --2- 172.21.15.123:0/1750534612 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbbf006c940 0x7fbbf006ee00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc00006fd0 tx=0x7fbc00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7f68ee200640 1 Processor -- start 2024-04-24T05:46:24.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7f68ee200640 1 -- start start 2024-04-24T05:46:24.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.371+0000 7f6222800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f621c073bc0 0x7f621c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.371+0000 7f6222800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f621c0ad150 0x7f621c0ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7f6222800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f621c1641f0 con 0x7f621c15d360 2024-04-24T05:46:24.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7f6222800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f621c0adac0 con 0x7f621c073bc0 2024-04-24T05:46:24.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7f6222800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f621c163cc0 con 0x7f621c0ad150 2024-04-24T05:46:24.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7f3da6000640 1 --2- 172.21.15.123:0/46717419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3da003e840 0x7f3da0053bf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009870 tx=0x7f3d9402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c6edde9939b52df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7f621aa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f621c073bc0 0x7f621c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7fc097400640 1 -- 172.21.15.123:0/4249753357 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0980125f0 con 0x7fc09c0e32d0 2024-04-24T05:46:24.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7fc097400640 1 -- 172.21.15.123:0/4249753357 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc098018070 con 0x7fc09c0e32d0 2024-04-24T05:46:24.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7fc097400640 1 -- 172.21.15.123:0/4249753357 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0980129f0 con 0x7fc09c0e32d0 2024-04-24T05:46:24.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7fee0d000640 1 -- 172.21.15.123:0/878097232 shutdown_connections 2024-04-24T05:46:24.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7fee0d000640 1 --2- 172.21.15.123:0/878097232 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee080d35f0 0x7fee080b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7fee0d000640 1 --2- 172.21.15.123:0/878097232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee0815ebf0 0x7fee0806ed20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7fee0d000640 1 -- 172.21.15.123:0/878097232 >> 172.21.15.123:0/878097232 conn(0x7fee080b9570 msgr2=0x7fee080b9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7fc0a9000640 1 -- 172.21.15.123:0/4249753357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09c0d92e0 con 0x7fc09c0e32d0 2024-04-24T05:46:24.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7f621aa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f621c073bc0 0x7f621c0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56618/0 (socket says 172.21.15.123:56618) 2024-04-24T05:46:24.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7f621aa00640 1 -- 172.21.15.123:0/505929840 learned_addr learned my addr 172.21.15.123:0/505929840 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7f621be00640 1 --2- 172.21.15.123:0/505929840 >> v2:172.21.15.155:3300/0 conn(0x7f621c0ad150 0x7f621c0ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7f3da5600640 1 -- 172.21.15.123:0/46717419 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94037070 con 0x7f3da003e840 2024-04-24T05:46:24.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7f3da5600640 1 -- 172.21.15.123:0/46717419 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d9402ae50 con 0x7f3da003e840 2024-04-24T05:46:24.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7f3da5600640 1 -- 172.21.15.123:0/46717419 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94032860 con 0x7f3da003e840 2024-04-24T05:46:24.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.372+0000 7fee0d000640 1 -- 172.21.15.123:0/878097232 shutdown_connections 2024-04-24T05:46:24.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.373+0000 7fc0a9000640 1 -- 172.21.15.123:0/4249753357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc09c0e45d0 con 0x7fc09c0e32d0 2024-04-24T05:46:24.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.373+0000 7fee0d000640 1 -- 172.21.15.123:0/878097232 wait complete. 2024-04-24T05:46:24.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.373+0000 7f621aa00640 1 -- 172.21.15.123:0/505929840 >> v2:172.21.15.155:3300/0 conn(0x7f621c0ad150 msgr2=0x7f621c0ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:24.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.373+0000 7f621aa00640 1 --2- 172.21.15.123:0/505929840 >> v2:172.21.15.155:3300/0 conn(0x7f621c0ad150 0x7f621c0ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.373+0000 7f621aa00640 1 -- 172.21.15.123:0/505929840 >> v1:172.21.15.123:6789/0 conn(0x7f621c15d360 legacy=0x7f621c06c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.373+0000 7f621aa00640 1 -- 172.21.15.123:0/505929840 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621c06bb30 con 0x7f621c073bc0 2024-04-24T05:46:24.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.374+0000 7f68ee200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68e80b0f10 0x7f68e80af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.374+0000 7f68ee200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f68e80ce940 0x7f68e80afb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.374+0000 7f68ee200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68e80b0040 con 0x7f68e80ce560 2024-04-24T05:46:24.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.374+0000 7f68ee200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f68e80ae520 con 0x7f68e80ce940 2024-04-24T05:46:24.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.374+0000 7f68ee200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f68e80ae800 con 0x7f68e80b0f10 2024-04-24T05:46:24.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.374+0000 7f3da7e00640 1 -- 172.21.15.123:0/46717419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3da003e840 msgr2=0x7f3da0053bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.374+0000 7f3da7e00640 1 --2- 172.21.15.123:0/46717419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3da003e840 0x7f3da0053bf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009870 tx=0x7f3d9402a300 comp rx=0 tx=0).stop 2024-04-24T05:46:24.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.374+0000 7f3da7e00640 1 -- 172.21.15.123:0/46717419 shutdown_connections 2024-04-24T05:46:24.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.374+0000 7f3da7e00640 1 --2- 172.21.15.123:0/46717419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3da003e840 0x7f3da0053bf0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.374+0000 7f3da7e00640 1 --2- 172.21.15.123:0/46717419 >> v2:172.21.15.123:3300/0 conn(0x7f3da00e30c0 0x7f3da0057d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.374+0000 7f3da7e00640 1 -- 172.21.15.123:0/46717419 >> 172.21.15.123:0/46717419 conn(0x7f3da00559e0 msgr2=0x7f3da0055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.374+0000 7f68dea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68e80b0f10 0x7f68e80af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.375+0000 7f68dea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68e80b0f10 0x7f68e80af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38662/0 (socket says 172.21.15.123:38662) 2024-04-24T05:46:24.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.375+0000 7f68dea00640 1 -- 172.21.15.123:0/481280615 learned_addr learned my addr 172.21.15.123:0/481280615 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.375+0000 7f68e7e00640 1 --2- 172.21.15.123:0/481280615 >> v2:172.21.15.123:3300/0 conn(0x7f68e80ce940 0x7f68e80afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.375+0000 7f621aa00640 1 --2- 172.21.15.123:0/505929840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621c073bc0 0x7f621c0b5da0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f621c072cd0 tx=0x7f621402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2eacae3ddb4169f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.375+0000 7f3da7e00640 1 -- 172.21.15.123:0/46717419 shutdown_connections 2024-04-24T05:46:24.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.375+0000 7f68dea00640 1 -- 172.21.15.123:0/481280615 >> v2:172.21.15.123:3300/0 conn(0x7f68e80ce940 msgr2=0x7f68e80afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:24.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.375+0000 7f68dea00640 1 --2- 172.21.15.123:0/481280615 >> v2:172.21.15.123:3300/0 conn(0x7f68e80ce940 0x7f68e80afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.375+0000 7f68dea00640 1 -- 172.21.15.123:0/481280615 >> v1:172.21.15.123:6789/0 conn(0x7f68e80ce560 legacy=0x7f68e80cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.375+0000 7f68dea00640 1 -- 172.21.15.123:0/481280615 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e80aeae0 con 0x7f68e80b0f10 2024-04-24T05:46:24.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.375+0000 7f68e6a00640 1 -- 172.21.15.123:0/481280615 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3074599295 0 0) 0x7f68e80b0040 con 0x7f68e80ce560 2024-04-24T05:46:24.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.374+0000 7fc0a9000640 1 -- 172.21.15.123:0/4249753357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc09c0089d0 con 0x7fc09c0e32d0 2024-04-24T05:46:24.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.376+0000 7fee0d000640 1 Processor -- start 2024-04-24T05:46:24.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.376+0000 7f621a000640 1 -- 172.21.15.123:0/505929840 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214037070 con 0x7f621c073bc0 2024-04-24T05:46:24.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.375+0000 7f3da7e00640 1 -- 172.21.15.123:0/46717419 wait complete. 2024-04-24T05:46:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.376+0000 7f621a000640 1 -- 172.21.15.123:0/505929840 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6214032440 con 0x7f621c073bc0 2024-04-24T05:46:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.376+0000 7f621a000640 1 -- 172.21.15.123:0/505929840 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214032850 con 0x7f621c073bc0 2024-04-24T05:46:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.376+0000 7fee0d000640 1 -- start start 2024-04-24T05:46:24.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.377+0000 7fc097400640 1 -- 172.21.15.123:0/4249753357 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc098012b90 con 0x7fc09c0e32d0 2024-04-24T05:46:24.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.377+0000 7f68dea00640 1 --2- 172.21.15.123:0/481280615 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68e80b0f10 0x7f68e80af5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009870 tx=0x7f68d802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d476b47e2b745a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.377+0000 7f6222800640 1 -- 172.21.15.123:0/505929840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621c073bc0 msgr2=0x7f621c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.377+0000 7f6222800640 1 --2- 172.21.15.123:0/505929840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621c073bc0 0x7f621c0b5da0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f621c072cd0 tx=0x7f621402a210 comp rx=0 tx=0).stop 2024-04-24T05:46:24.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.377+0000 7f6222800640 1 -- 172.21.15.123:0/505929840 shutdown_connections 2024-04-24T05:46:24.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.377+0000 7f6222800640 1 --2- 172.21.15.123:0/505929840 >> v2:172.21.15.155:3300/0 conn(0x7f621c0ad150 0x7f621c0ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.377+0000 7f6222800640 1 --2- 172.21.15.123:0/505929840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621c073bc0 0x7f621c0b5da0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.377+0000 7f6222800640 1 -- 172.21.15.123:0/505929840 >> 172.21.15.123:0/505929840 conn(0x7f621c0b4fd0 msgr2=0x7f621c0b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.377+0000 7f3da7e00640 1 Processor -- start 2024-04-24T05:46:24.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.377+0000 7f3da7e00640 1 -- start start 2024-04-24T05:46:24.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.377+0000 7f68e6a00640 1 -- 172.21.15.123:0/481280615 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8037070 con 0x7f68e80b0f10 2024-04-24T05:46:24.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.377+0000 7f68e6a00640 1 -- 172.21.15.123:0/481280615 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d8032440 con 0x7f68e80b0f10 2024-04-24T05:46:24.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.377+0000 7f68e6a00640 1 -- 172.21.15.123:0/481280615 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8032810 con 0x7f68e80b0f10 2024-04-24T05:46:24.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7fee0d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee080d1500 0x7fee0815dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7fee0d000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee080d35f0 0x7fee08160a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7fee0d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee0815ebf0 0x7fee0806ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7fee0d000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee0806f3e0 con 0x7fee0815ebf0 2024-04-24T05:46:24.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7fee0d000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee0806f260 con 0x7fee080d35f0 2024-04-24T05:46:24.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7fee0d000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee080b6ce0 con 0x7fee080d1500 2024-04-24T05:46:24.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7f3da7e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3da003e840 0x7f3da004f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7f3da7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da0054130 0x7f3da0179f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7f6222800640 1 -- 172.21.15.123:0/505929840 shutdown_connections 2024-04-24T05:46:24.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7f3da7e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3da00e30c0 0x7f3da0052270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7f3da7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3da00e34a0 con 0x7f3da0054130 2024-04-24T05:46:24.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7f3da7e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3da003bfc0 con 0x7f3da00e30c0 2024-04-24T05:46:24.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7f6222800640 1 -- 172.21.15.123:0/505929840 wait complete. 2024-04-24T05:46:24.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7f3da7e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3da00078e0 con 0x7f3da003e840 2024-04-24T05:46:24.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7f68ee200640 1 -- 172.21.15.123:0/481280615 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68e80b0f10 msgr2=0x7f68e80af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7f68ee200640 1 --2- 172.21.15.123:0/481280615 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68e80b0f10 0x7f68e80af5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009870 tx=0x7f68d802a250 comp rx=0 tx=0).stop 2024-04-24T05:46:24.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7f3da6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da0054130 0x7f3da0179f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7f3da6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da0054130 0x7f3da0179f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56630/0 (socket says 172.21.15.123:56630) 2024-04-24T05:46:24.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7f3da6000640 1 -- 172.21.15.123:0/3738521259 learned_addr learned my addr 172.21.15.123:0/3738521259 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7fee05e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee080d1500 0x7fee0815dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7fee05400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee080d35f0 0x7fee08160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7fee05400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee080d35f0 0x7fee08160a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:50074/0 (socket says 172.21.15.123:50074) 2024-04-24T05:46:24.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7fee05400640 1 -- 172.21.15.123:0/2846387206 learned_addr learned my addr 172.21.15.123:0/2846387206 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7f68ee200640 1 -- 172.21.15.123:0/481280615 shutdown_connections 2024-04-24T05:46:24.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7f68ee200640 1 --2- 172.21.15.123:0/481280615 >> v2:172.21.15.123:3300/0 conn(0x7f68e80ce940 0x7f68e80afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7f68ee200640 1 --2- 172.21.15.123:0/481280615 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68e80b0f10 0x7f68e80af5c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7f68ee200640 1 -- 172.21.15.123:0/481280615 >> 172.21.15.123:0/481280615 conn(0x7f68e809a9e0 msgr2=0x7f68e80b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7f3da7400640 1 --2- 172.21.15.123:0/3738521259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3da00e30c0 0x7f3da0052270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7f68ee200640 1 -- 172.21.15.123:0/481280615 shutdown_connections 2024-04-24T05:46:24.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7f68ee200640 1 -- 172.21.15.123:0/481280615 wait complete. 2024-04-24T05:46:24.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7f3da6a00640 1 --2- 172.21.15.123:0/3738521259 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3da003e840 0x7f3da004f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.378+0000 7fc097400640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc08406c8f0 0x7fc08406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7fc097400640 1 -- 172.21.15.123:0/4249753357 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc09808be80 con 0x7fc09c0e32d0 2024-04-24T05:46:24.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.379+0000 7fc0a3400640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc08406c8f0 0x7fc08406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.380+0000 7f3da6000640 1 -- 172.21.15.123:0/3738521259 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3da003e840 msgr2=0x7f3da004f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.380+0000 7f3da6000640 1 --2- 172.21.15.123:0/3738521259 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3da003e840 0x7f3da004f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.380+0000 7f3da6000640 1 -- 172.21.15.123:0/3738521259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3da00e30c0 msgr2=0x7f3da0052270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.380+0000 7f3da6000640 1 --2- 172.21.15.123:0/3738521259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3da00e30c0 0x7f3da0052270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.380+0000 7f3da6000640 1 -- 172.21.15.123:0/3738521259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da004f720 con 0x7f3da0054130 2024-04-24T05:46:24.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.380+0000 7fc0a3400640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc08406c8f0 0x7fc08406edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc09c008680 tx=0x7fc0a406f030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.380+0000 7f3da6000640 1 --2- 172.21.15.123:0/3738521259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da0054130 0x7f3da0179f40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009870 tx=0x7f3d940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.380+0000 7fee05400640 1 -- 172.21.15.123:0/2846387206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee080d1500 msgr2=0x7fee0815dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.381+0000 7f6222800640 1 Processor -- start 2024-04-24T05:46:24.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.380+0000 7fee05400640 1 --2- 172.21.15.123:0/2846387206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee080d1500 0x7fee0815dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.380+0000 7fee05400640 1 -- 172.21.15.123:0/2846387206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee0815ebf0 msgr2=0x7fee0806ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.380+0000 7fee05400640 1 --2- 172.21.15.123:0/2846387206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee0815ebf0 0x7fee0806ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.380+0000 7fee05400640 1 -- 172.21.15.123:0/2846387206 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee081610f0 con 0x7fee080d35f0 2024-04-24T05:46:24.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.381+0000 7f6222800640 1 -- start start 2024-04-24T05:46:24.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.381+0000 7f68ee200640 1 Processor -- start 2024-04-24T05:46:24.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.381+0000 7fee05400640 1 --2- 172.21.15.123:0/2846387206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee080d35f0 0x7fee08160a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fedf400e900 tx=0x7fedf400edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.381+0000 7f68ee200640 1 -- start start 2024-04-24T05:46:24.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f6222800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f621c0ad150 0x7f621c164750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f6222800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621c15d360 0x7f621c09a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f6222800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f621c09d960 0x7f621c09ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f6222800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f621c15d740 con 0x7f621c15d360 2024-04-24T05:46:24.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f6222800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f621c06c140 con 0x7f621c09d960 2024-04-24T05:46:24.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f6222800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f621c06bdd0 con 0x7f621c0ad150 2024-04-24T05:46:24.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f68ee200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e80b0f10 0x7f68e8158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f68ee200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e80ce560 0x7f68e8147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f68ee200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68e80ce940 0x7f68e8147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f68ee200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68e80ccfa0 con 0x7f68e80ce560 2024-04-24T05:46:24.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f68ee200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f68e80af0a0 con 0x7f68e80b0f10 2024-04-24T05:46:24.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f68ee200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f68e80ae160 con 0x7f68e80ce940 2024-04-24T05:46:24.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f68dea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e80ce560 0x7f68e8147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f621aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621c15d360 0x7f621c09a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f621aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621c15d360 0x7f621c09a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56648/0 (socket says 172.21.15.123:56648) 2024-04-24T05:46:24.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f621aa00640 1 -- 172.21.15.123:0/2636357611 learned_addr learned my addr 172.21.15.123:0/2636357611 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f68dea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e80ce560 0x7f68e8147650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56656/0 (socket says 172.21.15.123:56656) 2024-04-24T05:46:24.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.382+0000 7f68dea00640 1 -- 172.21.15.123:0/3451533473 learned_addr learned my addr 172.21.15.123:0/3451533473 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.383+0000 7f621be00640 1 --2- 172.21.15.123:0/2636357611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f621c09d960 0x7f621c09ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.383+0000 7f68dea00640 1 -- 172.21.15.123:0/3451533473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68e80ce940 msgr2=0x7f68e8147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.383+0000 7f621b400640 1 --2- 172.21.15.123:0/2636357611 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f621c0ad150 0x7f621c164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.383+0000 7f621aa00640 1 -- 172.21.15.123:0/2636357611 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f621c0ad150 msgr2=0x7f621c164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.383+0000 7f621aa00640 1 --2- 172.21.15.123:0/2636357611 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f621c0ad150 0x7f621c164750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.383+0000 7f621aa00640 1 -- 172.21.15.123:0/2636357611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f621c09d960 msgr2=0x7f621c09ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.383+0000 7f621aa00640 1 --2- 172.21.15.123:0/2636357611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f621c09d960 0x7f621c09ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.383+0000 7f68dea00640 1 --2- 172.21.15.123:0/3451533473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68e80ce940 0x7f68e8147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.383+0000 7f68dea00640 1 -- 172.21.15.123:0/3451533473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e80b0f10 msgr2=0x7f68e8158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.383+0000 7f68dea00640 1 --2- 172.21.15.123:0/3451533473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e80b0f10 0x7f68e8158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.383+0000 7f68dea00640 1 -- 172.21.15.123:0/3451533473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e815bfa0 con 0x7f68e80ce560 2024-04-24T05:46:24.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.383+0000 7f621aa00640 1 -- 172.21.15.123:0/2636357611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621c15ed60 con 0x7f621c15d360 2024-04-24T05:46:24.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.383+0000 7f68dea00640 1 --2- 172.21.15.123:0/3451533473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e80ce560 0x7f68e8147650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009870 tx=0x7f68d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.383+0000 7f621aa00640 1 --2- 172.21.15.123:0/2636357611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621c15d360 0x7f621c09a260 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f62140090b0 tx=0x7f62140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.384+0000 7f3d9ea00640 1 -- 172.21.15.123:0/3738521259 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94041070 con 0x7f3da0054130 2024-04-24T05:46:24.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.384+0000 7f3d9ea00640 1 -- 172.21.15.123:0/3738521259 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d9403c680 con 0x7f3da0054130 2024-04-24T05:46:24.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.384+0000 7f3d9ea00640 1 -- 172.21.15.123:0/3738521259 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94037040 con 0x7f3da0054130 2024-04-24T05:46:24.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.384+0000 7f3da7e00640 1 -- 172.21.15.123:0/3738521259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da00dd1d0 con 0x7f3da0054130 2024-04-24T05:46:24.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.384+0000 7f3da7e00640 1 -- 172.21.15.123:0/3738521259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da00dd790 con 0x7f3da0054130 2024-04-24T05:46:24.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.384+0000 7f6213400640 1 -- 172.21.15.123:0/2636357611 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214037040 con 0x7f621c15d360 2024-04-24T05:46:24.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.384+0000 7fc097400640 1 -- 172.21.15.123:0/4249753357 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc098059880 con 0x7fc09c0e32d0 2024-04-24T05:46:24.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.384+0000 7f6213400640 1 -- 172.21.15.123:0/2636357611 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6214004230 con 0x7f621c15d360 2024-04-24T05:46:24.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.384+0000 7f68dfe00640 1 -- 172.21.15.123:0/3451533473 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8041070 con 0x7f68e80ce560 2024-04-24T05:46:24.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.384+0000 7f68dfe00640 1 -- 172.21.15.123:0/3451533473 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d803c890 con 0x7f68e80ce560 2024-04-24T05:46:24.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.384+0000 7f68dfe00640 1 -- 172.21.15.123:0/3451533473 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8037040 con 0x7f68e80ce560 2024-04-24T05:46:24.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.384+0000 7f68ee200640 1 -- 172.21.15.123:0/3451533473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e815b6a0 con 0x7f68e80ce560 2024-04-24T05:46:24.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.384+0000 7f68ee200640 1 -- 172.21.15.123:0/3451533473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e815c240 con 0x7f68e80ce560 2024-04-24T05:46:24.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.385+0000 7f3da7e00640 1 -- 172.21.15.123:0/3738521259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da00057e0 con 0x7f3da0054130 2024-04-24T05:46:24.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.386+0000 7f6213400640 1 -- 172.21.15.123:0/2636357611 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214003610 con 0x7f621c15d360 2024-04-24T05:46:24.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.386+0000 7f6222800640 1 -- 172.21.15.123:0/2636357611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621c15f040 con 0x7f621c15d360 2024-04-24T05:46:24.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.386+0000 7f6222800640 1 -- 172.21.15.123:0/2636357611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621c15f600 con 0x7f621c15d360 2024-04-24T05:46:24.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.387+0000 7fbc06000640 1 -- 172.21.15.123:0/1750534612 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc08010040 con 0x7fbc1015e610 2024-04-24T05:46:24.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.387+0000 7f1286a00640 1 -- 172.21.15.123:0/2951515772 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c017070 con 0x7f1288054130 2024-04-24T05:46:24.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.387+0000 7f6222800640 1 -- 172.21.15.123:0/2636357611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f621c09beb0 con 0x7f621c15d360 2024-04-24T05:46:24.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.387+0000 7f1286a00640 1 -- 172.21.15.123:0/2951515772 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127c00c940 con 0x7f1288054130 2024-04-24T05:46:24.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.387+0000 7f1286a00640 1 -- 172.21.15.123:0/2951515772 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c00ccf0 con 0x7f1288054130 2024-04-24T05:46:24.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.387+0000 7f3348a00640 1 -- 172.21.15.123:0/3961744483 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c037070 con 0x7f334c071e40 2024-04-24T05:46:24.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.387+0000 7fedfe000640 1 -- 172.21.15.123:0/2846387206 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4017070 con 0x7fee080d35f0 2024-04-24T05:46:24.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.387+0000 7f1294a00640 1 -- 172.21.15.123:0/2951515772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12880d9d30 con 0x7f1288054130 2024-04-24T05:46:24.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.387+0000 7fedfe000640 1 -- 172.21.15.123:0/2846387206 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf400ca00 con 0x7fee080d35f0 2024-04-24T05:46:24.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.387+0000 7fedfe000640 1 -- 172.21.15.123:0/2846387206 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf400cdd0 con 0x7fee080d35f0 2024-04-24T05:46:24.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.387+0000 7f3348a00640 1 -- 172.21.15.123:0/3961744483 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f333c032440 con 0x7f334c071e40 2024-04-24T05:46:24.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.387+0000 7f3348a00640 1 -- 172.21.15.123:0/3961744483 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c032810 con 0x7f334c071e40 2024-04-24T05:46:24.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.387+0000 7fee0d000640 1 -- 172.21.15.123:0/2846387206 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee081613d0 con 0x7fee080d35f0 2024-04-24T05:46:24.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.387+0000 7fee0d000640 1 -- 172.21.15.123:0/2846387206 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee08161990 con 0x7fee080d35f0 2024-04-24T05:46:24.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.388+0000 7f1294a00640 1 -- 172.21.15.123:0/2951515772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12880d6ca0 con 0x7f1288054130 2024-04-24T05:46:24.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.388+0000 7f3350a00640 1 -- 172.21.15.123:0/3961744483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f334c071e40 msgr2=0x7f334c072220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.388+0000 7f3350a00640 1 --2- 172.21.15.123:0/3961744483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f334c071e40 0x7f334c072220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f333c009870 tx=0x7f333c02a1d0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.389+0000 7f3d9ea00640 1 -- 172.21.15.123:0/3738521259 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d94044070 con 0x7f3da0054130 2024-04-24T05:46:24.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.389+0000 7f3350a00640 1 -- 172.21.15.123:0/3961744483 shutdown_connections 2024-04-24T05:46:24.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.389+0000 7f3350a00640 1 --2- 172.21.15.123:0/3961744483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f334c14c1d0 0x7f334c06efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.389+0000 7f3350a00640 1 --2- 172.21.15.123:0/3961744483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f334c071e40 0x7f334c072220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.389+0000 7f3350a00640 1 -- 172.21.15.123:0/3961744483 >> 172.21.15.123:0/3961744483 conn(0x7f334c0b1e70 msgr2=0x7f334c0b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.388+0000 7f68ee200640 1 -- 172.21.15.123:0/3451533473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e8068a20 con 0x7f68e80ce560 2024-04-24T05:46:24.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.390+0000 7fee0d000640 1 -- 172.21.15.123:0/2846387206 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0808ca80 con 0x7fee080d35f0 2024-04-24T05:46:24.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.390+0000 7f3d9ea00640 1 --2- 172.21.15.123:0/3738521259 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3d8806c9c0 0x7f3d8806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.390+0000 7f3d9ea00640 1 -- 172.21.15.123:0/3738521259 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3d940acb60 con 0x7f3da0054130 2024-04-24T05:46:24.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.390+0000 7f3da6a00640 1 --2- 172.21.15.123:0/3738521259 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3d8806c9c0 0x7f3d8806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.390+0000 7f3350a00640 1 -- 172.21.15.123:0/3961744483 shutdown_connections 2024-04-24T05:46:24.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.391+0000 7f3da6a00640 1 --2- 172.21.15.123:0/3738521259 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3d8806c9c0 0x7f3d8806ee80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009b40 tx=0x7f3d98009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.392+0000 7f3350a00640 1 -- 172.21.15.123:0/3961744483 wait complete. 2024-04-24T05:46:24.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.393+0000 7f3350a00640 1 Processor -- start 2024-04-24T05:46:24.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.392+0000 7f68dfe00640 1 -- 172.21.15.123:0/3451533473 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68d8044070 con 0x7f68e80ce560 2024-04-24T05:46:24.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.391+0000 7f1294a00640 1 -- 172.21.15.123:0/2951515772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1268005290 con 0x7f1288054130 2024-04-24T05:46:24.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.394+0000 7f3350a00640 1 -- start start 2024-04-24T05:46:24.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.393+0000 7f68dfe00640 1 --2- 172.21.15.123:0/3451533473 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f68c406c8f0 0x7f68c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.394+0000 7f68dfe00640 1 -- 172.21.15.123:0/3451533473 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f68d8078480 con 0x7f68e80ce560 2024-04-24T05:46:24.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.394+0000 7f68e7400640 1 --2- 172.21.15.123:0/3451533473 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f68c406c8f0 0x7f68c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.395+0000 7f1286a00640 1 -- 172.21.15.123:0/2951515772 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f127c01a070 con 0x7f1288054130 2024-04-24T05:46:24.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.395+0000 7f3350a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f334c071e40 0x7f334c1652e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.395+0000 7f3350a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f334c14c1d0 0x7f334c165820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.394+0000 7fedfe000640 1 -- 172.21.15.123:0/2846387206 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fedf401a070 con 0x7fee080d35f0 2024-04-24T05:46:24.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.395+0000 7f3350a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f334c15e9b0 0x7f334c15d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.395+0000 7f3350a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f334c06dbe0 con 0x7f334c14c1d0 2024-04-24T05:46:24.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.395+0000 7f3350a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f334c06e7d0 con 0x7f334c15e9b0 2024-04-24T05:46:24.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.395+0000 7f3350a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f334c14c720 con 0x7f334c071e40 2024-04-24T05:46:24.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.396+0000 7f3349400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f334c14c1d0 0x7f334c165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.395+0000 7f68e7400640 1 --2- 172.21.15.123:0/3451533473 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f68c406c8f0 0x7f68c406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f68d0001560 tx=0x7f68d0007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.396+0000 7fedfe000640 1 --2- 172.21.15.123:0/2846387206 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fede806c8f0 0x7fede806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.396+0000 7f3349e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f334c071e40 0x7f334c1652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.396+0000 7f334a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f334c15e9b0 0x7f334c15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.396+0000 7f3349400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f334c14c1d0 0x7f334c165820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56662/0 (socket says 172.21.15.123:56662) 2024-04-24T05:46:24.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.396+0000 7f3349400640 1 -- 172.21.15.123:0/3890041968 learned_addr learned my addr 172.21.15.123:0/3890041968 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:24.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.396+0000 7fedfe000640 1 -- 172.21.15.123:0/2846387206 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fedf408c2d0 con 0x7fee080d35f0 2024-04-24T05:46:24.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.396+0000 7f1286a00640 1 --2- 172.21.15.123:0/2951515772 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f127006caf0 0x7f127006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.396+0000 7fee05e00640 1 --2- 172.21.15.123:0/2846387206 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fede806c8f0 0x7fede806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.397+0000 7f3349e00640 1 -- 172.21.15.123:0/3890041968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f334c15e9b0 msgr2=0x7f334c15d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.397+0000 7f3349e00640 1 --2- 172.21.15.123:0/3890041968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f334c15e9b0 0x7f334c15d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.397+0000 7f3349e00640 1 -- 172.21.15.123:0/3890041968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f334c14c1d0 msgr2=0x7f334c165820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.397+0000 7f3349e00640 1 --2- 172.21.15.123:0/3890041968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f334c14c1d0 0x7f334c165820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.397+0000 7f128e800640 1 --2- 172.21.15.123:0/2951515772 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f127006caf0 0x7f127006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.397+0000 7f3349e00640 1 -- 172.21.15.123:0/3890041968 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c15ee40 con 0x7f334c071e40 2024-04-24T05:46:24.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.398+0000 7f3d9ea00640 1 -- 172.21.15.123:0/3738521259 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d9404b030 con 0x7f3da0054130 2024-04-24T05:46:24.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.398+0000 7f3349400640 1 --2- 172.21.15.123:0/3890041968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f334c14c1d0 0x7f334c165820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:46:24.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.398+0000 7f128e800640 1 --2- 172.21.15.123:0/2951515772 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f127006caf0 0x7f127006efb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f12880dcc40 tx=0x7f1280009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.398+0000 7f3349e00640 1 --2- 172.21.15.123:0/3890041968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f334c071e40 0x7f334c1652e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f334c0d1340 tx=0x7f333c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.399+0000 7fee05e00640 1 --2- 172.21.15.123:0/2846387206 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fede806c8f0 0x7fede806edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fedf8034000 tx=0x7fedf802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.399+0000 7f6213400640 1 -- 172.21.15.123:0/2636357611 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62140037b0 con 0x7f621c15d360 2024-04-24T05:46:24.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.401+0000 7f6213400640 1 --2- 172.21.15.123:0/2636357611 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f61f806c940 0x7f61f806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.401+0000 7f621b400640 1 --2- 172.21.15.123:0/2636357611 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f61f806c940 0x7f61f806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.402+0000 7f621b400640 1 --2- 172.21.15.123:0/2636357611 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f61f806c940 0x7f61f806ee00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f620c001d40 tx=0x7f620c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.404+0000 7f3342000640 1 -- 172.21.15.123:0/3890041968 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c041020 con 0x7f334c071e40 2024-04-24T05:46:24.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.404+0000 7f3342000640 1 -- 172.21.15.123:0/3890041968 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f333c030780 con 0x7f334c071e40 2024-04-24T05:46:24.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.404+0000 7f3342000640 1 -- 172.21.15.123:0/3890041968 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c037040 con 0x7f334c071e40 2024-04-24T05:46:24.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.404+0000 7f3350a00640 1 -- 172.21.15.123:0/3890041968 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334c15f120 con 0x7f334c071e40 2024-04-24T05:46:24.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.404+0000 7f3350a00640 1 -- 172.21.15.123:0/3890041968 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f334c15f6e0 con 0x7f334c071e40 2024-04-24T05:46:24.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.405+0000 7f3350a00640 1 -- 172.21.15.123:0/3890041968 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f334c068a20 con 0x7f334c071e40 2024-04-24T05:46:24.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.407+0000 7f6213400640 1 -- 172.21.15.123:0/2636357611 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6214078dc0 con 0x7f621c15d360 2024-04-24T05:46:24.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.407+0000 7fedfe000640 1 -- 172.21.15.123:0/2846387206 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee0808ca80 con 0x7fee080d35f0 2024-04-24T05:46:24.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.407+0000 7f68dfe00640 1 -- 172.21.15.123:0/3451533473 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f68d807aae0 con 0x7f68e80ce560 2024-04-24T05:46:24.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.408+0000 7f3342000640 1 -- 172.21.15.123:0/3890041968 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f333c044020 con 0x7f334c071e40 2024-04-24T05:46:24.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.409+0000 7f3342000640 1 --2- 172.21.15.123:0/3890041968 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f332806c9c0 0x7f332806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:24.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.409+0000 7f3342000640 1 -- 172.21.15.123:0/3890041968 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f333c0ac6d0 con 0x7f334c071e40 2024-04-24T05:46:24.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.412+0000 7f3349400640 1 --2- 172.21.15.123:0/3890041968 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f332806c9c0 0x7f332806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:24.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.412+0000 7f1286a00640 1 -- 172.21.15.123:0/2951515772 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f127c00caf0 con 0x7f1288054130 2024-04-24T05:46:24.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.414+0000 7f3349400640 1 --2- 172.21.15.123:0/3890041968 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f332806c9c0 0x7f332806ee80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3334007c00 tx=0x7f3334007b50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:24.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.415+0000 7f6213400640 1 -- 172.21.15.123:0/2636357611 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f621c09beb0 con 0x7f621c15d360 2024-04-24T05:46:24.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.426+0000 7f1286a00640 1 -- 172.21.15.123:0/2951515772 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f127c058f20 con 0x7f1288054130 2024-04-24T05:46:24.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.426+0000 7f3342000640 1 -- 172.21.15.123:0/3890041968 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f333c04b030 con 0x7f334c071e40 2024-04-24T05:46:24.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.796+0000 7fbc15000640 1 -- 172.21.15.123:0/1750534612 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fbc100d38d0 con 0x7fbc1015e610 2024-04-24T05:46:24.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.797+0000 7fbc06000640 1 -- 172.21.15.123:0/1750534612 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbc0805aea0 con 0x7fbc1015e610 2024-04-24T05:46:24.799 INFO:teuthology.orchestra.run.smithi123.stdout:55834574857 2024-04-24T05:46:24.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.801+0000 7fbc15000640 1 -- 172.21.15.123:0/1750534612 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbbf006c940 msgr2=0x7fbbf006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.801+0000 7fbc15000640 1 --2- 172.21.15.123:0/1750534612 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbbf006c940 0x7fbbf006ee00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc00006fd0 tx=0x7fbc00008040 comp rx=0 tx=0).stop 2024-04-24T05:46:24.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.801+0000 7fbc15000640 1 -- 172.21.15.123:0/1750534612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc1015e610 msgr2=0x7fbc1015adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.801+0000 7fbc15000640 1 --2- 172.21.15.123:0/1750534612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc1015e610 0x7fbc1015adb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800efc0 tx=0x7fbc0800c490 comp rx=0 tx=0).stop 2024-04-24T05:46:24.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.802+0000 7fbc15000640 1 -- 172.21.15.123:0/1750534612 shutdown_connections 2024-04-24T05:46:24.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.802+0000 7fbc15000640 1 --2- 172.21.15.123:0/1750534612 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbbf006c940 0x7fbbf006ee00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.802+0000 7fbc15000640 1 --2- 172.21.15.123:0/1750534612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc1015e9f0 0x7fbc1015b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.802+0000 7fbc15000640 1 --2- 172.21.15.123:0/1750534612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc1015e610 0x7fbc1015adb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.802+0000 7fbc15000640 1 --2- 172.21.15.123:0/1750534612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc100b3340 0x7fbc100b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.802+0000 7fbc15000640 1 -- 172.21.15.123:0/1750534612 >> 172.21.15.123:0/1750534612 conn(0x7fbc100d0f10 msgr2=0x7fbc100b83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.803+0000 7fbc15000640 1 -- 172.21.15.123:0/1750534612 shutdown_connections 2024-04-24T05:46:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.803+0000 7fbc15000640 1 -- 172.21.15.123:0/1750534612 wait complete. 2024-04-24T05:46:24.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.823+0000 7fc0a9000640 1 -- 172.21.15.123:0/4249753357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc09c0077b0 con 0x7fc09c0e32d0 2024-04-24T05:46:24.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.824+0000 7fc097400640 1 -- 172.21.15.123:0/4249753357 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc09805db80 con 0x7fc09c0e32d0 2024-04-24T05:46:24.829 INFO:teuthology.orchestra.run.smithi123.stdout:55834574856 2024-04-24T05:46:24.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.829+0000 7fc0a9000640 1 -- 172.21.15.123:0/4249753357 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc08406c8f0 msgr2=0x7fc08406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.829+0000 7fc0a9000640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc08406c8f0 0x7fc08406edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc09c008680 tx=0x7fc0a406f030 comp rx=0 tx=0).stop 2024-04-24T05:46:24.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.829+0000 7fc0a9000640 1 -- 172.21.15.123:0/4249753357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c0e32d0 msgr2=0x7fc09c04bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.829+0000 7fc0a9000640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c0e32d0 0x7fc09c04bb90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc09800d490 tx=0x7fc09800d950 comp rx=0 tx=0).stop 2024-04-24T05:46:24.831 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.0 2024-04-24T05:46:24.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.829+0000 7fc0a9000640 1 -- 172.21.15.123:0/4249753357 shutdown_connections 2024-04-24T05:46:24.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.829+0000 7fc0a9000640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc08406c8f0 0x7fc08406edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.829+0000 7fc0a9000640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c0e32d0 0x7fc09c04bb90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.829+0000 7fc0a9000640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc09c053190 0x7fc09c0e6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.829+0000 7fc0a9000640 1 --2- 172.21.15.123:0/4249753357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc09c04ff80 0x7fc09c0e4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.830+0000 7fc0a9000640 1 -- 172.21.15.123:0/4249753357 >> 172.21.15.123:0/4249753357 conn(0x7fc09c009fb0 msgr2=0x7fc09c0e1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.830+0000 7fc0a9000640 1 -- 172.21.15.123:0/4249753357 shutdown_connections 2024-04-24T05:46:24.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.830+0000 7fc0a9000640 1 -- 172.21.15.123:0/4249753357 wait complete. 2024-04-24T05:46:24.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.838+0000 7f3da7e00640 1 -- 172.21.15.123:0/3738521259 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3da003e150 con 0x7f3da0054130 2024-04-24T05:46:24.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.839+0000 7f3d9ea00640 1 -- 172.21.15.123:0/3738521259 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d9407ad50 con 0x7f3da0054130 2024-04-24T05:46:24.840 INFO:teuthology.orchestra.run.smithi123.stdout:55834574855 2024-04-24T05:46:24.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.839+0000 7fee0d000640 1 -- 172.21.15.123:0/2846387206 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fee080d12e0 con 0x7fee080d35f0 2024-04-24T05:46:24.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.840+0000 7fedfe000640 1 -- 172.21.15.123:0/2846387206 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fedf4059d50 con 0x7fee080d35f0 2024-04-24T05:46:24.842 INFO:teuthology.orchestra.run.smithi123.stdout:55834574857 2024-04-24T05:46:24.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.841+0000 7f68ee200640 1 -- 172.21.15.123:0/3451533473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f68e8070a60 con 0x7f68e80ce560 2024-04-24T05:46:24.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.842+0000 7f68dfe00640 1 -- 172.21.15.123:0/3451533473 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f68d807a3e0 con 0x7f68e80ce560 2024-04-24T05:46:24.844 INFO:teuthology.orchestra.run.smithi123.stdout:55834574857 2024-04-24T05:46:24.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.842+0000 7f3da7e00640 1 -- 172.21.15.123:0/3738521259 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3d8806c9c0 msgr2=0x7f3d8806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.842+0000 7f3da7e00640 1 --2- 172.21.15.123:0/3738521259 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3d8806c9c0 0x7f3d8806ee80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009b40 tx=0x7f3d98009190 comp rx=0 tx=0).stop 2024-04-24T05:46:24.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.842+0000 7f3da7e00640 1 -- 172.21.15.123:0/3738521259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da0054130 msgr2=0x7f3da0179f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.842+0000 7f3da7e00640 1 --2- 172.21.15.123:0/3738521259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da0054130 0x7f3da0179f40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009870 tx=0x7f3d940046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.843+0000 7f3da7e00640 1 -- 172.21.15.123:0/3738521259 shutdown_connections 2024-04-24T05:46:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.843+0000 7f3da7e00640 1 --2- 172.21.15.123:0/3738521259 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3d8806c9c0 0x7f3d8806ee80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.843+0000 7f3da7e00640 1 --2- 172.21.15.123:0/3738521259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3da00e30c0 0x7f3da0052270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.843+0000 7f3da7e00640 1 --2- 172.21.15.123:0/3738521259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da0054130 0x7f3da0179f40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.843+0000 7f3da7e00640 1 --2- 172.21.15.123:0/3738521259 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3da003e840 0x7f3da004f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.843+0000 7f3da7e00640 1 -- 172.21.15.123:0/3738521259 >> 172.21.15.123:0/3738521259 conn(0x7f3da00559e0 msgr2=0x7f3da00068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.843+0000 7f3da7e00640 1 -- 172.21.15.123:0/3738521259 shutdown_connections 2024-04-24T05:46:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.843+0000 7f3da7e00640 1 -- 172.21.15.123:0/3738521259 wait complete. 2024-04-24T05:46:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.844+0000 7fee0d000640 1 -- 172.21.15.123:0/2846387206 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fede806c8f0 msgr2=0x7fede806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.844+0000 7fee0d000640 1 --2- 172.21.15.123:0/2846387206 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fede806c8f0 0x7fede806edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fedf8034000 tx=0x7fedf802e000 comp rx=0 tx=0).stop 2024-04-24T05:46:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.844+0000 7fee0d000640 1 -- 172.21.15.123:0/2846387206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee080d35f0 msgr2=0x7fee08160a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.844+0000 7fee0d000640 1 --2- 172.21.15.123:0/2846387206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee080d35f0 0x7fee08160a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fedf400e900 tx=0x7fedf400edc0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.844+0000 7fee0d000640 1 -- 172.21.15.123:0/2846387206 shutdown_connections 2024-04-24T05:46:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.844+0000 7fee0d000640 1 --2- 172.21.15.123:0/2846387206 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fede806c8f0 0x7fede806edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.844+0000 7fee0d000640 1 --2- 172.21.15.123:0/2846387206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee0815ebf0 0x7fee0806ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.844+0000 7fee0d000640 1 --2- 172.21.15.123:0/2846387206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee080d35f0 0x7fee08160a30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.845+0000 7fee0d000640 1 --2- 172.21.15.123:0/2846387206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee080d1500 0x7fee0815dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.845+0000 7fee0d000640 1 -- 172.21.15.123:0/2846387206 >> 172.21.15.123:0/2846387206 conn(0x7fee080b9570 msgr2=0x7fee080b55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.845+0000 7fee0d000640 1 -- 172.21.15.123:0/2846387206 shutdown_connections 2024-04-24T05:46:24.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.845+0000 7fee0d000640 1 -- 172.21.15.123:0/2846387206 wait complete. 2024-04-24T05:46:24.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.846+0000 7f3350a00640 1 -- 172.21.15.123:0/3890041968 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f334c06c990 con 0x7f334c071e40 2024-04-24T05:46:24.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.846+0000 7f68ee200640 1 -- 172.21.15.123:0/3451533473 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f68c406c8f0 msgr2=0x7f68c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.846+0000 7f68ee200640 1 --2- 172.21.15.123:0/3451533473 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f68c406c8f0 0x7f68c406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f68d0001560 tx=0x7f68d0007930 comp rx=0 tx=0).stop 2024-04-24T05:46:24.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.846+0000 7f68ee200640 1 -- 172.21.15.123:0/3451533473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e80ce560 msgr2=0x7f68e8147650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.846+0000 7f68ee200640 1 --2- 172.21.15.123:0/3451533473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e80ce560 0x7f68e8147650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009870 tx=0x7f68d80046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.846+0000 7f68ee200640 1 -- 172.21.15.123:0/3451533473 shutdown_connections 2024-04-24T05:46:24.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.846+0000 7f68ee200640 1 --2- 172.21.15.123:0/3451533473 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f68c406c8f0 0x7f68c406edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.846+0000 7f68ee200640 1 --2- 172.21.15.123:0/3451533473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68e80ce940 0x7f68e8147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.846+0000 7f68ee200640 1 --2- 172.21.15.123:0/3451533473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e80ce560 0x7f68e8147650 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.846+0000 7f68ee200640 1 --2- 172.21.15.123:0/3451533473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68e80b0f10 0x7f68e8158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.846+0000 7f68ee200640 1 -- 172.21.15.123:0/3451533473 >> 172.21.15.123:0/3451533473 conn(0x7f68e809a9e0 msgr2=0x7f68e80cd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.846+0000 7f68ee200640 1 -- 172.21.15.123:0/3451533473 shutdown_connections 2024-04-24T05:46:24.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.847+0000 7f3342000640 1 -- 172.21.15.123:0/3890041968 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f333c07a150 con 0x7f334c071e40 2024-04-24T05:46:24.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.847+0000 7f68ee200640 1 -- 172.21.15.123:0/3451533473 wait complete. 2024-04-24T05:46:24.851 INFO:teuthology.orchestra.run.smithi123.stdout:55834574857 2024-04-24T05:46:24.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.850+0000 7f3350a00640 1 -- 172.21.15.123:0/3890041968 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f332806c9c0 msgr2=0x7f332806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.850+0000 7f3350a00640 1 --2- 172.21.15.123:0/3890041968 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f332806c9c0 0x7f332806ee80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3334007c00 tx=0x7f3334007b50 comp rx=0 tx=0).stop 2024-04-24T05:46:24.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.850+0000 7f3350a00640 1 -- 172.21.15.123:0/3890041968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f334c071e40 msgr2=0x7f334c1652e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.850+0000 7f3350a00640 1 --2- 172.21.15.123:0/3890041968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f334c071e40 0x7f334c1652e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f334c0d1340 tx=0x7f333c0046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.853+0000 7f3350a00640 1 -- 172.21.15.123:0/3890041968 shutdown_connections 2024-04-24T05:46:24.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.853+0000 7f3350a00640 1 --2- 172.21.15.123:0/3890041968 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f332806c9c0 0x7f332806ee80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.853+0000 7f3350a00640 1 --2- 172.21.15.123:0/3890041968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f334c15e9b0 0x7f334c15d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.853+0000 7f3350a00640 1 --2- 172.21.15.123:0/3890041968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f334c14c1d0 0x7f334c165820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.853+0000 7f3350a00640 1 --2- 172.21.15.123:0/3890041968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f334c071e40 0x7f334c1652e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.853+0000 7f3350a00640 1 -- 172.21.15.123:0/3890041968 >> 172.21.15.123:0/3890041968 conn(0x7f334c0b1e70 msgr2=0x7f334c06cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.853+0000 7f3350a00640 1 -- 172.21.15.123:0/3890041968 shutdown_connections 2024-04-24T05:46:24.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.853+0000 7f3350a00640 1 -- 172.21.15.123:0/3890041968 wait complete. 2024-04-24T05:46:24.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.2 2024-04-24T05:46:24.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.856+0000 7f6222800640 1 -- 172.21.15.123:0/2636357611 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f621c068a20 con 0x7f621c15d360 2024-04-24T05:46:24.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.858+0000 7f6213400640 1 -- 172.21.15.123:0/2636357611 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f621407b420 con 0x7f621c15d360 2024-04-24T05:46:24.860 INFO:teuthology.orchestra.run.smithi123.stdout:55834574856 2024-04-24T05:46:24.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.862+0000 7f6222800640 1 -- 172.21.15.123:0/2636357611 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f61f806c940 msgr2=0x7f61f806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.862+0000 7f6222800640 1 --2- 172.21.15.123:0/2636357611 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f61f806c940 0x7f61f806ee00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f620c001d40 tx=0x7f620c009040 comp rx=0 tx=0).stop 2024-04-24T05:46:24.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.862+0000 7f6222800640 1 -- 172.21.15.123:0/2636357611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621c15d360 msgr2=0x7f621c09a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.862+0000 7f6222800640 1 --2- 172.21.15.123:0/2636357611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621c15d360 0x7f621c09a260 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f62140090b0 tx=0x7f62140046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.862+0000 7f6222800640 1 -- 172.21.15.123:0/2636357611 shutdown_connections 2024-04-24T05:46:24.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.862+0000 7f6222800640 1 --2- 172.21.15.123:0/2636357611 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f61f806c940 0x7f61f806ee00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.862+0000 7f6222800640 1 --2- 172.21.15.123:0/2636357611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f621c09d960 0x7f621c09ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.862+0000 7f6222800640 1 --2- 172.21.15.123:0/2636357611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f621c15d360 0x7f621c09a260 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.862+0000 7f6222800640 1 --2- 172.21.15.123:0/2636357611 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f621c0ad150 0x7f621c164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.862+0000 7f6222800640 1 -- 172.21.15.123:0/2636357611 >> 172.21.15.123:0/2636357611 conn(0x7f621c0b4fd0 msgr2=0x7f621c0adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.862+0000 7f6222800640 1 -- 172.21.15.123:0/2636357611 shutdown_connections 2024-04-24T05:46:24.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.863+0000 7f6222800640 1 -- 172.21.15.123:0/2636357611 wait complete. 2024-04-24T05:46:24.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.864+0000 7f1294a00640 1 -- 172.21.15.123:0/2951515772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1268005580 con 0x7f1288054130 2024-04-24T05:46:24.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.865+0000 7f1286a00640 1 -- 172.21.15.123:0/2951515772 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f127c05d220 con 0x7f1288054130 2024-04-24T05:46:24.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.3 2024-04-24T05:46:24.868 INFO:teuthology.orchestra.run.smithi123.stdout:55834574855 2024-04-24T05:46:24.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.869+0000 7f1294a00640 1 -- 172.21.15.123:0/2951515772 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f127006caf0 msgr2=0x7f127006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.869+0000 7f1294a00640 1 --2- 172.21.15.123:0/2951515772 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f127006caf0 0x7f127006efb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f12880dcc40 tx=0x7f1280009040 comp rx=0 tx=0).stop 2024-04-24T05:46:24.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.869+0000 7f1294a00640 1 -- 172.21.15.123:0/2951515772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1288054130 msgr2=0x7f1288051430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:24.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.869+0000 7f1294a00640 1 --2- 172.21.15.123:0/2951515772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1288054130 0x7f1288051430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f127c00e850 tx=0x7f127c00ed10 comp rx=0 tx=0).stop 2024-04-24T05:46:24.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.869+0000 7f1294a00640 1 -- 172.21.15.123:0/2951515772 shutdown_connections 2024-04-24T05:46:24.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.869+0000 7f1294a00640 1 --2- 172.21.15.123:0/2951515772 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f127006caf0 0x7f127006efb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.869+0000 7f1294a00640 1 --2- 172.21.15.123:0/2951515772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12880e30c0 0x7f12880e1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.869+0000 7f1294a00640 1 --2- 172.21.15.123:0/2951515772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1288054130 0x7f1288051430 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.869+0000 7f1294a00640 1 --2- 172.21.15.123:0/2951515772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f128803e840 0x7f12880d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:24.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.869+0000 7f1294a00640 1 -- 172.21.15.123:0/2951515772 >> 172.21.15.123:0/2951515772 conn(0x7f12880559e0 msgr2=0x7f128803a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:24.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.871+0000 7f1294a00640 1 -- 172.21.15.123:0/2951515772 shutdown_connections 2024-04-24T05:46:24.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:24.871+0000 7f1294a00640 1 -- 172.21.15.123:0/2951515772 wait complete. 2024-04-24T05:46:24.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.7 2024-04-24T05:46:24.874 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.6 2024-04-24T05:46:24.876 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.1 2024-04-24T05:46:24.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.4 2024-04-24T05:46:24.890 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.5 2024-04-24T05:46:25.832 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T05:46:25.857 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T05:46:25.868 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T05:46:25.874 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T05:46:25.875 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T05:46:25.877 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T05:46:25.883 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T05:46:25.891 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T05:46:26.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.100+0000 7ff683200640 1 Processor -- start 2024-04-24T05:46:26.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.101+0000 7ff683200640 1 -- start start 2024-04-24T05:46:26.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.102+0000 7ff683200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff67c04fa90 0x7ff67c04fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.104+0000 7ff682800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff67c054270 0x7ff67c0389d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35592/0 (socket says 172.21.15.123:35592) 2024-04-24T05:46:26.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.104+0000 7ff681400640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7ff67c0df420 0x7ff67c0382c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.123:46316/0 (socket says 172.21.15.123:46316) 2024-04-24T05:46:26.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.104+0000 7ff682800640 1 -- 172.21.15.123:0/1545047720 learned_addr learned my addr 172.21.15.123:0/1545047720 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:26.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.105+0000 7ff683200640 1 -- 172.21.15.123:0/1545047720 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff67c0503b0 con 0x7ff67c054270 2024-04-24T05:46:26.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.106+0000 7ff683200640 1 -- 172.21.15.123:0/1545047720 --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff67c006af0 con 0x7ff67c0df420 2024-04-24T05:46:26.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.106+0000 7ff683200640 1 -- 172.21.15.123:0/1545047720 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff67c0068a0 con 0x7ff67c04fa90 2024-04-24T05:46:26.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.106+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1346563016 0 0) 0x7ff67c0503b0 con 0x7ff67c054270 2024-04-24T05:46:26.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.107+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff668003520 con 0x7ff67c054270 2024-04-24T05:46:26.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.107+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 686634981 0 0) 0x7ff67c006af0 con 0x7ff67c0df420 2024-04-24T05:46:26.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.107+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff67c0503b0 con 0x7ff67c0df420 2024-04-24T05:46:26.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.108+0000 7ff681e00640 1 --2- 172.21.15.123:0/1545047720 >> v2:172.21.15.123:3300/0 conn(0x7ff67c04fa90 0x7ff67c04fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.108+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2980833452 0 0) 0x7ff668003520 con 0x7ff67c054270 2024-04-24T05:46:26.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.108+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 >> v2:172.21.15.123:3300/0 conn(0x7ff67c04fa90 msgr2=0x7ff67c04fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.108+0000 7ff680a00640 1 --2- 172.21.15.123:0/1545047720 >> v2:172.21.15.123:3300/0 conn(0x7ff67c04fa90 0x7ff67c04fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.108+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 >> v1:172.21.15.155:6789/0 conn(0x7ff67c0df420 legacy=0x7ff67c0382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.109+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67c006dd0 con 0x7ff67c054270 2024-04-24T05:46:26.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.109+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7ff674001670 con 0x7ff67c054270 2024-04-24T05:46:26.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.109+0000 7ff681e00640 1 --2- 172.21.15.123:0/1545047720 >> v2:172.21.15.123:3300/0 conn(0x7ff67c04fa90 0x7ff67c04fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:46:26.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.110+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 >> v1:172.21.15.123:6789/0 conn(0x7ff67c054270 legacy=0x7ff67c0389d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.110+0000 7ff680a00640 1 --2- 172.21.15.123:0/1545047720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff668003b70 0x7ff668003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.110+0000 7ff680a00640 1 --2- 172.21.15.123:0/1545047720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668004800 0x7ff668004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.111+0000 7ff681400640 1 --2- 172.21.15.123:0/1545047720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668004800 0x7ff668004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.111+0000 7ff680a00640 1 --2- 172.21.15.123:0/1545047720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6680051c0 0x7ff66800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.111+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c0068a0 con 0x7ff668004800 2024-04-24T05:46:26.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.112+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff66800bfb0 con 0x7ff6680051c0 2024-04-24T05:46:26.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.112+0000 7ff682800640 1 --2- 172.21.15.123:0/1545047720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6680051c0 0x7ff66800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.112+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff66800c1b0 con 0x7ff668003b70 2024-04-24T05:46:26.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.113+0000 7ff681e00640 1 --2- 172.21.15.123:0/1545047720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff668003b70 0x7ff668003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.113+0000 7ff681400640 1 -- 172.21.15.123:0/1545047720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff668003b70 msgr2=0x7ff668003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.117+0000 7ff681400640 1 --2- 172.21.15.123:0/1545047720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff668003b70 0x7ff668003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.117+0000 7ff681400640 1 -- 172.21.15.123:0/1545047720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6680051c0 msgr2=0x7ff66800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.117+0000 7ff681400640 1 --2- 172.21.15.123:0/1545047720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6680051c0 0x7ff66800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.117+0000 7ff681400640 1 -- 172.21.15.123:0/1545047720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff668003520 con 0x7ff668004800 2024-04-24T05:46:26.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.117+0000 7ff682800640 1 --2- 172.21.15.123:0/1545047720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6680051c0 0x7ff66800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:46:26.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.117+0000 7ff681e00640 1 --2- 172.21.15.123:0/1545047720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff668003b70 0x7ff668003f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:26.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.117+0000 7ff682800640 1 -- 172.21.15.123:0/1545047720 reap_dead start 2024-04-24T05:46:26.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.118+0000 7ff681400640 1 --2- 172.21.15.123:0/1545047720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668004800 0x7ff668004c80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff684051690 tx=0x7ff668005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb305552d7a4f031 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.119+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684068a20 con 0x7ff668004800 2024-04-24T05:46:26.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.119+0000 7ff680a00640 1 -- 172.21.15.123:0/1545047720 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6840aa070 con 0x7ff668004800 2024-04-24T05:46:26.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.120+0000 7ff683200640 1 -- 172.21.15.123:0/1545047720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668004800 msgr2=0x7ff668004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.120+0000 7ff683200640 1 --2- 172.21.15.123:0/1545047720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668004800 0x7ff668004c80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff684051690 tx=0x7ff668005830 comp rx=0 tx=0).stop 2024-04-24T05:46:26.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.121+0000 7ff683200640 1 -- 172.21.15.123:0/1545047720 shutdown_connections 2024-04-24T05:46:26.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.121+0000 7ff683200640 1 --2- 172.21.15.123:0/1545047720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668004800 0x7ff668004c80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.121+0000 7ff683200640 1 -- 172.21.15.123:0/1545047720 >> 172.21.15.123:0/1545047720 conn(0x7ff67c009fb0 msgr2=0x7ff67c00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.122+0000 7ff683200640 1 -- 172.21.15.123:0/1545047720 shutdown_connections 2024-04-24T05:46:26.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.122+0000 7ff683200640 1 -- 172.21.15.123:0/1545047720 wait complete. 2024-04-24T05:46:26.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.125+0000 7ff683200640 1 Processor -- start 2024-04-24T05:46:26.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.125+0000 7ff683200640 1 -- start start 2024-04-24T05:46:26.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.128+0000 7ff683200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668004800 0x7ff67c1858e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.128+0000 7ff683200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff67c0d4090 0x7ff67c04d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.129+0000 7ff683200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff67c0d4510 0x7ff67c04d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.129+0000 7ff683200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c052b40 con 0x7ff668004800 2024-04-24T05:46:26.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.129+0000 7ff683200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c007390 con 0x7ff67c0d4510 2024-04-24T05:46:26.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.130+0000 7ff683200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff67c052cc0 con 0x7ff67c0d4090 2024-04-24T05:46:26.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.130+0000 7ff681400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff67c0d4090 0x7ff67c04d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.130+0000 7ff682800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff67c0d4510 0x7ff67c04d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.130+0000 7ff682800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff67c0d4510 0x7ff67c04d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:50118/0 (socket says 172.21.15.123:50118) 2024-04-24T05:46:26.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.130+0000 7ff682800640 1 -- 172.21.15.123:0/22517481 learned_addr learned my addr 172.21.15.123:0/22517481 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:26.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.130+0000 7ff681e00640 1 --2- 172.21.15.123:0/22517481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668004800 0x7ff67c1858e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.132+0000 7ff682800640 1 -- 172.21.15.123:0/22517481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff67c0d4090 msgr2=0x7ff67c04d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.132+0000 7ff682800640 1 --2- 172.21.15.123:0/22517481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff67c0d4090 0x7ff67c04d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.132+0000 7ff682800640 1 -- 172.21.15.123:0/22517481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668004800 msgr2=0x7ff67c1858e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.132+0000 7ff682800640 1 --2- 172.21.15.123:0/22517481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668004800 0x7ff67c1858e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.132+0000 7ff682800640 1 -- 172.21.15.123:0/22517481 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67c04cb60 con 0x7ff67c0d4510 2024-04-24T05:46:26.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.133+0000 7ff681e00640 1 --2- 172.21.15.123:0/22517481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668004800 0x7ff67c1858e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:46:26.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.133+0000 7ff682800640 1 --2- 172.21.15.123:0/22517481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff67c0d4510 0x7ff67c04d960 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00bd20 tx=0x7ff67400f530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.134+0000 7ff67a000640 1 -- 172.21.15.123:0/22517481 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67401a070 con 0x7ff67c0d4510 2024-04-24T05:46:26.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.135+0000 7ff683200640 1 -- 172.21.15.123:0/22517481 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff67c0df8a0 con 0x7ff67c0d4510 2024-04-24T05:46:26.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.135+0000 7ff683200640 1 -- 172.21.15.123:0/22517481 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff67c0dfe60 con 0x7ff67c0d4510 2024-04-24T05:46:26.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.135+0000 7ff67a000640 1 -- 172.21.15.123:0/22517481 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff674008730 con 0x7ff67c0d4510 2024-04-24T05:46:26.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.135+0000 7ff67a000640 1 -- 172.21.15.123:0/22517481 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff674004b60 con 0x7ff67c0d4510 2024-04-24T05:46:26.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.137+0000 7ff683200640 1 -- 172.21.15.123:0/22517481 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff65c005290 con 0x7ff67c0d4510 2024-04-24T05:46:26.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.140+0000 7ff67a000640 1 -- 172.21.15.123:0/22517481 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff67401d070 con 0x7ff67c0d4510 2024-04-24T05:46:26.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.141+0000 7ff67a000640 1 --2- 172.21.15.123:0/22517481 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff66006c9f0 0x7ff66006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.141+0000 7ff67a000640 1 -- 172.21.15.123:0/22517481 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff67408b4b0 con 0x7ff67c0d4510 2024-04-24T05:46:26.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.143+0000 7ff681e00640 1 --2- 172.21.15.123:0/22517481 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff66006c9f0 0x7ff66006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.144+0000 7ff681e00640 1 --2- 172.21.15.123:0/22517481 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff66006c9f0 0x7ff66006eeb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00a790 tx=0x7ff66c00a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.151+0000 7ff49c400640 1 Processor -- start 2024-04-24T05:46:26.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.151+0000 7ff49c400640 1 -- start start 2024-04-24T05:46:26.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.153+0000 7ff49c400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff494155020 0x7ff49415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.153+0000 7ff49c400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff49415ae30 0x7ff49406ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.153+0000 7ff49c400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff49406c7d0 con 0x7ff4940b5da0 2024-04-24T05:46:26.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.153+0000 7ff49c400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff49415c5d0 con 0x7ff494155020 2024-04-24T05:46:26.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.153+0000 7ff49c400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff49415c8b0 con 0x7ff49415ae30 2024-04-24T05:46:26.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.153+0000 7ff499a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff4940b5da0 0x7ff494158a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35602/0 (socket says 172.21.15.123:35602) 2024-04-24T05:46:26.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.153+0000 7ff499a00640 1 -- 172.21.15.123:0/81583928 learned_addr learned my addr 172.21.15.123:0/81583928 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:26.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.154+0000 7ff499000640 1 --2- 172.21.15.123:0/81583928 >> v2:172.21.15.155:3300/0 conn(0x7ff494155020 0x7ff49415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.154+0000 7ff49a400640 1 --2- 172.21.15.123:0/81583928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff49415ae30 0x7ff49406ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.154+0000 7ff493e00640 1 -- 172.21.15.123:0/81583928 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1711755879 0 0) 0x7ff49406c7d0 con 0x7ff4940b5da0 2024-04-24T05:46:26.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.154+0000 7ff493e00640 1 -- 172.21.15.123:0/81583928 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff47c003520 con 0x7ff4940b5da0 2024-04-24T05:46:26.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.155+0000 7ff493e00640 1 -- 172.21.15.123:0/81583928 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4097786370 0 0) 0x7ff47c003520 con 0x7ff4940b5da0 2024-04-24T05:46:26.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.155+0000 7ff493e00640 1 -- 172.21.15.123:0/81583928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff49415ae30 msgr2=0x7ff49406ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.155+0000 7ff493e00640 1 --2- 172.21.15.123:0/81583928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff49415ae30 0x7ff49406ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.155+0000 7ff493e00640 1 -- 172.21.15.123:0/81583928 >> v2:172.21.15.155:3300/0 conn(0x7ff494155020 msgr2=0x7ff49415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.155+0000 7ff493e00640 1 --2- 172.21.15.123:0/81583928 >> v2:172.21.15.155:3300/0 conn(0x7ff494155020 0x7ff49415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.155+0000 7ff493e00640 1 -- 172.21.15.123:0/81583928 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49415cb90 con 0x7ff4940b5da0 2024-04-24T05:46:26.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.155+0000 7ff493e00640 1 -- 172.21.15.123:0/81583928 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7ff488001670 con 0x7ff4940b5da0 2024-04-24T05:46:26.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.155+0000 7ff493e00640 1 -- 172.21.15.123:0/81583928 >> v1:172.21.15.123:6789/0 conn(0x7ff4940b5da0 legacy=0x7ff494158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.155+0000 7ff493e00640 1 --2- 172.21.15.123:0/81583928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff47c0039b0 0x7ff47c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.155+0000 7ff493e00640 1 --2- 172.21.15.123:0/81583928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff47c0043a0 0x7ff47c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.155+0000 7ff493e00640 1 --2- 172.21.15.123:0/81583928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff47c004ef0 0x7ff47c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.155+0000 7ff493e00640 1 -- 172.21.15.123:0/81583928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff49415c5d0 con 0x7ff47c004ef0 2024-04-24T05:46:26.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.155+0000 7ff493e00640 1 -- 172.21.15.123:0/81583928 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff49415c8b0 con 0x7ff47c0043a0 2024-04-24T05:46:26.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.155+0000 7ff493e00640 1 -- 172.21.15.123:0/81583928 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff47c00bf00 con 0x7ff47c0039b0 2024-04-24T05:46:26.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.156+0000 7ff49a400640 1 --2- 172.21.15.123:0/81583928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff47c004ef0 0x7ff47c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.156+0000 7ff499000640 1 --2- 172.21.15.123:0/81583928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff47c0043a0 0x7ff47c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.156+0000 7ff67a000640 1 -- 172.21.15.123:0/22517481 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff674058fc0 con 0x7ff67c0d4510 2024-04-24T05:46:26.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.157+0000 7ff49a400640 1 -- 172.21.15.123:0/81583928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff47c0039b0 msgr2=0x7ff47c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.157+0000 7ff499a00640 1 --2- 172.21.15.123:0/81583928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff47c0039b0 0x7ff47c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.157+0000 7ff49a400640 1 --2- 172.21.15.123:0/81583928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff47c0039b0 0x7ff47c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.157+0000 7ff49a400640 1 -- 172.21.15.123:0/81583928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff47c0043a0 msgr2=0x7ff47c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.157+0000 7ff49a400640 1 --2- 172.21.15.123:0/81583928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff47c0043a0 0x7ff47c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.157+0000 7ff49a400640 1 -- 172.21.15.123:0/81583928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49415cb90 con 0x7ff47c004ef0 2024-04-24T05:46:26.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.159+0000 7ff49a400640 1 --2- 172.21.15.123:0/81583928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff47c004ef0 0x7ff47c00b9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff49415b210 tx=0x7ff47c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc52dfe218e18b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.159+0000 7eff3a800640 1 Processor -- start 2024-04-24T05:46:26.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.160+0000 7ff493e00640 1 -- 172.21.15.123:0/81583928 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c036070 con 0x7ff47c004ef0 2024-04-24T05:46:26.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.160+0000 7ff493e00640 1 -- 172.21.15.123:0/81583928 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c030750 con 0x7ff47c004ef0 2024-04-24T05:46:26.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.160+0000 7ff493e00640 1 -- 172.21.15.123:0/81583928 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c00dde0 con 0x7ff47c004ef0 2024-04-24T05:46:26.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.159+0000 7eff3a800640 1 -- start start 2024-04-24T05:46:26.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.161+0000 7eff3a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff340d35f0 0x7eff340b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.161+0000 7eff3a800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7eff340d1500 0x7eff340d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.161+0000 7eff3a800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff340d3af0 con 0x7eff3415ebf0 2024-04-24T05:46:26.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.161+0000 7eff3a800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7eff340b6030 con 0x7eff340d1500 2024-04-24T05:46:26.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.161+0000 7eff3a800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7eff340b6310 con 0x7eff340d35f0 2024-04-24T05:46:26.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.161+0000 7ff49c400640 1 -- 172.21.15.123:0/81583928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff47c004ef0 msgr2=0x7ff47c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.161+0000 7ff49c400640 1 --2- 172.21.15.123:0/81583928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff47c004ef0 0x7ff47c00b9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff49415b210 tx=0x7ff47c005830 comp rx=0 tx=0).stop 2024-04-24T05:46:26.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.161+0000 7ff49a400640 1 -- 172.21.15.123:0/81583928 reap_dead start 2024-04-24T05:46:26.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.162+0000 7eff33e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7eff340d1500 0x7eff340d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.162+0000 7eff33e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7eff340d1500 0x7eff340d1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56722/0 (socket says 172.21.15.123:56722) 2024-04-24T05:46:26.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.162+0000 7eff33e00640 1 -- 172.21.15.123:0/1785671383 learned_addr learned my addr 172.21.15.123:0/1785671383 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:26.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.162+0000 7eff32a00640 1 --2- 172.21.15.123:0/1785671383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff340d35f0 0x7eff340b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.162+0000 7ff49c400640 1 -- 172.21.15.123:0/81583928 shutdown_connections 2024-04-24T05:46:26.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.163+0000 7ff49c400640 1 -- 172.21.15.123:0/81583928 >> 172.21.15.123:0/81583928 conn(0x7ff4940b4fd0 msgr2=0x7ff4940b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.163+0000 7f55ce400640 1 Processor -- start 2024-04-24T05:46:26.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.163+0000 7eff32a00640 1 -- 172.21.15.123:0/1785671383 >> v2:172.21.15.123:3300/0 conn(0x7eff340d1500 msgr2=0x7eff340d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.163+0000 7ff49c400640 1 -- 172.21.15.123:0/81583928 shutdown_connections 2024-04-24T05:46:26.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.163+0000 7ff49c400640 1 -- 172.21.15.123:0/81583928 wait complete. 2024-04-24T05:46:26.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.163+0000 7f55ce400640 1 -- start start 2024-04-24T05:46:26.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.163+0000 7eff32a00640 1 --2- 172.21.15.123:0/1785671383 >> v2:172.21.15.123:3300/0 conn(0x7eff340d1500 0x7eff340d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.163+0000 7eff32a00640 1 -- 172.21.15.123:0/1785671383 >> v1:172.21.15.155:6789/0 conn(0x7eff3415ebf0 legacy=0x7eff3406ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.163+0000 7eff32a00640 1 -- 172.21.15.123:0/1785671383 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff340d3dd0 con 0x7eff340d35f0 2024-04-24T05:46:26.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.165+0000 7eff33e00640 1 --2- 172.21.15.123:0/1785671383 >> v2:172.21.15.123:3300/0 conn(0x7eff340d1500 0x7eff340d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:46:26.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.165+0000 7eff32a00640 1 --2- 172.21.15.123:0/1785671383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff340d35f0 0x7eff340b67a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7eff240097a0 tx=0x7eff2402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=857dabcde540265f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.166+0000 7eff32000640 1 -- 172.21.15.123:0/1785671383 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24037070 con 0x7eff340d35f0 2024-04-24T05:46:26.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.166+0000 7f55ce400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f55c0006200 0x7f55c0056fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.166+0000 7f55ce400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55c0057a70 con 0x7f55c00574f0 2024-04-24T05:46:26.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.166+0000 7f55ce400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55c0057d50 con 0x7f55c00075f0 2024-04-24T05:46:26.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.166+0000 7f55ce400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f55c0058030 con 0x7f55c0006200 2024-04-24T05:46:26.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.166+0000 7eff32000640 1 -- 172.21.15.123:0/1785671383 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff24032440 con 0x7eff340d35f0 2024-04-24T05:46:26.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.166+0000 7eff32000640 1 -- 172.21.15.123:0/1785671383 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24032850 con 0x7eff340d35f0 2024-04-24T05:46:26.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.166+0000 7ff49c400640 1 Processor -- start 2024-04-24T05:46:26.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.167+0000 7f55cda00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f55c00075f0 0x7f55c003c720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.123:46346/0 (socket says 172.21.15.123:46346) 2024-04-24T05:46:26.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.167+0000 7f55cda00640 1 -- 172.21.15.123:0/1671674010 learned_addr learned my addr 172.21.15.123:0/1671674010 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:26.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.167+0000 7f55cd000640 1 --2- 172.21.15.123:0/1671674010 >> v2:172.21.15.155:3300/0 conn(0x7f55c0006200 0x7f55c0056fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.167+0000 7ff49c400640 1 -- start start 2024-04-24T05:46:26.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.167+0000 7eff3a800640 1 -- 172.21.15.123:0/1785671383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff340d35f0 msgr2=0x7eff340b67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.167+0000 7eff3a800640 1 --2- 172.21.15.123:0/1785671383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff340d35f0 0x7eff340b67a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7eff240097a0 tx=0x7eff2402a1a0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.168+0000 7eff3a800640 1 -- 172.21.15.123:0/1785671383 shutdown_connections 2024-04-24T05:46:26.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.168+0000 7eff3a800640 1 --2- 172.21.15.123:0/1785671383 >> v2:172.21.15.123:3300/0 conn(0x7eff340d1500 0x7eff340d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.168+0000 7eff3a800640 1 --2- 172.21.15.123:0/1785671383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff340d35f0 0x7eff340b67a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.168+0000 7eff3a800640 1 -- 172.21.15.123:0/1785671383 >> 172.21.15.123:0/1785671383 conn(0x7eff340b9570 msgr2=0x7eff340b9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.168+0000 7eff3a800640 1 -- 172.21.15.123:0/1785671383 shutdown_connections 2024-04-24T05:46:26.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.168+0000 7f55c7400640 1 -- 172.21.15.123:0/1671674010 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3186932178 0 0) 0x7f55c0057a70 con 0x7f55c00574f0 2024-04-24T05:46:26.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.168+0000 7f55c7400640 1 -- 172.21.15.123:0/1671674010 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55b0003520 con 0x7f55c00574f0 2024-04-24T05:46:26.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.169+0000 7eff3a800640 1 -- 172.21.15.123:0/1785671383 wait complete. 2024-04-24T05:46:26.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.171+0000 7ff49c400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff47c004ef0 0x7ff494154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.171+0000 7ff49c400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4940cff40 0x7ff4940ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.169+0000 7f55c7400640 1 -- 172.21.15.123:0/1671674010 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3400718188 0 0) 0x7f55c0057d50 con 0x7f55c00075f0 2024-04-24T05:46:26.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.169+0000 7f55c7400640 1 -- 172.21.15.123:0/1671674010 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55c0057a70 con 0x7f55c00075f0 2024-04-24T05:46:26.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.169+0000 7f55cd000640 1 -- 172.21.15.123:0/1671674010 >> v1:172.21.15.155:6789/0 conn(0x7f55c00075f0 legacy=0x7f55c003c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.169+0000 7f55cd000640 1 -- 172.21.15.123:0/1671674010 >> v1:172.21.15.123:6789/0 conn(0x7f55c00574f0 legacy=0x7f55c003c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.169+0000 7f55cd000640 1 -- 172.21.15.123:0/1671674010 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c0058310 con 0x7f55c0006200 2024-04-24T05:46:26.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.172+0000 7ff49c400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4940d03c0 0x7ff4940cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.172+0000 7f55cd000640 1 --2- 172.21.15.123:0/1671674010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0006200 0x7f55c0056fb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0097a0 tx=0x7f55bc02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c36288b7667482cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.172+0000 7ff49c400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff49406ba00 con 0x7ff4940d03c0 2024-04-24T05:46:26.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.173+0000 7eff3a800640 1 Processor -- start 2024-04-24T05:46:26.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.172+0000 7ff49c400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff49415d150 con 0x7ff47c004ef0 2024-04-24T05:46:26.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.172+0000 7ff49c400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff49406bb80 con 0x7ff4940cff40 2024-04-24T05:46:26.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.173+0000 7ff499a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff47c004ef0 0x7ff494154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.173+0000 7ff499000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4940cff40 0x7ff4940ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.173+0000 7ff499a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff47c004ef0 0x7ff494154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:50152/0 (socket says 172.21.15.123:50152) 2024-04-24T05:46:26.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.173+0000 7ff499a00640 1 -- 172.21.15.123:0/1387968442 learned_addr learned my addr 172.21.15.123:0/1387968442 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:26.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.173+0000 7ff49a400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4940d03c0 0x7ff4940cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.173+0000 7eff3a800640 1 -- start start 2024-04-24T05:46:26.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.173+0000 7f55c7400640 1 -- 172.21.15.123:0/1671674010 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc037070 con 0x7f55c0006200 2024-04-24T05:46:26.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.173+0000 7f55c7400640 1 -- 172.21.15.123:0/1671674010 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55bc032440 con 0x7f55c0006200 2024-04-24T05:46:26.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.174+0000 7ff49a400640 1 -- 172.21.15.123:0/1387968442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4940cff40 msgr2=0x7ff4940ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.174+0000 7ff49a400640 1 --2- 172.21.15.123:0/1387968442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4940cff40 0x7ff4940ccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.174+0000 7ff49a400640 1 -- 172.21.15.123:0/1387968442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff47c004ef0 msgr2=0x7ff494154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.174+0000 7ff49a400640 1 --2- 172.21.15.123:0/1387968442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff47c004ef0 0x7ff494154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.174+0000 7ff49a400640 1 -- 172.21.15.123:0/1387968442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4940cef10 con 0x7ff4940d03c0 2024-04-24T05:46:26.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.174+0000 7f55ce400640 1 -- 172.21.15.123:0/1671674010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0006200 msgr2=0x7f55c0056fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.174+0000 7f55ce400640 1 --2- 172.21.15.123:0/1671674010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0006200 0x7f55c0056fb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0097a0 tx=0x7f55bc02a270 comp rx=0 tx=0).stop 2024-04-24T05:46:26.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.174+0000 7f55ce400640 1 -- 172.21.15.123:0/1671674010 shutdown_connections 2024-04-24T05:46:26.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.174+0000 7f55ce400640 1 --2- 172.21.15.123:0/1671674010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c0006200 0x7f55c0056fb0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0097a0 tx=0x7f55bc02a270 comp rx=0 tx=0).stop 2024-04-24T05:46:26.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.174+0000 7f55ce400640 1 -- 172.21.15.123:0/1671674010 >> 172.21.15.123:0/1671674010 conn(0x7f55c0058ee0 msgr2=0x7f55c00592f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.175+0000 7f55ce400640 1 -- 172.21.15.123:0/1671674010 shutdown_connections 2024-04-24T05:46:26.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.175+0000 7eff3a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff340d1500 0x7eff34161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.175+0000 7ff49a400640 1 --2- 172.21.15.123:0/1387968442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4940d03c0 0x7ff4940cd460 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff48c035000 tx=0x7ff48c00d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.176+0000 7eff3a800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff340d35f0 0x7eff3415d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.176+0000 7eff3a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff3415ebf0 0x7eff3406da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.176+0000 7ff491600640 1 -- 172.21.15.123:0/1387968442 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c036040 con 0x7ff4940d03c0 2024-04-24T05:46:26.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.176+0000 7ff491600640 1 -- 172.21.15.123:0/1387968442 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c042070 con 0x7ff4940d03c0 2024-04-24T05:46:26.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.176+0000 7eff3a800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff340b6e60 con 0x7eff340d1500 2024-04-24T05:46:26.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.176+0000 7ff49c400640 1 -- 172.21.15.123:0/1387968442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4941fcc30 con 0x7ff4940d03c0 2024-04-24T05:46:26.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.177+0000 7ff491600640 1 -- 172.21.15.123:0/1387968442 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c031470 con 0x7ff4940d03c0 2024-04-24T05:46:26.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.175+0000 7f55ce400640 1 -- 172.21.15.123:0/1671674010 wait complete. 2024-04-24T05:46:26.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.178+0000 7f55ce400640 1 Processor -- start 2024-04-24T05:46:26.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.177+0000 7eff32a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff340d35f0 0x7eff3415d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.177+0000 7eff3a800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7eff340b6ce0 con 0x7eff340d35f0 2024-04-24T05:46:26.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.177+0000 7eff3a800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7eff341625e0 con 0x7eff3415ebf0 2024-04-24T05:46:26.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.177+0000 7eff32a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff340d35f0 0x7eff3415d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:50156/0 (socket says 172.21.15.123:50156) 2024-04-24T05:46:26.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.177+0000 7eff32a00640 1 -- 172.21.15.123:0/1998362936 learned_addr learned my addr 172.21.15.123:0/1998362936 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:26.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.177+0000 7eff33e00640 1 --2- 172.21.15.123:0/1998362936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff3415ebf0 0x7eff3406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.178+0000 7eff32a00640 1 -- 172.21.15.123:0/1998362936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff3415ebf0 msgr2=0x7eff3406da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.178+0000 7eff32a00640 1 --2- 172.21.15.123:0/1998362936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff3415ebf0 0x7eff3406da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.178+0000 7eff32a00640 1 -- 172.21.15.123:0/1998362936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff340d1500 msgr2=0x7eff34161cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.178+0000 7eff32a00640 1 --2- 172.21.15.123:0/1998362936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff340d1500 0x7eff34161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.178+0000 7eff32a00640 1 -- 172.21.15.123:0/1998362936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3415ff50 con 0x7eff340d35f0 2024-04-24T05:46:26.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.177+0000 7ff49c400640 1 -- 172.21.15.123:0/1387968442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4940cf460 con 0x7ff4940d03c0 2024-04-24T05:46:26.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.179+0000 7f55ce400640 1 -- start start 2024-04-24T05:46:26.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.180+0000 7f55ce400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55c00075f0 0x7f55c0038ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.180+0000 7f55ce400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55c00574f0 0x7f55c00e4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.180+0000 7f55ce400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c00e4cd0 0x7f55c0178a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.180+0000 7f55ce400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55c003edd0 con 0x7f55c00574f0 2024-04-24T05:46:26.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.180+0000 7f55ce400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f55c00e3520 con 0x7f55c00e4cd0 2024-04-24T05:46:26.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.180+0000 7f55ce400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f55c0007a40 con 0x7f55c00075f0 2024-04-24T05:46:26.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.181+0000 7f55c7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55c00574f0 0x7f55c00e4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.181+0000 7f55c7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55c00574f0 0x7f55c00e4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56750/0 (socket says 172.21.15.123:56750) 2024-04-24T05:46:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.181+0000 7f55c7e00640 1 -- 172.21.15.123:0/901733879 learned_addr learned my addr 172.21.15.123:0/901733879 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.181+0000 7f55cd000640 1 --2- 172.21.15.123:0/901733879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55c00075f0 0x7f55c0038ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.181+0000 7f55cda00640 1 --2- 172.21.15.123:0/901733879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c00e4cd0 0x7f55c0178a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.180+0000 7eff32a00640 1 --2- 172.21.15.123:0/1998362936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff340d35f0 0x7eff3415d340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff24035040 tx=0x7eff240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.180+0000 7eff2b400640 1 -- 172.21.15.123:0/1998362936 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24041030 con 0x7eff340d35f0 2024-04-24T05:46:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.181+0000 7eff2b400640 1 -- 172.21.15.123:0/1998362936 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff24032e20 con 0x7eff340d35f0 2024-04-24T05:46:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.181+0000 7eff2b400640 1 -- 172.21.15.123:0/1998362936 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24037040 con 0x7eff340d35f0 2024-04-24T05:46:26.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.180+0000 7f74be000640 1 Processor -- start 2024-04-24T05:46:26.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.180+0000 7f74be000640 1 -- start start 2024-04-24T05:46:26.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.179+0000 7ff49c400640 1 -- 172.21.15.123:0/1387968442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff494068a20 con 0x7ff4940d03c0 2024-04-24T05:46:26.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.182+0000 7f74be000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b80d35f0 0x7f74b80b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.182+0000 7f74be000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f74b80d1500 0x7f74b80d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.182+0000 7f8851c00640 1 Processor -- start 2024-04-24T05:46:26.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.182+0000 7ff491600640 1 -- 172.21.15.123:0/1387968442 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff48c00b040 con 0x7ff4940d03c0 2024-04-24T05:46:26.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.182+0000 7f74b7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f74b80d1500 0x7f74b80d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.182+0000 7f55c7e00640 1 -- 172.21.15.123:0/901733879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55c00075f0 msgr2=0x7f55c0038ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.182+0000 7f55c7e00640 1 --2- 172.21.15.123:0/901733879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55c00075f0 0x7f55c0038ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.182+0000 7f55c7e00640 1 -- 172.21.15.123:0/901733879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c00e4cd0 msgr2=0x7f55c0178a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.182+0000 7f55c7e00640 1 --2- 172.21.15.123:0/901733879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c00e4cd0 0x7f55c0178a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.182+0000 7f55c7e00640 1 -- 172.21.15.123:0/901733879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c00e54c0 con 0x7f55c00574f0 2024-04-24T05:46:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.183+0000 7f74b7e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f74b80d1500 0x7f74b80d1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56754/0 (socket says 172.21.15.123:56754) 2024-04-24T05:46:26.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.183+0000 7f74b7400640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f74b815ebf0 0x7f74b806ed20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.123:46352/0 (socket says 172.21.15.123:46352) 2024-04-24T05:46:26.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.183+0000 7f74b7e00640 1 -- 172.21.15.123:0/761441801 learned_addr learned my addr 172.21.15.123:0/761441801 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:26.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.183+0000 7f74be000640 1 -- 172.21.15.123:0/761441801 --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74b80d3af0 con 0x7f74b815ebf0 2024-04-24T05:46:26.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.183+0000 7f74be000640 1 -- 172.21.15.123:0/761441801 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f74b80b6030 con 0x7f74b80d1500 2024-04-24T05:46:26.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.183+0000 7f74be000640 1 -- 172.21.15.123:0/761441801 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f74b80b6310 con 0x7f74b80d35f0 2024-04-24T05:46:26.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.183+0000 7f74b6000640 1 -- 172.21.15.123:0/761441801 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3321496428 0 0) 0x7f74b80d3af0 con 0x7f74b815ebf0 2024-04-24T05:46:26.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.183+0000 7f74b6000640 1 -- 172.21.15.123:0/761441801 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f749c003520 con 0x7f74b815ebf0 2024-04-24T05:46:26.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.184+0000 7f74b7e00640 1 -- 172.21.15.123:0/761441801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b80d35f0 msgr2=0x7f74b80b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.184+0000 7f74b6000640 1 -- 172.21.15.123:0/761441801 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3699903452 0 0) 0x7f749c003520 con 0x7f74b815ebf0 2024-04-24T05:46:26.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.183+0000 7ff491600640 1 --2- 172.21.15.123:0/1387968442 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.183+0000 7ff491600640 1 -- 172.21.15.123:0/1387968442 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff48c0ac4b0 con 0x7ff4940d03c0 2024-04-24T05:46:26.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.184+0000 7ff499a00640 1 --2- 172.21.15.123:0/1387968442 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.186+0000 7ff499a00640 1 --2- 172.21.15.123:0/1387968442 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff47406c8f0 0x7ff47406edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff488000c00 tx=0x7ff488009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.184+0000 7f55c7e00640 1 --2- 172.21.15.123:0/901733879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55c00574f0 0x7f55c00e4790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f55c80512e0 tx=0x7f55c8069500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.184+0000 7eff3a800640 1 -- 172.21.15.123:0/1998362936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff34160230 con 0x7eff340d35f0 2024-04-24T05:46:26.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.184+0000 7eff3a800640 1 -- 172.21.15.123:0/1998362936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff341607f0 con 0x7eff340d35f0 2024-04-24T05:46:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.182+0000 7f8851c00640 1 -- start start 2024-04-24T05:46:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.184+0000 7f8851c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f884c15e680 0x7f884c06cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.184+0000 7f8851c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f884c0b76a0 0x7f884c0b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.184+0000 7f8851c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f884c0cf060 con 0x7f884c0ceac0 2024-04-24T05:46:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.184+0000 7f8851c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f884c0b8000 con 0x7f884c0b76a0 2024-04-24T05:46:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.184+0000 7f8851c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f884c0cf340 con 0x7f884c15e680 2024-04-24T05:46:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.184+0000 7f884be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f884c15e680 0x7f884c06cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.185+0000 7f884be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f884c15e680 0x7f884c06cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:50174/0 (socket says 172.21.15.123:50174) 2024-04-24T05:46:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.185+0000 7f884be00640 1 -- 172.21.15.123:0/1171236915 learned_addr learned my addr 172.21.15.123:0/1171236915 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.185+0000 7f8851200640 1 --2- 172.21.15.123:0/1171236915 >> v2:172.21.15.123:3300/0 conn(0x7f884c0b76a0 0x7f884c0b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.185+0000 7f8851200640 1 -- 172.21.15.123:0/1171236915 >> v2:172.21.15.155:3300/0 conn(0x7f884c15e680 msgr2=0x7f884c06cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.186+0000 7f8851200640 1 --2- 172.21.15.123:0/1171236915 >> v2:172.21.15.155:3300/0 conn(0x7f884c15e680 0x7f884c06cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.186+0000 7f8851200640 1 -- 172.21.15.123:0/1171236915 >> v1:172.21.15.155:6789/0 conn(0x7f884c0ceac0 legacy=0x7f884c0d0100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T05:46:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.186+0000 7f8851200640 1 -- 172.21.15.123:0/1171236915 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884c0cf620 con 0x7f884c0b76a0 2024-04-24T05:46:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.186+0000 7f74b6a00640 1 --2- 172.21.15.123:0/761441801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b80d35f0 0x7f74b80b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.186+0000 7f74b7e00640 1 --2- 172.21.15.123:0/761441801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b80d35f0 0x7f74b80b67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.186+0000 7f74b7e00640 1 -- 172.21.15.123:0/761441801 >> v1:172.21.15.155:6789/0 conn(0x7f74b815ebf0 legacy=0x7f74b806ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.186+0000 7f74b7e00640 1 -- 172.21.15.123:0/761441801 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b80d3dd0 con 0x7f74b80d1500 2024-04-24T05:46:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.186+0000 7eff28c00640 1 -- 172.21.15.123:0/1998362936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff3408ca80 con 0x7eff340d35f0 2024-04-24T05:46:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.187+0000 7f8851200640 1 --2- 172.21.15.123:0/1171236915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c0b76a0 0x7f884c0b7ac0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f884c08c980 tx=0x7f883802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ed9f8825562036 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.188+0000 7f74b7e00640 1 --2- 172.21.15.123:0/761441801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b80d1500 0x7f74b80d1990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009870 tx=0x7f74a002a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abfee7d12869ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.188+0000 7eff2b400640 1 -- 172.21.15.123:0/1998362936 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff24044070 con 0x7eff340d35f0 2024-04-24T05:46:26.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.189+0000 7eff2b400640 1 --2- 172.21.15.123:0/1998362936 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7eff1406c9c0 0x7eff1406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.189+0000 7eff2b400640 1 -- 172.21.15.123:0/1998362936 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7eff240ac900 con 0x7eff340d35f0 2024-04-24T05:46:26.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.189+0000 7eff33400640 1 --2- 172.21.15.123:0/1998362936 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7eff1406c9c0 0x7eff1406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.190+0000 7eff33400640 1 --2- 172.21.15.123:0/1998362936 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7eff1406c9c0 0x7eff1406ee80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff20000f60 tx=0x7eff200079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.193+0000 7f74b6000640 1 -- 172.21.15.123:0/761441801 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0037070 con 0x7f74b80d1500 2024-04-24T05:46:26.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.193+0000 7f55c4c00640 1 -- 172.21.15.123:0/901733879 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c80748a0 con 0x7f55c00574f0 2024-04-24T05:46:26.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.193+0000 7f55c4c00640 1 -- 172.21.15.123:0/901733879 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c8074e70 con 0x7f55c00574f0 2024-04-24T05:46:26.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.193+0000 7f55c4c00640 1 -- 172.21.15.123:0/901733879 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c80752a0 con 0x7f55c00574f0 2024-04-24T05:46:26.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.193+0000 7f74b6000640 1 -- 172.21.15.123:0/761441801 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a002ac80 con 0x7f74b80d1500 2024-04-24T05:46:26.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.193+0000 7f74b6000640 1 -- 172.21.15.123:0/761441801 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b00019f0 con 0x7f74b80d1500 2024-04-24T05:46:26.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.193+0000 7f884aa00640 1 -- 172.21.15.123:0/1171236915 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838037070 con 0x7f884c0b76a0 2024-04-24T05:46:26.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.193+0000 7f884aa00640 1 -- 172.21.15.123:0/1171236915 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8838032440 con 0x7f884c0b76a0 2024-04-24T05:46:26.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.193+0000 7f884aa00640 1 -- 172.21.15.123:0/1171236915 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838032810 con 0x7f884c0b76a0 2024-04-24T05:46:26.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.193+0000 7ff491600640 1 -- 172.21.15.123:0/1387968442 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff48c07a470 con 0x7ff4940d03c0 2024-04-24T05:46:26.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.194+0000 7f74be000640 1 -- 172.21.15.123:0/761441801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b80d1500 msgr2=0x7f74b80d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.194+0000 7f74be000640 1 --2- 172.21.15.123:0/761441801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b80d1500 0x7f74b80d1990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009870 tx=0x7f74a002a1f0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.194+0000 7f55ce400640 1 -- 172.21.15.123:0/901733879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c00e57a0 con 0x7f55c00574f0 2024-04-24T05:46:26.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.194+0000 7f55ce400640 1 -- 172.21.15.123:0/901733879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c00e5d60 con 0x7f55c00574f0 2024-04-24T05:46:26.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.194+0000 7f74be000640 1 -- 172.21.15.123:0/761441801 shutdown_connections 2024-04-24T05:46:26.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.194+0000 7f74be000640 1 --2- 172.21.15.123:0/761441801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b80d1500 0x7f74b80d1990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.194+0000 7f74be000640 1 --2- 172.21.15.123:0/761441801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b80d35f0 0x7f74b80b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.194+0000 7f74be000640 1 -- 172.21.15.123:0/761441801 >> 172.21.15.123:0/761441801 conn(0x7f74b80b9570 msgr2=0x7f74b80b9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.194+0000 7f8851c00640 1 -- 172.21.15.123:0/1171236915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c0b76a0 msgr2=0x7f884c0b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.194+0000 7f8851c00640 1 --2- 172.21.15.123:0/1171236915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c0b76a0 0x7f884c0b7ac0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f884c08c980 tx=0x7f883802a160 comp rx=0 tx=0).stop 2024-04-24T05:46:26.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.195+0000 7f74be000640 1 -- 172.21.15.123:0/761441801 shutdown_connections 2024-04-24T05:46:26.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.195+0000 7f8851c00640 1 -- 172.21.15.123:0/1171236915 shutdown_connections 2024-04-24T05:46:26.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.195+0000 7f8851c00640 1 --2- 172.21.15.123:0/1171236915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c0b76a0 0x7f884c0b7ac0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.195+0000 7f8851c00640 1 --2- 172.21.15.123:0/1171236915 >> v2:172.21.15.155:3300/0 conn(0x7f884c15e680 0x7f884c06cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.195+0000 7f8851c00640 1 -- 172.21.15.123:0/1171236915 >> 172.21.15.123:0/1171236915 conn(0x7f884c0b5f60 msgr2=0x7f884c0b6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.195+0000 7f8851c00640 1 -- 172.21.15.123:0/1171236915 shutdown_connections 2024-04-24T05:46:26.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.195+0000 7f8851c00640 1 -- 172.21.15.123:0/1171236915 wait complete. 2024-04-24T05:46:26.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.196+0000 7f55ce400640 1 -- 172.21.15.123:0/901733879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c00057e0 con 0x7f55c00574f0 2024-04-24T05:46:26.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.195+0000 7f74be000640 1 -- 172.21.15.123:0/761441801 wait complete. 2024-04-24T05:46:26.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.197+0000 7f74be000640 1 Processor -- start 2024-04-24T05:46:26.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.198+0000 7f55c4c00640 1 -- 172.21.15.123:0/901733879 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55c8093400 con 0x7f55c00574f0 2024-04-24T05:46:26.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.198+0000 7f8851c00640 1 Processor -- start 2024-04-24T05:46:26.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.198+0000 7f8851c00640 1 -- start start 2024-04-24T05:46:26.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.198+0000 7f74be000640 1 -- start start 2024-04-24T05:46:26.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.199+0000 7f74be000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b80d1500 0x7f74b8161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.199+0000 7f74be000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b80d35f0 0x7f74b80c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.199+0000 7f74be000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74b815ebf0 0x7f74b80ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.199+0000 7f74be000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74b80b6e60 con 0x7f74b80d35f0 2024-04-24T05:46:26.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.199+0000 7f74be000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f74b80b6ce0 con 0x7f74b815ebf0 2024-04-24T05:46:26.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.199+0000 7f74be000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f74b81625e0 con 0x7f74b80d1500 2024-04-24T05:46:26.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.199+0000 7f55c4c00640 1 --2- 172.21.15.123:0/901733879 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f55a806c940 0x7f55a806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.200+0000 7f55c4c00640 1 -- 172.21.15.123:0/901733879 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f55c80d29e0 con 0x7f55c00574f0 2024-04-24T05:46:26.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.200+0000 7f8851c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c0b76a0 0x7f884c1579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.200+0000 7f55cd000640 1 --2- 172.21.15.123:0/901733879 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f55a806c940 0x7f55a806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.200+0000 7f74b7400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b80d1500 0x7f74b8161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.200+0000 7f74b7400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b80d1500 0x7f74b8161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38798/0 (socket says 172.21.15.123:38798) 2024-04-24T05:46:26.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.200+0000 7f8851c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f884c0ceac0 0x7f884c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.200+0000 7f8851c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f884c15e680 0x7f884c0b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.200+0000 7f74b7400640 1 -- 172.21.15.123:0/1860066016 learned_addr learned my addr 172.21.15.123:0/1860066016 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:26.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.201+0000 7f8851c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f884c15ea60 con 0x7f884c0b76a0 2024-04-24T05:46:26.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.201+0000 7f8851c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f884c0cfbe0 con 0x7f884c0ceac0 2024-04-24T05:46:26.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.201+0000 7f8851c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f884c06d4c0 con 0x7f884c15e680 2024-04-24T05:46:26.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.201+0000 7f74b7400640 1 -- 172.21.15.123:0/1860066016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74b815ebf0 msgr2=0x7f74b80ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.201+0000 7f74b7400640 1 --2- 172.21.15.123:0/1860066016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74b815ebf0 0x7f74b80ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.201+0000 7f74b7400640 1 -- 172.21.15.123:0/1860066016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b80d35f0 msgr2=0x7f74b80c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.201+0000 7f74b7400640 1 --2- 172.21.15.123:0/1860066016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b80d35f0 0x7f74b80c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.201+0000 7f74b7400640 1 -- 172.21.15.123:0/1860066016 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b80cc570 con 0x7f74b80d1500 2024-04-24T05:46:26.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.201+0000 7f884be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c0b76a0 0x7f884c1579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.201+0000 7f74b7400640 1 --2- 172.21.15.123:0/1860066016 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b80d1500 0x7f74b8161040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f74b000caf0 tx=0x7f74b0002170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.201+0000 7f884be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c0b76a0 0x7f884c1579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56778/0 (socket says 172.21.15.123:56778) 2024-04-24T05:46:26.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.201+0000 7f884be00640 1 -- 172.21.15.123:0/30238947 learned_addr learned my addr 172.21.15.123:0/30238947 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:26.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.202+0000 7f74af400640 1 -- 172.21.15.123:0/1860066016 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0017070 con 0x7f74b80d1500 2024-04-24T05:46:26.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.202+0000 7f884be00640 1 -- 172.21.15.123:0/30238947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f884c15e680 msgr2=0x7f884c0b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.202+0000 7f55cd000640 1 --2- 172.21.15.123:0/901733879 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f55a806c940 0x7f55a806ee00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f55bc034000 tx=0x7f55bc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.202+0000 7f74af400640 1 -- 172.21.15.123:0/1860066016 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b000a680 con 0x7f74b80d1500 2024-04-24T05:46:26.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.202+0000 7f74af400640 1 -- 172.21.15.123:0/1860066016 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b000aab0 con 0x7f74b80d1500 2024-04-24T05:46:26.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.202+0000 7f884be00640 1 --2- 172.21.15.123:0/30238947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f884c15e680 0x7f884c0b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.202+0000 7f884be00640 1 -- 172.21.15.123:0/30238947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f884c0ceac0 msgr2=0x7f884c0b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.202+0000 7f884be00640 1 --2- 172.21.15.123:0/30238947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f884c0ceac0 0x7f884c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.202+0000 7f74be000640 1 -- 172.21.15.123:0/1860066016 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b80cc850 con 0x7f74b80d1500 2024-04-24T05:46:26.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.202+0000 7f884be00640 1 -- 172.21.15.123:0/30238947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884c0cb540 con 0x7f884c0b76a0 2024-04-24T05:46:26.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.203+0000 7f884be00640 1 --2- 172.21.15.123:0/30238947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c0b76a0 0x7f884c1579b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f884400efc0 tx=0x7f884400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.203+0000 7f74be000640 1 -- 172.21.15.123:0/1860066016 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b80cce10 con 0x7f74b80d1500 2024-04-24T05:46:26.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.204+0000 7eff2b400640 1 -- 172.21.15.123:0/1998362936 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7eff3408ca80 con 0x7eff340d35f0 2024-04-24T05:46:26.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.204+0000 7f74be000640 1 -- 172.21.15.123:0/1860066016 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b808ca80 con 0x7f74b80d1500 2024-04-24T05:46:26.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.207+0000 7f74af400640 1 -- 172.21.15.123:0/1860066016 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74b001a070 con 0x7f74b80d1500 2024-04-24T05:46:26.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.208+0000 7f74af400640 1 --2- 172.21.15.123:0/1860066016 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f749406c8f0 0x7f749406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.209+0000 7f74af400640 1 -- 172.21.15.123:0/1860066016 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f74b008b710 con 0x7f74b80d1500 2024-04-24T05:46:26.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.209+0000 7f74b6a00640 1 --2- 172.21.15.123:0/1860066016 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f749406c8f0 0x7f749406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.211+0000 7f74b6a00640 1 --2- 172.21.15.123:0/1860066016 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f749406c8f0 0x7f749406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f74a8006fd0 tx=0x7f74a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.211+0000 7f55c4c00640 1 -- 172.21.15.123:0/901733879 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55c808f030 con 0x7f55c00574f0 2024-04-24T05:46:26.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.212+0000 7f8843e00640 1 -- 172.21.15.123:0/30238947 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884400cda0 con 0x7f884c0b76a0 2024-04-24T05:46:26.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.212+0000 7f8843e00640 1 -- 172.21.15.123:0/30238947 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8844019070 con 0x7f884c0b76a0 2024-04-24T05:46:26.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.212+0000 7f8843e00640 1 -- 172.21.15.123:0/30238947 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88440146a0 con 0x7f884c0b76a0 2024-04-24T05:46:26.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.212+0000 7f8851c00640 1 -- 172.21.15.123:0/30238947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884c0cb820 con 0x7f884c0b76a0 2024-04-24T05:46:26.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.212+0000 7f8851c00640 1 -- 172.21.15.123:0/30238947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884c157ef0 con 0x7f884c0b76a0 2024-04-24T05:46:26.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.213+0000 7f8851c00640 1 -- 172.21.15.123:0/30238947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f884c075b70 con 0x7f884c0b76a0 2024-04-24T05:46:26.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.216+0000 7f8843e00640 1 -- 172.21.15.123:0/30238947 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8844005d10 con 0x7f884c0b76a0 2024-04-24T05:46:26.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.217+0000 7f8843e00640 1 --2- 172.21.15.123:0/30238947 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f882c06c940 0x7f882c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.217+0000 7f8843e00640 1 -- 172.21.15.123:0/30238947 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8844093410 con 0x7f884c0b76a0 2024-04-24T05:46:26.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.218+0000 7f884b400640 1 --2- 172.21.15.123:0/30238947 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f882c06c940 0x7f882c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.220+0000 7f884b400640 1 --2- 172.21.15.123:0/30238947 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f882c06c940 0x7f882c06ee00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f884c073990 tx=0x7f883c006c40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.221+0000 7f74af400640 1 -- 172.21.15.123:0/1860066016 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f74b808ca80 con 0x7f74b80d1500 2024-04-24T05:46:26.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.226+0000 7f8843e00640 1 -- 172.21.15.123:0/30238947 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8844010040 con 0x7f884c0b76a0 2024-04-24T05:46:26.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.229+0000 7f1bf6000640 1 Processor -- start 2024-04-24T05:46:26.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.230+0000 7f1bf6000640 1 -- start start 2024-04-24T05:46:26.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.232+0000 7f1bf6000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1bf015e4f0 0x7f1bf00cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.232+0000 7f1bf6000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf00d3440 0x7f1bf00d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.232+0000 7f1bf6000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bf00cfe30 con 0x7f1bf00d4390 2024-04-24T05:46:26.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.232+0000 7f1bf6000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1bf00d0cd0 con 0x7f1bf015e4f0 2024-04-24T05:46:26.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.232+0000 7f1bf6000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1bf00d1fd0 con 0x7f1bf00d3440 2024-04-24T05:46:26.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.233+0000 7ff4bee00640 1 Processor -- start 2024-04-24T05:46:26.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.233+0000 7ff4bee00640 1 -- start start 2024-04-24T05:46:26.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.233+0000 7f1befe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf00d3440 0x7f1bf00d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.233+0000 7f1befe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf00d3440 0x7f1bf00d3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38818/0 (socket says 172.21.15.123:38818) 2024-04-24T05:46:26.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.233+0000 7f1befe00640 1 -- 172.21.15.123:0/2346287088 learned_addr learned my addr 172.21.15.123:0/2346287088 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:26.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.235+0000 7ff4bee00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff4b806be90 0x7ff4b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.235+0000 7ff4bee00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4b806edd0 con 0x7ff4b8071e40 2024-04-24T05:46:26.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.235+0000 7ff4bee00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4b806c3e0 con 0x7ff4b806ba00 2024-04-24T05:46:26.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.235+0000 7ff4bee00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff4b806c6c0 con 0x7ff4b806be90 2024-04-24T05:46:26.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.234+0000 7f1befe00640 1 -- 172.21.15.123:0/2346287088 >> v2:172.21.15.155:3300/0 conn(0x7f1bf015e4f0 msgr2=0x7f1bf00cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.234+0000 7f1befe00640 1 --2- 172.21.15.123:0/2346287088 >> v2:172.21.15.155:3300/0 conn(0x7f1bf015e4f0 0x7f1bf00cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.236+0000 7f1befe00640 1 -- 172.21.15.123:0/2346287088 >> v1:172.21.15.123:6789/0 conn(0x7f1bf00d4390 legacy=0x7f1bf00d47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.236+0000 7f1befe00640 1 -- 172.21.15.123:0/2346287088 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf00d2290 con 0x7f1bf00d3440 2024-04-24T05:46:26.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.236+0000 7ff4b7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff4b8071e40 0x7ff4b8072220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:35640/0 (socket says 172.21.15.123:35640) 2024-04-24T05:46:26.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.236+0000 7ff4b7e00640 1 -- 172.21.15.123:0/4008343897 learned_addr learned my addr 172.21.15.123:0/4008343897 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:26.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.237+0000 7ff4bce00640 1 --2- 172.21.15.123:0/4008343897 >> v2:172.21.15.123:3300/0 conn(0x7ff4b806be90 0x7ff4b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.237+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3670823250 0 0) 0x7ff4b806edd0 con 0x7ff4b8071e40 2024-04-24T05:46:26.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff49c003520 con 0x7ff4b8071e40 2024-04-24T05:46:26.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7f1befe00640 1 --2- 172.21.15.123:0/2346287088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf00d3440 0x7f1bf00d3860 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009870 tx=0x7f1bdc02a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81e075776729f6e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 516398185 0 0) 0x7ff4b806c3e0 con 0x7ff4b806ba00 2024-04-24T05:46:26.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4b806edd0 con 0x7ff4b806ba00 2024-04-24T05:46:26.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4183579702 0 0) 0x7ff49c003520 con 0x7ff4b8071e40 2024-04-24T05:46:26.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 >> v2:172.21.15.123:3300/0 conn(0x7ff4b806be90 msgr2=0x7ff4b806e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 --2- 172.21.15.123:0/4008343897 >> v2:172.21.15.123:3300/0 conn(0x7ff4b806be90 0x7ff4b806e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 >> v1:172.21.15.155:6789/0 conn(0x7ff4b806ba00 legacy=0x7ff4b806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b806c9a0 con 0x7ff4b8071e40 2024-04-24T05:46:26.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7ff4a8001670 con 0x7ff4b8071e40 2024-04-24T05:46:26.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7f1bef400640 1 -- 172.21.15.123:0/2346287088 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc037070 con 0x7f1bf00d3440 2024-04-24T05:46:26.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 >> v1:172.21.15.123:6789/0 conn(0x7ff4b8071e40 legacy=0x7ff4b8072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 --2- 172.21.15.123:0/4008343897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff49c003b40 0x7ff49c003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 --2- 172.21.15.123:0/4008343897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff49c004800 0x7ff49c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 --2- 172.21.15.123:0/4008343897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff49c0051c0 0x7ff49c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b806c6c0 con 0x7ff49c004800 2024-04-24T05:46:26.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c00c0c0 con 0x7ff49c0051c0 2024-04-24T05:46:26.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.238+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff49c00c2c0 con 0x7ff49c003b40 2024-04-24T05:46:26.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.239+0000 7ff4bce00640 1 --2- 172.21.15.123:0/4008343897 >> v2:172.21.15.123:3300/0 conn(0x7ff4b806be90 0x7ff4b806e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:46:26.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.239+0000 7f1bef400640 1 -- 172.21.15.123:0/2346287088 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bdc02adf0 con 0x7f1bf00d3440 2024-04-24T05:46:26.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.239+0000 7f1bef400640 1 -- 172.21.15.123:0/2346287088 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc037070 con 0x7f1bf00d3440 2024-04-24T05:46:26.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.239+0000 7ff4b7e00640 1 --2- 172.21.15.123:0/4008343897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff49c003b40 0x7ff49c003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.239+0000 7ff4bce00640 1 --2- 172.21.15.123:0/4008343897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff49c0051c0 0x7ff49c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.240+0000 7ff4b7400640 1 --2- 172.21.15.123:0/4008343897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff49c004800 0x7ff49c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.240+0000 7f1bf6000640 1 -- 172.21.15.123:0/2346287088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf00d3440 msgr2=0x7f1bf00d3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.240+0000 7f1bf6000640 1 --2- 172.21.15.123:0/2346287088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf00d3440 0x7f1bf00d3860 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009870 tx=0x7f1bdc02a160 comp rx=0 tx=0).stop 2024-04-24T05:46:26.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.241+0000 7f1bf6000640 1 -- 172.21.15.123:0/2346287088 shutdown_connections 2024-04-24T05:46:26.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.241+0000 7f1bf6000640 1 --2- 172.21.15.123:0/2346287088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf00d3440 0x7f1bf00d3860 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.241+0000 7f1bf6000640 1 --2- 172.21.15.123:0/2346287088 >> v2:172.21.15.155:3300/0 conn(0x7f1bf015e4f0 0x7f1bf00cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.241+0000 7f1bf6000640 1 -- 172.21.15.123:0/2346287088 >> 172.21.15.123:0/2346287088 conn(0x7f1bf009d780 msgr2=0x7f1bf00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.241+0000 7f1bf6000640 1 -- 172.21.15.123:0/2346287088 shutdown_connections 2024-04-24T05:46:26.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.241+0000 7f1bf6000640 1 -- 172.21.15.123:0/2346287088 wait complete. 2024-04-24T05:46:26.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.241+0000 7ff4bce00640 1 -- 172.21.15.123:0/4008343897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff49c003b40 msgr2=0x7ff49c003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.241+0000 7ff4bce00640 1 --2- 172.21.15.123:0/4008343897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff49c003b40 0x7ff49c003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.241+0000 7ff4bce00640 1 -- 172.21.15.123:0/4008343897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff49c004800 msgr2=0x7ff49c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.241+0000 7ff4bce00640 1 --2- 172.21.15.123:0/4008343897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff49c004800 0x7ff49c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.241+0000 7ff4bce00640 1 -- 172.21.15.123:0/4008343897 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c003520 con 0x7ff49c0051c0 2024-04-24T05:46:26.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.241+0000 7ff4b7400640 1 --2- 172.21.15.123:0/4008343897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff49c004800 0x7ff49c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:46:26.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.243+0000 7ff4bce00640 1 -- 172.21.15.123:0/4008343897 reap_dead start 2024-04-24T05:46:26.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.244+0000 7ff4bce00640 1 --2- 172.21.15.123:0/4008343897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff49c0051c0 0x7ff49c00b9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000e350 tx=0x7ff49c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cec2cbd972c435f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.245+0000 7f1bf6000640 1 Processor -- start 2024-04-24T05:46:26.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.245+0000 7f1bf6000640 1 -- start start 2024-04-24T05:46:26.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.244+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40014c0 con 0x7ff49c0051c0 2024-04-24T05:46:26.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.245+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b0003bd0 con 0x7ff49c0051c0 2024-04-24T05:46:26.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.245+0000 7ff4b6a00640 1 -- 172.21.15.123:0/4008343897 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0009de0 con 0x7ff49c0051c0 2024-04-24T05:46:26.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.247+0000 7f1bf6000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf00d3440 0x7f1bf006dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.247+0000 7f1bf6000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bf00d4390 0x7f1bf006dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.247+0000 7f1bf6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf015e4f0 0x7f1bf006c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.247+0000 7f1bf6000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf00d2b30 con 0x7f1bf015e4f0 2024-04-24T05:46:26.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.247+0000 7f1bf6000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf00d2850 con 0x7f1bf00d4390 2024-04-24T05:46:26.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.247+0000 7f1bf6000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1bf015e8d0 con 0x7f1bf00d3440 2024-04-24T05:46:26.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.248+0000 7f1bf4c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf00d3440 0x7f1bf006dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.248+0000 7f1befe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bf00d4390 0x7f1bf006dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.248+0000 7f1bf4c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf00d3440 0x7f1bf006dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38822/0 (socket says 172.21.15.123:38822) 2024-04-24T05:46:26.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.248+0000 7f1bf4c00640 1 -- 172.21.15.123:0/2293222606 learned_addr learned my addr 172.21.15.123:0/2293222606 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:26.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.249+0000 7f1bf5600640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf015e4f0 0x7f1bf006c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.249+0000 7f1befe00640 1 -- 172.21.15.123:0/2293222606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf00d3440 msgr2=0x7f1bf006dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.249+0000 7f1befe00640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf00d3440 0x7f1bf006dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.249+0000 7f1befe00640 1 -- 172.21.15.123:0/2293222606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf015e4f0 msgr2=0x7f1bf006c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.249+0000 7f1befe00640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf015e4f0 0x7f1bf006c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.249+0000 7f1befe00640 1 -- 172.21.15.123:0/2293222606 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf00caad0 con 0x7f1bf00d4390 2024-04-24T05:46:26.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.249+0000 7ff4bee00640 1 -- 172.21.15.123:0/4008343897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff49c0051c0 msgr2=0x7ff49c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.249+0000 7ff4bee00640 1 --2- 172.21.15.123:0/4008343897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff49c0051c0 0x7ff49c00b9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000e350 tx=0x7ff49c005830 comp rx=0 tx=0).stop 2024-04-24T05:46:26.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.250+0000 7f1bf4c00640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf00d3440 0x7f1bf006dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:46:26.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.250+0000 7ff4bee00640 1 -- 172.21.15.123:0/4008343897 shutdown_connections 2024-04-24T05:46:26.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.250+0000 7ff4bee00640 1 --2- 172.21.15.123:0/4008343897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff49c0051c0 0x7ff49c00b9c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.250+0000 7ff4bee00640 1 -- 172.21.15.123:0/4008343897 >> 172.21.15.123:0/4008343897 conn(0x7ff4b80b1e70 msgr2=0x7ff4b80b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.250+0000 7f1bf5600640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf015e4f0 0x7f1bf006c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:46:26.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.250+0000 7ff4bee00640 1 -- 172.21.15.123:0/4008343897 shutdown_connections 2024-04-24T05:46:26.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.250+0000 7ff4bee00640 1 -- 172.21.15.123:0/4008343897 wait complete. 2024-04-24T05:46:26.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.250+0000 7f1befe00640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bf00d4390 0x7f1bf006dff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc031000 tx=0x7f1bdc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.251+0000 7f1becc00640 1 -- 172.21.15.123:0/2293222606 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc041020 con 0x7f1bf00d4390 2024-04-24T05:46:26.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.251+0000 7f1becc00640 1 -- 172.21.15.123:0/2293222606 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bdc032440 con 0x7f1bf00d4390 2024-04-24T05:46:26.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.251+0000 7f1becc00640 1 -- 172.21.15.123:0/2293222606 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc037040 con 0x7f1bf00d4390 2024-04-24T05:46:26.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.253+0000 7ff4bee00640 1 Processor -- start 2024-04-24T05:46:26.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.254+0000 7ff4bee00640 1 -- start start 2024-04-24T05:46:26.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.253+0000 7f1bf6000640 1 -- 172.21.15.123:0/2293222606 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf00c80d0 con 0x7f1bf00d4390 2024-04-24T05:46:26.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.253+0000 7f1bf6000640 1 -- 172.21.15.123:0/2293222606 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf00c8690 con 0x7f1bf00d4390 2024-04-24T05:46:26.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.256+0000 7ff4bee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff49c0051c0 0x7ff4b8153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.256+0000 7ff4bee00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b80cd9f0 0x7ff4b80cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.257+0000 7ff4bee00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80b0680 0x7ff4b80b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.257+0000 7ff4bee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81623f0 con 0x7ff49c0051c0 2024-04-24T05:46:26.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.257+0000 7ff4b7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff49c0051c0 0x7ff4b8153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.258+0000 7ff4bce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80b0680 0x7ff4b80b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.258+0000 7ff4bce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80b0680 0x7ff4b80b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:38824/0 (socket says 172.21.15.123:38824) 2024-04-24T05:46:26.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.258+0000 7ff4bce00640 1 -- 172.21.15.123:0/3160973148 learned_addr learned my addr 172.21.15.123:0/3160973148 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:26.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.258+0000 7ff4bee00640 1 -- 172.21.15.123:0/3160973148 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b806cf60 con 0x7ff4b80cd9f0 2024-04-24T05:46:26.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.258+0000 7ff4bee00640 1 -- 172.21.15.123:0/3160973148 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b8162570 con 0x7ff4b80b0680 2024-04-24T05:46:26.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.258+0000 7ff4b7400640 1 --2- 172.21.15.123:0/3160973148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b80cd9f0 0x7ff4b80cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.258+0000 7f1becc00640 1 -- 172.21.15.123:0/2293222606 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1bdc044020 con 0x7f1bf00d4390 2024-04-24T05:46:26.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.258+0000 7f1bf6000640 1 -- 172.21.15.123:0/2293222606 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf0075b70 con 0x7f1bf00d4390 2024-04-24T05:46:26.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.259+0000 7f1becc00640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1bd006c9c0 0x7f1bd006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.259+0000 7f1becc00640 1 -- 172.21.15.123:0/2293222606 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1bdc0ac310 con 0x7f1bf00d4390 2024-04-24T05:46:26.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.259+0000 7ff4b7e00640 1 -- 172.21.15.123:0/3160973148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80b0680 msgr2=0x7ff4b80b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.259+0000 7ff4b7e00640 1 --2- 172.21.15.123:0/3160973148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80b0680 0x7ff4b80b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.259+0000 7ff4b7e00640 1 -- 172.21.15.123:0/3160973148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b80cd9f0 msgr2=0x7ff4b80cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.259+0000 7ff4b7e00640 1 --2- 172.21.15.123:0/3160973148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b80cd9f0 0x7ff4b80cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.259+0000 7ff4b7e00640 1 -- 172.21.15.123:0/3160973148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b80ce7e0 con 0x7ff49c0051c0 2024-04-24T05:46:26.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.260+0000 7ff4b7e00640 1 --2- 172.21.15.123:0/3160973148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff49c0051c0 0x7ff4b8153e00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8004e20 tx=0x7ff4a8007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.261+0000 7ff4b7400640 1 --2- 172.21.15.123:0/3160973148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b80cd9f0 0x7ff4b80cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:26.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.261+0000 7ff4afe00640 1 -- 172.21.15.123:0/3160973148 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800f750 con 0x7ff49c0051c0 2024-04-24T05:46:26.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.261+0000 7ff4afe00640 1 -- 172.21.15.123:0/3160973148 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a8018070 con 0x7ff49c0051c0 2024-04-24T05:46:26.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.262+0000 7ff4afe00640 1 -- 172.21.15.123:0/3160973148 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800fb30 con 0x7ff49c0051c0 2024-04-24T05:46:26.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.262+0000 7ff4bee00640 1 -- 172.21.15.123:0/3160973148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b80ceac0 con 0x7ff49c0051c0 2024-04-24T05:46:26.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.262+0000 7ff4bee00640 1 -- 172.21.15.123:0/3160973148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b80cf080 con 0x7ff49c0051c0 2024-04-24T05:46:26.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.261+0000 7f1bf4c00640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1bd006c9c0 0x7f1bd006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.266 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.264+0000 7f1bf4c00640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1bd006c9c0 0x7f1bd006ee80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00ca290 tx=0x7f1be4006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.264+0000 7ff4bee00640 1 -- 172.21.15.123:0/3160973148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b8068a20 con 0x7ff49c0051c0 2024-04-24T05:46:26.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.266+0000 7ff4afe00640 1 -- 172.21.15.123:0/3160973148 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4a800fcd0 con 0x7ff49c0051c0 2024-04-24T05:46:26.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.267+0000 7ff4afe00640 1 --2- 172.21.15.123:0/3160973148 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff49806c970 0x7ff49806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.268+0000 7ff4afe00640 1 -- 172.21.15.123:0/3160973148 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff4a808b920 con 0x7ff49c0051c0 2024-04-24T05:46:26.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.269+0000 7ff4b7400640 1 --2- 172.21.15.123:0/3160973148 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff49806c970 0x7ff49806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.270+0000 7ff4b7400640 1 --2- 172.21.15.123:0/3160973148 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff49806c970 0x7ff49806ee30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80d1200 tx=0x7ff4a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.274+0000 7f1becc00640 1 -- 172.21.15.123:0/2293222606 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1bdc04b030 con 0x7f1bf00d4390 2024-04-24T05:46:26.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.276+0000 7ff4afe00640 1 -- 172.21.15.123:0/3160973148 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4a8059960 con 0x7ff49c0051c0 2024-04-24T05:46:26.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.572+0000 7ff683200640 1 -- 172.21.15.123:0/22517481 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff65c005580 con 0x7ff67c0d4510 2024-04-24T05:46:26.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.573+0000 7ff67a000640 1 -- 172.21.15.123:0/22517481 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff67405d2c0 con 0x7ff67c0d4510 2024-04-24T05:46:26.578 INFO:teuthology.orchestra.run.smithi123.stdout:55834574859 2024-04-24T05:46:26.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.577+0000 7ff683200640 1 -- 172.21.15.123:0/22517481 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff66006c9f0 msgr2=0x7ff66006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.577+0000 7ff683200640 1 --2- 172.21.15.123:0/22517481 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff66006c9f0 0x7ff66006eeb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00a790 tx=0x7ff66c00a040 comp rx=0 tx=0).stop 2024-04-24T05:46:26.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.577+0000 7ff683200640 1 -- 172.21.15.123:0/22517481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff67c0d4510 msgr2=0x7ff67c04d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.577+0000 7ff683200640 1 --2- 172.21.15.123:0/22517481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff67c0d4510 0x7ff67c04d960 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00bd20 tx=0x7ff67400f530 comp rx=0 tx=0).stop 2024-04-24T05:46:26.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.577+0000 7ff683200640 1 -- 172.21.15.123:0/22517481 shutdown_connections 2024-04-24T05:46:26.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.577+0000 7ff683200640 1 --2- 172.21.15.123:0/22517481 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff66006c9f0 0x7ff66006eeb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.577+0000 7ff683200640 1 --2- 172.21.15.123:0/22517481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff67c0d4510 0x7ff67c04d960 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.577+0000 7ff683200640 1 --2- 172.21.15.123:0/22517481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff67c0d4090 0x7ff67c04d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.577+0000 7ff683200640 1 --2- 172.21.15.123:0/22517481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668004800 0x7ff67c1858e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.577+0000 7ff683200640 1 -- 172.21.15.123:0/22517481 >> 172.21.15.123:0/22517481 conn(0x7ff67c009fb0 msgr2=0x7ff67c033700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.578+0000 7ff683200640 1 -- 172.21.15.123:0/22517481 shutdown_connections 2024-04-24T05:46:26.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.578+0000 7ff683200640 1 -- 172.21.15.123:0/22517481 wait complete. 2024-04-24T05:46:26.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2024-04-24T05:46:26.607 DEBUG:teuthology.parallel:result is None 2024-04-24T05:46:26.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.609+0000 7eff28c00640 1 -- 172.21.15.123:0/1998362936 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7eff340d12e0 con 0x7eff340d35f0 2024-04-24T05:46:26.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.610+0000 7eff2b400640 1 -- 172.21.15.123:0/1998362936 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7eff2407ab80 con 0x7eff340d35f0 2024-04-24T05:46:26.611 INFO:teuthology.orchestra.run.smithi123.stdout:55834574859 2024-04-24T05:46:26.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.613+0000 7eff28c00640 1 -- 172.21.15.123:0/1998362936 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7eff1406c9c0 msgr2=0x7eff1406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.613+0000 7eff28c00640 1 --2- 172.21.15.123:0/1998362936 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7eff1406c9c0 0x7eff1406ee80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff20000f60 tx=0x7eff200079b0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.613+0000 7eff28c00640 1 -- 172.21.15.123:0/1998362936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff340d35f0 msgr2=0x7eff3415d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.613+0000 7eff28c00640 1 --2- 172.21.15.123:0/1998362936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff340d35f0 0x7eff3415d340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff24035040 tx=0x7eff240046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.614+0000 7eff28c00640 1 -- 172.21.15.123:0/1998362936 shutdown_connections 2024-04-24T05:46:26.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.614+0000 7eff28c00640 1 --2- 172.21.15.123:0/1998362936 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7eff1406c9c0 0x7eff1406ee80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.614+0000 7eff28c00640 1 --2- 172.21.15.123:0/1998362936 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff3415ebf0 0x7eff3406da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.614+0000 7eff28c00640 1 --2- 172.21.15.123:0/1998362936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff340d35f0 0x7eff3415d340 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.614+0000 7eff28c00640 1 --2- 172.21.15.123:0/1998362936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff340d1500 0x7eff34161cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.614+0000 7eff28c00640 1 -- 172.21.15.123:0/1998362936 >> 172.21.15.123:0/1998362936 conn(0x7eff340b9570 msgr2=0x7eff340d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.614+0000 7eff28c00640 1 -- 172.21.15.123:0/1998362936 shutdown_connections 2024-04-24T05:46:26.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.614+0000 7eff28c00640 1 -- 172.21.15.123:0/1998362936 wait complete. 2024-04-24T05:46:26.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.627+0000 7ff49c400640 1 -- 172.21.15.123:0/1387968442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff49406df20 con 0x7ff4940d03c0 2024-04-24T05:46:26.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.628+0000 7ff491600640 1 -- 172.21.15.123:0/1387968442 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff48c079d70 con 0x7ff4940d03c0 2024-04-24T05:46:26.631 INFO:teuthology.orchestra.run.smithi123.stdout:55834574858 2024-04-24T05:46:26.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.631+0000 7ff49c400640 1 -- 172.21.15.123:0/1387968442 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff47406c8f0 msgr2=0x7ff47406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.631+0000 7ff49c400640 1 --2- 172.21.15.123:0/1387968442 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff47406c8f0 0x7ff47406edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff488000c00 tx=0x7ff488009040 comp rx=0 tx=0).stop 2024-04-24T05:46:26.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.631+0000 7ff49c400640 1 -- 172.21.15.123:0/1387968442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4940d03c0 msgr2=0x7ff4940cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.631+0000 7ff49c400640 1 --2- 172.21.15.123:0/1387968442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4940d03c0 0x7ff4940cd460 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff48c035000 tx=0x7ff48c00d960 comp rx=0 tx=0).stop 2024-04-24T05:46:26.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.632+0000 7ff49c400640 1 -- 172.21.15.123:0/1387968442 shutdown_connections 2024-04-24T05:46:26.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.632+0000 7ff49c400640 1 --2- 172.21.15.123:0/1387968442 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.632+0000 7ff49c400640 1 --2- 172.21.15.123:0/1387968442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4940d03c0 0x7ff4940cd460 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.632+0000 7ff49c400640 1 --2- 172.21.15.123:0/1387968442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4940cff40 0x7ff4940ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.632+0000 7ff49c400640 1 --2- 172.21.15.123:0/1387968442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff47c004ef0 0x7ff494154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.632+0000 7ff49c400640 1 -- 172.21.15.123:0/1387968442 >> 172.21.15.123:0/1387968442 conn(0x7ff4940b4fd0 msgr2=0x7ff49406e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.632+0000 7ff49c400640 1 -- 172.21.15.123:0/1387968442 shutdown_connections 2024-04-24T05:46:26.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.632+0000 7ff49c400640 1 -- 172.21.15.123:0/1387968442 wait complete. 2024-04-24T05:46:26.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.641+0000 7f74be000640 1 -- 172.21.15.123:0/1860066016 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f74b80d12e0 con 0x7f74b80d1500 2024-04-24T05:46:26.643 INFO:teuthology.orchestra.run.smithi123.stdout:55834574859 2024-04-24T05:46:26.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.641+0000 7f74af400640 1 -- 172.21.15.123:0/1860066016 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f74b00597f0 con 0x7f74b80d1500 2024-04-24T05:46:26.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2024-04-24T05:46:26.644 DEBUG:teuthology.parallel:result is None 2024-04-24T05:46:26.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.644+0000 7f74be000640 1 -- 172.21.15.123:0/1860066016 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f749406c8f0 msgr2=0x7f749406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.644+0000 7f74be000640 1 --2- 172.21.15.123:0/1860066016 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f749406c8f0 0x7f749406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f74a8006fd0 tx=0x7f74a8008040 comp rx=0 tx=0).stop 2024-04-24T05:46:26.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.644+0000 7f74be000640 1 -- 172.21.15.123:0/1860066016 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b80d1500 msgr2=0x7f74b8161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.644+0000 7f74be000640 1 --2- 172.21.15.123:0/1860066016 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b80d1500 0x7f74b8161040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f74b000caf0 tx=0x7f74b0002170 comp rx=0 tx=0).stop 2024-04-24T05:46:26.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.644+0000 7f74be000640 1 -- 172.21.15.123:0/1860066016 shutdown_connections 2024-04-24T05:46:26.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.644+0000 7f74be000640 1 --2- 172.21.15.123:0/1860066016 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f749406c8f0 0x7f749406edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.644+0000 7f74be000640 1 --2- 172.21.15.123:0/1860066016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74b815ebf0 0x7f74b80ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.644+0000 7f74be000640 1 --2- 172.21.15.123:0/1860066016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b80d35f0 0x7f74b80c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.644+0000 7f74be000640 1 --2- 172.21.15.123:0/1860066016 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b80d1500 0x7f74b8161040 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.644+0000 7f74be000640 1 -- 172.21.15.123:0/1860066016 >> 172.21.15.123:0/1860066016 conn(0x7f74b80b9570 msgr2=0x7f74b81e96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.644+0000 7f74be000640 1 -- 172.21.15.123:0/1860066016 shutdown_connections 2024-04-24T05:46:26.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.644+0000 7f74be000640 1 -- 172.21.15.123:0/1860066016 wait complete. 2024-04-24T05:46:26.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.661+0000 7f55ce400640 1 -- 172.21.15.123:0/901733879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f55c00e2a30 con 0x7f55c00574f0 2024-04-24T05:46:26.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-04-24T05:46:26.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.661+0000 7f55c4c00640 1 -- 172.21.15.123:0/901733879 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f55c80d5040 con 0x7f55c00574f0 2024-04-24T05:46:26.663 INFO:teuthology.orchestra.run.smithi123.stdout:55834574857 2024-04-24T05:46:26.664 DEBUG:teuthology.parallel:result is None 2024-04-24T05:46:26.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.663+0000 7f8851c00640 1 -- 172.21.15.123:0/30238947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f884c161640 con 0x7f884c0b76a0 2024-04-24T05:46:26.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.663+0000 7f8843e00640 1 -- 172.21.15.123:0/30238947 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8844059b90 con 0x7f884c0b76a0 2024-04-24T05:46:26.665 INFO:teuthology.orchestra.run.smithi123.stdout:55834574859 2024-04-24T05:46:26.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.664+0000 7f55ce400640 1 -- 172.21.15.123:0/901733879 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f55a806c940 msgr2=0x7f55a806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.664+0000 7f55ce400640 1 --2- 172.21.15.123:0/901733879 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f55a806c940 0x7f55a806ee00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f55bc034000 tx=0x7f55bc02e000 comp rx=0 tx=0).stop 2024-04-24T05:46:26.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.664+0000 7f55ce400640 1 -- 172.21.15.123:0/901733879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55c00574f0 msgr2=0x7f55c00e4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.664+0000 7f55ce400640 1 --2- 172.21.15.123:0/901733879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55c00574f0 0x7f55c00e4790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f55c80512e0 tx=0x7f55c8069500 comp rx=0 tx=0).stop 2024-04-24T05:46:26.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.665+0000 7f55ce400640 1 -- 172.21.15.123:0/901733879 shutdown_connections 2024-04-24T05:46:26.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.665+0000 7f55ce400640 1 --2- 172.21.15.123:0/901733879 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f55a806c940 0x7f55a806ee00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.665+0000 7f55ce400640 1 --2- 172.21.15.123:0/901733879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55c00e4cd0 0x7f55c0178a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.665+0000 7f55ce400640 1 --2- 172.21.15.123:0/901733879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55c00574f0 0x7f55c00e4790 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f55c80512e0 tx=0x7f55c8069500 comp rx=0 tx=0).stop 2024-04-24T05:46:26.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.665+0000 7f55ce400640 1 --2- 172.21.15.123:0/901733879 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55c00075f0 0x7f55c0038ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.665+0000 7f55ce400640 1 -- 172.21.15.123:0/901733879 >> 172.21.15.123:0/901733879 conn(0x7f55c0058ee0 msgr2=0x7f55c004ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.665+0000 7f55ce400640 1 -- 172.21.15.123:0/901733879 shutdown_connections 2024-04-24T05:46:26.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.665+0000 7f55ce400640 1 -- 172.21.15.123:0/901733879 wait complete. 2024-04-24T05:46:26.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.666+0000 7f8851c00640 1 -- 172.21.15.123:0/30238947 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f882c06c940 msgr2=0x7f882c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.666+0000 7f8851c00640 1 --2- 172.21.15.123:0/30238947 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f882c06c940 0x7f882c06ee00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f884c073990 tx=0x7f883c006c40 comp rx=0 tx=0).stop 2024-04-24T05:46:26.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.666+0000 7f8851c00640 1 -- 172.21.15.123:0/30238947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c0b76a0 msgr2=0x7f884c1579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.666+0000 7f8851c00640 1 --2- 172.21.15.123:0/30238947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c0b76a0 0x7f884c1579b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f884400efc0 tx=0x7f884400c490 comp rx=0 tx=0).stop 2024-04-24T05:46:26.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.666+0000 7f8851c00640 1 -- 172.21.15.123:0/30238947 shutdown_connections 2024-04-24T05:46:26.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.666+0000 7f8851c00640 1 --2- 172.21.15.123:0/30238947 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f882c06c940 0x7f882c06ee00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.666+0000 7f8851c00640 1 --2- 172.21.15.123:0/30238947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f884c15e680 0x7f884c0b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.666+0000 7f8851c00640 1 --2- 172.21.15.123:0/30238947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f884c0ceac0 0x7f884c0b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.666+0000 7f8851c00640 1 --2- 172.21.15.123:0/30238947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f884c0b76a0 0x7f884c1579b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.666+0000 7f8851c00640 1 -- 172.21.15.123:0/30238947 >> 172.21.15.123:0/30238947 conn(0x7f884c0b5f60 msgr2=0x7f884c06c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.667+0000 7f8851c00640 1 -- 172.21.15.123:0/30238947 shutdown_connections 2024-04-24T05:46:26.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.667+0000 7f8851c00640 1 -- 172.21.15.123:0/30238947 wait complete. 2024-04-24T05:46:26.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-04-24T05:46:26.671 DEBUG:teuthology.parallel:result is None 2024-04-24T05:46:26.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.676+0000 7ff4bee00640 1 -- 172.21.15.123:0/3160973148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff4b806edd0 con 0x7ff49c0051c0 2024-04-24T05:46:26.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.676+0000 7ff4afe00640 1 -- 172.21.15.123:0/3160973148 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff4a8059260 con 0x7ff49c0051c0 2024-04-24T05:46:26.679 INFO:teuthology.orchestra.run.smithi123.stdout:55834574858 2024-04-24T05:46:26.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.680+0000 7ff4bee00640 1 -- 172.21.15.123:0/3160973148 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff49806c970 msgr2=0x7ff49806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.680+0000 7ff4bee00640 1 --2- 172.21.15.123:0/3160973148 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff49806c970 0x7ff49806ee30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80d1200 tx=0x7ff4a4009040 comp rx=0 tx=0).stop 2024-04-24T05:46:26.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.680+0000 7ff4bee00640 1 -- 172.21.15.123:0/3160973148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff49c0051c0 msgr2=0x7ff4b8153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.680+0000 7ff4bee00640 1 --2- 172.21.15.123:0/3160973148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff49c0051c0 0x7ff4b8153e00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8004e20 tx=0x7ff4a8007430 comp rx=0 tx=0).stop 2024-04-24T05:46:26.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.680+0000 7ff4bee00640 1 -- 172.21.15.123:0/3160973148 shutdown_connections 2024-04-24T05:46:26.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.680+0000 7ff4bee00640 1 --2- 172.21.15.123:0/3160973148 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff49806c970 0x7ff49806ee30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.680+0000 7ff4bee00640 1 --2- 172.21.15.123:0/3160973148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80b0680 0x7ff4b80b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.680+0000 7ff4bee00640 1 --2- 172.21.15.123:0/3160973148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b80cd9f0 0x7ff4b80cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.680+0000 7ff4bee00640 1 --2- 172.21.15.123:0/3160973148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff49c0051c0 0x7ff4b8153e00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.680+0000 7ff4bee00640 1 -- 172.21.15.123:0/3160973148 >> 172.21.15.123:0/3160973148 conn(0x7ff4b80b1e70 msgr2=0x7ff4b815fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.680+0000 7ff4bee00640 1 -- 172.21.15.123:0/3160973148 shutdown_connections 2024-04-24T05:46:26.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.680+0000 7ff4bee00640 1 -- 172.21.15.123:0/3160973148 wait complete. 2024-04-24T05:46:26.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-04-24T05:46:26.690 DEBUG:teuthology.parallel:result is None 2024-04-24T05:46:26.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2024-04-24T05:46:26.691 DEBUG:teuthology.parallel:result is None 2024-04-24T05:46:26.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.697+0000 7f1bf6000640 1 -- 172.21.15.123:0/2293222606 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1bf00d5000 con 0x7f1bf00d4390 2024-04-24T05:46:26.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.697+0000 7f1becc00640 1 -- 172.21.15.123:0/2293222606 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1bf00caad0 con 0x7f1bf00d4390 2024-04-24T05:46:26.699 INFO:teuthology.orchestra.run.smithi123.stdout:55834574857 2024-04-24T05:46:26.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-04-24T05:46:26.700 DEBUG:teuthology.parallel:result is None 2024-04-24T05:46:26.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.700+0000 7f1bf6000640 1 -- 172.21.15.123:0/2293222606 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1bd006c9c0 msgr2=0x7f1bd006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.700+0000 7f1bf6000640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1bd006c9c0 0x7f1bd006ee80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00ca290 tx=0x7f1be4006f90 comp rx=0 tx=0).stop 2024-04-24T05:46:26.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.700+0000 7f1bf6000640 1 -- 172.21.15.123:0/2293222606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bf00d4390 msgr2=0x7f1bf006dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.700+0000 7f1bf6000640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bf00d4390 0x7f1bf006dff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc031000 tx=0x7f1bdc0046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.700+0000 7f1bf6000640 1 -- 172.21.15.123:0/2293222606 shutdown_connections 2024-04-24T05:46:26.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.700+0000 7f1bf6000640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1bd006c9c0 0x7f1bd006ee80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.700+0000 7f1bf6000640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf015e4f0 0x7f1bf006c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.700+0000 7f1bf6000640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bf00d4390 0x7f1bf006dff0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.700+0000 7f1bf6000640 1 --2- 172.21.15.123:0/2293222606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bf00d3440 0x7f1bf006dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.700+0000 7f1bf6000640 1 -- 172.21.15.123:0/2293222606 >> 172.21.15.123:0/2293222606 conn(0x7f1bf009d780 msgr2=0x7f1bf0161590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.700+0000 7f1bf6000640 1 -- 172.21.15.123:0/2293222606 shutdown_connections 2024-04-24T05:46:26.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.700+0000 7f1bf6000640 1 -- 172.21.15.123:0/2293222606 wait complete. 2024-04-24T05:46:26.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2024-04-24T05:46:26.718 DEBUG:teuthology.parallel:result is None 2024-04-24T05:46:26.718 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T05:46:26.718 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T05:46:26.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.952+0000 7f4b06800640 1 Processor -- start 2024-04-24T05:46:26.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.952+0000 7f4b06800640 1 -- start start 2024-04-24T05:46:26.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.953+0000 7f4b06800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b000adee0 0x7f4b000acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.953+0000 7f4b06800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4b000ae2e0 0x7f4b000ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.953+0000 7f4b06800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b00160120 con 0x7f4b0015c5d0 2024-04-24T05:46:26.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.953+0000 7f4b06800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4b00160470 con 0x7f4b000ae2e0 2024-04-24T05:46:26.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.953+0000 7f4b06800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4b00161010 con 0x7f4b000adee0 2024-04-24T05:46:26.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.954+0000 7f4b05e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4b000ae2e0 0x7f4b000ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.954+0000 7f4b05e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4b000ae2e0 0x7f4b000ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56818/0 (socket says 172.21.15.123:56818) 2024-04-24T05:46:26.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.954+0000 7f4b05e00640 1 -- 172.21.15.123:0/1559250815 learned_addr learned my addr 172.21.15.123:0/1559250815 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:26.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.954+0000 7f4b04a00640 1 --2- 172.21.15.123:0/1559250815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b000adee0 0x7f4b000acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.954+0000 7f4b05e00640 1 -- 172.21.15.123:0/1559250815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b000adee0 msgr2=0x7f4b000acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.954+0000 7f4b05e00640 1 --2- 172.21.15.123:0/1559250815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b000adee0 0x7f4b000acf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.954+0000 7f4b05e00640 1 -- 172.21.15.123:0/1559250815 >> v1:172.21.15.155:6789/0 conn(0x7f4b0015c5d0 legacy=0x7f4b00160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:26.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.954+0000 7f4b05e00640 1 -- 172.21.15.123:0/1559250815 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0006dd80 con 0x7f4b000ae2e0 2024-04-24T05:46:26.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.955+0000 7f4b05e00640 1 --2- 172.21.15.123:0/1559250815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b000ae2e0 0x7f4b000ad480 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009e50 tx=0x7f4af802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9024e13809eb2196 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.955+0000 7f4affe00640 1 -- 172.21.15.123:0/1559250815 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8036070 con 0x7f4b000ae2e0 2024-04-24T05:46:26.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.955+0000 7f4affe00640 1 -- 172.21.15.123:0/1559250815 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af8031600 con 0x7f4b000ae2e0 2024-04-24T05:46:26.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.955+0000 7f4affe00640 1 -- 172.21.15.123:0/1559250815 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af80319d0 con 0x7f4b000ae2e0 2024-04-24T05:46:26.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.956+0000 7f4b06800640 1 -- 172.21.15.123:0/1559250815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b000ae2e0 msgr2=0x7f4b000ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.956+0000 7f4b06800640 1 --2- 172.21.15.123:0/1559250815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b000ae2e0 0x7f4b000ad480 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009e50 tx=0x7f4af802a710 comp rx=0 tx=0).stop 2024-04-24T05:46:26.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.956+0000 7f4b06800640 1 -- 172.21.15.123:0/1559250815 shutdown_connections 2024-04-24T05:46:26.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.956+0000 7f4b06800640 1 --2- 172.21.15.123:0/1559250815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b000ae2e0 0x7f4b000ad480 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.956+0000 7f4b06800640 1 --2- 172.21.15.123:0/1559250815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b000adee0 0x7f4b000acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.956+0000 7f4b06800640 1 -- 172.21.15.123:0/1559250815 >> 172.21.15.123:0/1559250815 conn(0x7f4b000d0070 msgr2=0x7f4b000d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:26.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.956+0000 7f4b06800640 1 -- 172.21.15.123:0/1559250815 shutdown_connections 2024-04-24T05:46:26.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.956+0000 7f4b06800640 1 -- 172.21.15.123:0/1559250815 wait complete. 2024-04-24T05:46:26.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.958+0000 7f4b06800640 1 Processor -- start 2024-04-24T05:46:26.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.958+0000 7f4b06800640 1 -- start start 2024-04-24T05:46:26.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.959+0000 7f4b06800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b000adee0 0x7f4b000b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.959+0000 7f4b06800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b000ae2e0 0x7f4b001f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.959+0000 7f4b06800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b0015c5d0 0x7f4b001f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.959+0000 7f4b06800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b000d0b60 con 0x7f4b000adee0 2024-04-24T05:46:26.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.959+0000 7f4b06800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0015c9b0 con 0x7f4b000ae2e0 2024-04-24T05:46:26.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.959+0000 7f4b06800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4b000d0ce0 con 0x7f4b0015c5d0 2024-04-24T05:46:26.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.959+0000 7f4b05400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b000adee0 0x7f4b000b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.959+0000 7f4b05400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b000adee0 0x7f4b000b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56832/0 (socket says 172.21.15.123:56832) 2024-04-24T05:46:26.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.959+0000 7f4b05400640 1 -- 172.21.15.123:0/1873612147 learned_addr learned my addr 172.21.15.123:0/1873612147 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:26.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.959+0000 7f4b05400640 1 -- 172.21.15.123:0/1873612147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b0015c5d0 msgr2=0x7f4b001f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.959+0000 7f4b05400640 1 --2- 172.21.15.123:0/1873612147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b0015c5d0 0x7f4b001f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.959+0000 7f4b05400640 1 -- 172.21.15.123:0/1873612147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b000ae2e0 msgr2=0x7f4b001f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:26.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.959+0000 7f4b05400640 1 --2- 172.21.15.123:0/1873612147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b000ae2e0 0x7f4b001f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:26.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.959+0000 7f4b05400640 1 -- 172.21.15.123:0/1873612147 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b000b3060 con 0x7f4b000adee0 2024-04-24T05:46:26.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.960+0000 7f4b05400640 1 --2- 172.21.15.123:0/1873612147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b000adee0 0x7f4b000b3b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4af400c400 tx=0x7f4af400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.960+0000 7f4afd600640 1 -- 172.21.15.123:0/1873612147 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af40052d0 con 0x7f4b000adee0 2024-04-24T05:46:26.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.960+0000 7f4afd600640 1 -- 172.21.15.123:0/1873612147 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af4005490 con 0x7f4b000adee0 2024-04-24T05:46:26.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.960+0000 7f4b06800640 1 -- 172.21.15.123:0/1873612147 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b001f4e50 con 0x7f4b000adee0 2024-04-24T05:46:26.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.960+0000 7f4afd600640 1 -- 172.21.15.123:0/1873612147 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af4007260 con 0x7f4b000adee0 2024-04-24T05:46:26.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.960+0000 7f4b06800640 1 -- 172.21.15.123:0/1873612147 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b000b3300 con 0x7f4b000adee0 2024-04-24T05:46:26.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.961+0000 7f4b06800640 1 -- 172.21.15.123:0/1873612147 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b00075b70 con 0x7f4b000adee0 2024-04-24T05:46:26.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.962+0000 7f4afd600640 1 -- 172.21.15.123:0/1873612147 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4af4007400 con 0x7f4b000adee0 2024-04-24T05:46:26.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.963+0000 7f4afd600640 1 --2- 172.21.15.123:0/1873612147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ae006c8f0 0x7f4ae006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:26.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.963+0000 7f4afd600640 1 -- 172.21.15.123:0/1873612147 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4af408b460 con 0x7f4b000adee0 2024-04-24T05:46:26.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.963+0000 7f4b04a00640 1 --2- 172.21.15.123:0/1873612147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ae006c8f0 0x7f4ae006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:26.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.964+0000 7f4b04a00640 1 --2- 172.21.15.123:0/1873612147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ae006c8f0 0x7f4ae006edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00a810 tx=0x7f4aec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:26.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:26.967+0000 7f4afd600640 1 -- 172.21.15.123:0/1873612147 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4af40594b0 con 0x7f4b000adee0 2024-04-24T05:46:27.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.147+0000 7f4b06800640 1 -- 172.21.15.123:0/1873612147 --> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b000cfde0 con 0x7f4ae006c8f0 2024-04-24T05:46:27.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.157+0000 7f4afd600640 1 -- 172.21.15.123:0/1873612147 <== mgr.4101 v2:172.21.15.123:6832/1503718323 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7f4b000cfde0 con 0x7f4ae006c8f0 2024-04-24T05:46:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:27.165 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-24T05:46:27.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.162+0000 7f4b06800640 1 -- 172.21.15.123:0/1873612147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ae006c8f0 msgr2=0x7f4ae006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:27.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.162+0000 7f4b06800640 1 --2- 172.21.15.123:0/1873612147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ae006c8f0 0x7f4ae006edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00a810 tx=0x7f4aec008040 comp rx=0 tx=0).stop 2024-04-24T05:46:27.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.163+0000 7f4b06800640 1 -- 172.21.15.123:0/1873612147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b000adee0 msgr2=0x7f4b000b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:27.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.163+0000 7f4b06800640 1 --2- 172.21.15.123:0/1873612147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b000adee0 0x7f4b000b3b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4af400c400 tx=0x7f4af400c8c0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.163+0000 7f4b06800640 1 -- 172.21.15.123:0/1873612147 shutdown_connections 2024-04-24T05:46:27.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.163+0000 7f4b06800640 1 --2- 172.21.15.123:0/1873612147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ae006c8f0 0x7f4ae006edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.163+0000 7f4b06800640 1 --2- 172.21.15.123:0/1873612147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b0015c5d0 0x7f4b001f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.163+0000 7f4b06800640 1 --2- 172.21.15.123:0/1873612147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b000ae2e0 0x7f4b001f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.163+0000 7f4b06800640 1 --2- 172.21.15.123:0/1873612147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b000adee0 0x7f4b000b3b00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.163+0000 7f4b06800640 1 -- 172.21.15.123:0/1873612147 >> 172.21.15.123:0/1873612147 conn(0x7f4b000d0070 msgr2=0x7f4b00160900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:27.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.163+0000 7f4b06800640 1 -- 172.21.15.123:0/1873612147 shutdown_connections 2024-04-24T05:46:27.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.163+0000 7f4b06800640 1 -- 172.21.15.123:0/1873612147 wait complete. 2024-04-24T05:46:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-24T05:46:26.523325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219476,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755244,"statfs":{"total":773094113280,"available":772869369856,"internally_reserved":0,"allocated":3383296,"data_stored":2032520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007124"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063974+0000","last_change":"2024-04-24T05:45:59.220218+0000","last_active":"2024-04-24T05:46:06.063974+0000","last_peered":"2024-04-24T05:46:06.063974+0000","last_clean":"2024-04-24T05:46:06.063974+0000","last_became_active":"2024-04-24T05:45:59.219888+0000","last_became_peered":"2024-04-24T05:45:59.219888+0000","last_unstale":"2024-04-24T05:46:06.063974+0000","last_undegraded":"2024-04-24T05:46:06.063974+0000","last_fullsized":"2024-04-24T05:46:06.063974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062489+0000","last_change":"2024-04-24T05:46:00.218964+0000","last_active":"2024-04-24T05:46:06.062489+0000","last_peered":"2024-04-24T05:46:06.062489+0000","last_clean":"2024-04-24T05:46:06.062489+0000","last_became_active":"2024-04-24T05:46:00.218589+0000","last_became_peered":"2024-04-24T05:46:00.218589+0000","last_unstale":"2024-04-24T05:46:06.062489+0000","last_undegraded":"2024-04-24T05:46:06.062489+0000","last_fullsized":"2024-04-24T05:46:06.062489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:42.688050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059820+0000","last_change":"2024-04-24T05:46:04.525402+0000","last_active":"2024-04-24T05:46:06.059820+0000","last_peered":"2024-04-24T05:46:06.059820+0000","last_clean":"2024-04-24T05:46:06.059820+0000","last_became_active":"2024-04-24T05:46:04.524690+0000","last_became_peered":"2024-04-24T05:46:04.524690+0000","last_unstale":"2024-04-24T05:46:06.059820+0000","last_undegraded":"2024-04-24T05:46:06.059820+0000","last_fullsized":"2024-04-24T05:46:06.059820+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:15:19.979319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062738+0000","last_change":"2024-04-24T05:45:59.203824+0000","last_active":"2024-04-24T05:46:06.062738+0000","last_peered":"2024-04-24T05:46:06.062738+0000","last_clean":"2024-04-24T05:46:06.062738+0000","last_became_active":"2024-04-24T05:45:59.203396+0000","last_became_peered":"2024-04-24T05:45:59.203396+0000","last_unstale":"2024-04-24T05:46:06.062738+0000","last_undegraded":"2024-04-24T05:46:06.062738+0000","last_fullsized":"2024-04-24T05:46:06.062738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:53:31.596793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063701+0000","last_change":"2024-04-24T05:46:00.221369+0000","last_active":"2024-04-24T05:46:06.063701+0000","last_peered":"2024-04-24T05:46:06.063701+0000","last_clean":"2024-04-24T05:46:06.063701+0000","last_became_active":"2024-04-24T05:46:00.220890+0000","last_became_peered":"2024-04-24T05:46:00.220890+0000","last_unstale":"2024-04-24T05:46:06.063701+0000","last_undegraded":"2024-04-24T05:46:06.063701+0000","last_fullsized":"2024-04-24T05:46:06.063701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:27.080921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.246725+0000","last_change":"2024-04-24T05:46:04.274351+0000","last_active":"2024-04-24T05:46:05.246725+0000","last_peered":"2024-04-24T05:46:05.246725+0000","last_clean":"2024-04-24T05:46:05.246725+0000","last_became_active":"2024-04-24T05:46:04.273892+0000","last_became_peered":"2024-04-24T05:46:04.273892+0000","last_unstale":"2024-04-24T05:46:05.246725+0000","last_undegraded":"2024-04-24T05:46:05.246725+0000","last_fullsized":"2024-04-24T05:46:05.246725+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:39.015857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557980+0000","last_change":"2024-04-24T05:45:59.221936+0000","last_active":"2024-04-24T05:46:05.557980+0000","last_peered":"2024-04-24T05:46:05.557980+0000","last_clean":"2024-04-24T05:46:05.557980+0000","last_became_active":"2024-04-24T05:45:59.221563+0000","last_became_peered":"2024-04-24T05:45:59.221563+0000","last_unstale":"2024-04-24T05:46:05.557980+0000","last_undegraded":"2024-04-24T05:46:05.557980+0000","last_fullsized":"2024-04-24T05:46:05.557980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:34.072620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252559+0000","last_change":"2024-04-24T05:46:00.204200+0000","last_active":"2024-04-24T05:46:05.252559+0000","last_peered":"2024-04-24T05:46:05.252559+0000","last_clean":"2024-04-24T05:46:05.252559+0000","last_became_active":"2024-04-24T05:46:00.203884+0000","last_became_peered":"2024-04-24T05:46:00.203884+0000","last_unstale":"2024-04-24T05:46:05.252559+0000","last_undegraded":"2024-04-24T05:46:05.252559+0000","last_fullsized":"2024-04-24T05:46:05.252559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:05:06.286589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248046+0000","last_change":"2024-04-24T05:46:04.279449+0000","last_active":"2024-04-24T05:46:05.248046+0000","last_peered":"2024-04-24T05:46:05.248046+0000","last_clean":"2024-04-24T05:46:05.248046+0000","last_became_active":"2024-04-24T05:46:04.278915+0000","last_became_peered":"2024-04-24T05:46:04.278915+0000","last_unstale":"2024-04-24T05:46:05.248046+0000","last_undegraded":"2024-04-24T05:46:05.248046+0000","last_fullsized":"2024-04-24T05:46:05.248046+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:42.160187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557287+0000","last_change":"2024-04-24T05:45:59.220243+0000","last_active":"2024-04-24T05:46:05.557287+0000","last_peered":"2024-04-24T05:46:05.557287+0000","last_clean":"2024-04-24T05:46:05.557287+0000","last_became_active":"2024-04-24T05:45:59.219288+0000","last_became_peered":"2024-04-24T05:45:59.219288+0000","last_unstale":"2024-04-24T05:46:05.557287+0000","last_undegraded":"2024-04-24T05:46:05.557287+0000","last_fullsized":"2024-04-24T05:46:05.557287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253052+0000","last_change":"2024-04-24T05:46:00.208516+0000","last_active":"2024-04-24T05:46:05.253052+0000","last_peered":"2024-04-24T05:46:05.253052+0000","last_clean":"2024-04-24T05:46:05.253052+0000","last_became_active":"2024-04-24T05:46:00.208217+0000","last_became_peered":"2024-04-24T05:46:00.208217+0000","last_unstale":"2024-04-24T05:46:05.253052+0000","last_undegraded":"2024-04-24T05:46:05.253052+0000","last_fullsized":"2024-04-24T05:46:05.253052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:13.907547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.247497+0000","last_change":"2024-04-24T05:46:04.274319+0000","last_active":"2024-04-24T05:46:05.247497+0000","last_peered":"2024-04-24T05:46:05.247497+0000","last_clean":"2024-04-24T05:46:05.247497+0000","last_became_active":"2024-04-24T05:46:04.273687+0000","last_became_peered":"2024-04-24T05:46:04.273687+0000","last_unstale":"2024-04-24T05:46:05.247497+0000","last_undegraded":"2024-04-24T05:46:05.247497+0000","last_fullsized":"2024-04-24T05:46:05.247497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:46.740551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058089+0000","last_change":"2024-04-24T05:46:00.222735+0000","last_active":"2024-04-24T05:46:06.058089+0000","last_peered":"2024-04-24T05:46:06.058089+0000","last_clean":"2024-04-24T05:46:06.058089+0000","last_became_active":"2024-04-24T05:46:00.222428+0000","last_became_peered":"2024-04-24T05:46:00.222428+0000","last_unstale":"2024-04-24T05:46:06.058089+0000","last_undegraded":"2024-04-24T05:46:06.058089+0000","last_fullsized":"2024-04-24T05:46:06.058089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557728+0000","last_change":"2024-04-24T05:45:59.220847+0000","last_active":"2024-04-24T05:46:05.557728+0000","last_peered":"2024-04-24T05:46:05.557728+0000","last_clean":"2024-04-24T05:46:05.557728+0000","last_became_active":"2024-04-24T05:45:59.220187+0000","last_became_peered":"2024-04-24T05:45:59.220187+0000","last_unstale":"2024-04-24T05:46:05.557728+0000","last_undegraded":"2024-04-24T05:46:05.557728+0000","last_fullsized":"2024-04-24T05:46:05.557728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:58:40.227841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.249739+0000","last_change":"2024-04-24T05:46:04.271803+0000","last_active":"2024-04-24T05:46:05.249739+0000","last_peered":"2024-04-24T05:46:05.249739+0000","last_clean":"2024-04-24T05:46:05.249739+0000","last_became_active":"2024-04-24T05:46:04.270863+0000","last_became_peered":"2024-04-24T05:46:04.270863+0000","last_unstale":"2024-04-24T05:46:05.249739+0000","last_undegraded":"2024-04-24T05:46:05.249739+0000","last_fullsized":"2024-04-24T05:46:05.249739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:34.349539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061349+0000","last_change":"2024-04-24T05:46:00.216572+0000","last_active":"2024-04-24T05:46:06.061349+0000","last_peered":"2024-04-24T05:46:06.061349+0000","last_clean":"2024-04-24T05:46:06.061349+0000","last_became_active":"2024-04-24T05:46:00.216226+0000","last_became_peered":"2024-04-24T05:46:00.216226+0000","last_unstale":"2024-04-24T05:46:06.061349+0000","last_undegraded":"2024-04-24T05:46:06.061349+0000","last_fullsized":"2024-04-24T05:46:06.061349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607418+0000","last_change":"2024-04-24T05:45:59.201895+0000","last_active":"2024-04-24T05:46:05.607418+0000","last_peered":"2024-04-24T05:46:05.607418+0000","last_clean":"2024-04-24T05:46:05.607418+0000","last_became_active":"2024-04-24T05:45:59.201064+0000","last_became_peered":"2024-04-24T05:45:59.201064+0000","last_unstale":"2024-04-24T05:46:05.607418+0000","last_undegraded":"2024-04-24T05:46:05.607418+0000","last_fullsized":"2024-04-24T05:46:05.607418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248655+0000","last_change":"2024-04-24T05:46:04.278089+0000","last_active":"2024-04-24T05:46:05.248655+0000","last_peered":"2024-04-24T05:46:05.248655+0000","last_clean":"2024-04-24T05:46:05.248655+0000","last_became_active":"2024-04-24T05:46:04.277685+0000","last_became_peered":"2024-04-24T05:46:04.277685+0000","last_unstale":"2024-04-24T05:46:05.248655+0000","last_undegraded":"2024-04-24T05:46:05.248655+0000","last_fullsized":"2024-04-24T05:46:05.248655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:05:10.339090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059639+0000","last_change":"2024-04-24T05:46:00.210339+0000","last_active":"2024-04-24T05:46:06.059639+0000","last_peered":"2024-04-24T05:46:06.059639+0000","last_clean":"2024-04-24T05:46:06.059639+0000","last_became_active":"2024-04-24T05:46:00.209637+0000","last_became_peered":"2024-04-24T05:46:00.209637+0000","last_unstale":"2024-04-24T05:46:06.059639+0000","last_undegraded":"2024-04-24T05:46:06.059639+0000","last_fullsized":"2024-04-24T05:46:06.059639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606639+0000","last_change":"2024-04-24T05:45:59.204528+0000","last_active":"2024-04-24T05:46:05.606639+0000","last_peered":"2024-04-24T05:46:05.606639+0000","last_clean":"2024-04-24T05:46:05.606639+0000","last_became_active":"2024-04-24T05:45:59.204200+0000","last_became_peered":"2024-04-24T05:45:59.204200+0000","last_unstale":"2024-04-24T05:46:05.606639+0000","last_undegraded":"2024-04-24T05:46:05.606639+0000","last_fullsized":"2024-04-24T05:46:05.606639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:26.453566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608562+0000","last_change":"2024-04-24T05:46:04.521949+0000","last_active":"2024-04-24T05:46:05.608562+0000","last_peered":"2024-04-24T05:46:05.608562+0000","last_clean":"2024-04-24T05:46:05.608562+0000","last_became_active":"2024-04-24T05:46:04.521423+0000","last_became_peered":"2024-04-24T05:46:04.521423+0000","last_unstale":"2024-04-24T05:46:05.608562+0000","last_undegraded":"2024-04-24T05:46:05.608562+0000","last_fullsized":"2024-04-24T05:46:05.608562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:44.521141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248230+0000","last_change":"2024-04-24T05:46:00.209888+0000","last_active":"2024-04-24T05:46:05.248230+0000","last_peered":"2024-04-24T05:46:05.248230+0000","last_clean":"2024-04-24T05:46:05.248230+0000","last_became_active":"2024-04-24T05:46:00.209568+0000","last_became_peered":"2024-04-24T05:46:00.209568+0000","last_unstale":"2024-04-24T05:46:05.248230+0000","last_undegraded":"2024-04-24T05:46:05.248230+0000","last_fullsized":"2024-04-24T05:46:05.248230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:21.844840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557732+0000","last_change":"2024-04-24T05:45:59.218584+0000","last_active":"2024-04-24T05:46:05.557732+0000","last_peered":"2024-04-24T05:46:05.557732+0000","last_clean":"2024-04-24T05:46:05.557732+0000","last_became_active":"2024-04-24T05:45:59.218316+0000","last_became_peered":"2024-04-24T05:45:59.218316+0000","last_unstale":"2024-04-24T05:46:05.557732+0000","last_undegraded":"2024-04-24T05:46:05.557732+0000","last_fullsized":"2024-04-24T05:46:05.557732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.249754+0000","last_change":"2024-04-24T05:46:04.281339+0000","last_active":"2024-04-24T05:46:05.249754+0000","last_peered":"2024-04-24T05:46:05.249754+0000","last_clean":"2024-04-24T05:46:05.249754+0000","last_became_active":"2024-04-24T05:46:04.280972+0000","last_became_peered":"2024-04-24T05:46:04.280972+0000","last_unstale":"2024-04-24T05:46:05.249754+0000","last_undegraded":"2024-04-24T05:46:05.249754+0000","last_fullsized":"2024-04-24T05:46:05.249754+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:17.960048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.064250+0000","last_change":"2024-04-24T05:46:00.210893+0000","last_active":"2024-04-24T05:46:06.064250+0000","last_peered":"2024-04-24T05:46:06.064250+0000","last_clean":"2024-04-24T05:46:06.064250+0000","last_became_active":"2024-04-24T05:46:00.210566+0000","last_became_peered":"2024-04-24T05:46:00.210566+0000","last_unstale":"2024-04-24T05:46:06.064250+0000","last_undegraded":"2024-04-24T05:46:06.064250+0000","last_fullsized":"2024-04-24T05:46:06.064250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059489+0000","last_change":"2024-04-24T05:45:59.213341+0000","last_active":"2024-04-24T05:46:06.059489+0000","last_peered":"2024-04-24T05:46:06.059489+0000","last_clean":"2024-04-24T05:46:06.059489+0000","last_became_active":"2024-04-24T05:45:59.212756+0000","last_became_peered":"2024-04-24T05:45:59.212756+0000","last_unstale":"2024-04-24T05:46:06.059489+0000","last_undegraded":"2024-04-24T05:46:06.059489+0000","last_fullsized":"2024-04-24T05:46:06.059489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:34.072620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.247506+0000","last_change":"2024-04-24T05:46:04.524925+0000","last_active":"2024-04-24T05:46:05.247506+0000","last_peered":"2024-04-24T05:46:05.247506+0000","last_clean":"2024-04-24T05:46:05.247506+0000","last_became_active":"2024-04-24T05:46:04.523828+0000","last_became_peered":"2024-04-24T05:46:04.523828+0000","last_unstale":"2024-04-24T05:46:05.247506+0000","last_undegraded":"2024-04-24T05:46:05.247506+0000","last_fullsized":"2024-04-24T05:46:05.247506+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:45.904568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063326+0000","last_change":"2024-04-24T05:46:00.227429+0000","last_active":"2024-04-24T05:46:06.063326+0000","last_peered":"2024-04-24T05:46:06.063326+0000","last_clean":"2024-04-24T05:46:06.063326+0000","last_became_active":"2024-04-24T05:46:00.226909+0000","last_became_peered":"2024-04-24T05:46:00.226909+0000","last_unstale":"2024-04-24T05:46:06.063326+0000","last_undegraded":"2024-04-24T05:46:06.063326+0000","last_fullsized":"2024-04-24T05:46:06.063326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:45.039541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608337+0000","last_change":"2024-04-24T05:45:59.203340+0000","last_active":"2024-04-24T05:46:05.608337+0000","last_peered":"2024-04-24T05:46:05.608337+0000","last_clean":"2024-04-24T05:46:05.608337+0000","last_became_active":"2024-04-24T05:45:59.203004+0000","last_became_peered":"2024-04-24T05:45:59.203004+0000","last_unstale":"2024-04-24T05:46:05.608337+0000","last_undegraded":"2024-04-24T05:46:05.608337+0000","last_fullsized":"2024-04-24T05:46:05.608337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:34.072620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059109+0000","last_change":"2024-04-24T05:46:04.528328+0000","last_active":"2024-04-24T05:46:06.059109+0000","last_peered":"2024-04-24T05:46:06.059109+0000","last_clean":"2024-04-24T05:46:06.059109+0000","last_became_active":"2024-04-24T05:46:04.527760+0000","last_became_peered":"2024-04-24T05:46:04.527760+0000","last_unstale":"2024-04-24T05:46:06.059109+0000","last_undegraded":"2024-04-24T05:46:06.059109+0000","last_fullsized":"2024-04-24T05:46:06.059109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:16:08.677684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607804+0000","last_change":"2024-04-24T05:46:00.215704+0000","last_active":"2024-04-24T05:46:05.607804+0000","last_peered":"2024-04-24T05:46:05.607804+0000","last_clean":"2024-04-24T05:46:05.607804+0000","last_became_active":"2024-04-24T05:46:00.214751+0000","last_became_peered":"2024-04-24T05:46:00.214751+0000","last_unstale":"2024-04-24T05:46:05.607804+0000","last_undegraded":"2024-04-24T05:46:05.607804+0000","last_fullsized":"2024-04-24T05:46:05.607804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558293+0000","last_change":"2024-04-24T05:45:59.219396+0000","last_active":"2024-04-24T05:46:05.558293+0000","last_peered":"2024-04-24T05:46:05.558293+0000","last_clean":"2024-04-24T05:46:05.558293+0000","last_became_active":"2024-04-24T05:45:59.217765+0000","last_became_peered":"2024-04-24T05:45:59.217765+0000","last_unstale":"2024-04-24T05:46:05.558293+0000","last_undegraded":"2024-04-24T05:46:05.558293+0000","last_fullsized":"2024-04-24T05:46:05.558293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:21:56.958729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061548+0000","last_change":"2024-04-24T05:46:04.518477+0000","last_active":"2024-04-24T05:46:06.061548+0000","last_peered":"2024-04-24T05:46:06.061548+0000","last_clean":"2024-04-24T05:46:06.061548+0000","last_became_active":"2024-04-24T05:46:04.515078+0000","last_became_peered":"2024-04-24T05:46:04.515078+0000","last_unstale":"2024-04-24T05:46:06.061548+0000","last_undegraded":"2024-04-24T05:46:06.061548+0000","last_fullsized":"2024-04-24T05:46:06.061548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:31.102602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253112+0000","last_change":"2024-04-24T05:46:00.212998+0000","last_active":"2024-04-24T05:46:05.253112+0000","last_peered":"2024-04-24T05:46:05.253112+0000","last_clean":"2024-04-24T05:46:05.253112+0000","last_became_active":"2024-04-24T05:46:00.212446+0000","last_became_peered":"2024-04-24T05:46:00.212446+0000","last_unstale":"2024-04-24T05:46:05.253112+0000","last_undegraded":"2024-04-24T05:46:05.253112+0000","last_fullsized":"2024-04-24T05:46:05.253112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:27.050101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607823+0000","last_change":"2024-04-24T05:45:59.197842+0000","last_active":"2024-04-24T05:46:05.607823+0000","last_peered":"2024-04-24T05:46:05.607823+0000","last_clean":"2024-04-24T05:46:05.607823+0000","last_became_active":"2024-04-24T05:45:59.197510+0000","last_became_peered":"2024-04-24T05:45:59.197510+0000","last_unstale":"2024-04-24T05:46:05.607823+0000","last_undegraded":"2024-04-24T05:46:05.607823+0000","last_fullsized":"2024-04-24T05:46:05.607823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606699+0000","last_change":"2024-04-24T05:46:04.270957+0000","last_active":"2024-04-24T05:46:05.606699+0000","last_peered":"2024-04-24T05:46:05.606699+0000","last_clean":"2024-04-24T05:46:05.606699+0000","last_became_active":"2024-04-24T05:46:04.268656+0000","last_became_peered":"2024-04-24T05:46:04.268656+0000","last_unstale":"2024-04-24T05:46:05.606699+0000","last_undegraded":"2024-04-24T05:46:05.606699+0000","last_fullsized":"2024-04-24T05:46:05.606699+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:31.102602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248701+0000","last_change":"2024-04-24T05:46:00.205128+0000","last_active":"2024-04-24T05:46:05.248701+0000","last_peered":"2024-04-24T05:46:05.248701+0000","last_clean":"2024-04-24T05:46:05.248701+0000","last_became_active":"2024-04-24T05:46:00.202044+0000","last_became_peered":"2024-04-24T05:46:00.202044+0000","last_unstale":"2024-04-24T05:46:05.248701+0000","last_undegraded":"2024-04-24T05:46:05.248701+0000","last_fullsized":"2024-04-24T05:46:05.248701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:18.772951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252226+0000","last_change":"2024-04-24T05:45:59.199320+0000","last_active":"2024-04-24T05:46:05.252226+0000","last_peered":"2024-04-24T05:46:05.252226+0000","last_clean":"2024-04-24T05:46:05.252226+0000","last_became_active":"2024-04-24T05:45:59.198382+0000","last_became_peered":"2024-04-24T05:45:59.198382+0000","last_unstale":"2024-04-24T05:46:05.252226+0000","last_undegraded":"2024-04-24T05:46:05.252226+0000","last_fullsized":"2024-04-24T05:46:05.252226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607713+0000","last_change":"2024-04-24T05:46:04.273947+0000","last_active":"2024-04-24T05:46:05.607713+0000","last_peered":"2024-04-24T05:46:05.607713+0000","last_clean":"2024-04-24T05:46:05.607713+0000","last_became_active":"2024-04-24T05:46:04.271583+0000","last_became_peered":"2024-04-24T05:46:04.271583+0000","last_unstale":"2024-04-24T05:46:05.607713+0000","last_undegraded":"2024-04-24T05:46:05.607713+0000","last_fullsized":"2024-04-24T05:46:05.607713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:45.991179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606401+0000","last_change":"2024-04-24T05:46:00.211997+0000","last_active":"2024-04-24T05:46:05.606401+0000","last_peered":"2024-04-24T05:46:05.606401+0000","last_clean":"2024-04-24T05:46:05.606401+0000","last_became_active":"2024-04-24T05:46:00.209526+0000","last_became_peered":"2024-04-24T05:46:00.209526+0000","last_unstale":"2024-04-24T05:46:05.606401+0000","last_undegraded":"2024-04-24T05:46:05.606401+0000","last_fullsized":"2024-04-24T05:46:05.606401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:27.080921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058600+0000","last_change":"2024-04-24T05:45:59.198738+0000","last_active":"2024-04-24T05:46:06.058600+0000","last_peered":"2024-04-24T05:46:06.058600+0000","last_clean":"2024-04-24T05:46:06.058600+0000","last_became_active":"2024-04-24T05:45:59.198458+0000","last_became_peered":"2024-04-24T05:45:59.198458+0000","last_unstale":"2024-04-24T05:46:06.058600+0000","last_undegraded":"2024-04-24T05:46:06.058600+0000","last_fullsized":"2024-04-24T05:46:06.058600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:11.561126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608670+0000","last_change":"2024-04-24T05:46:04.266757+0000","last_active":"2024-04-24T05:46:05.608670+0000","last_peered":"2024-04-24T05:46:05.608670+0000","last_clean":"2024-04-24T05:46:05.608670+0000","last_became_active":"2024-04-24T05:46:04.266279+0000","last_became_peered":"2024-04-24T05:46:04.266279+0000","last_unstale":"2024-04-24T05:46:05.608670+0000","last_undegraded":"2024-04-24T05:46:05.608670+0000","last_fullsized":"2024-04-24T05:46:05.608670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:06:00.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058554+0000","last_change":"2024-04-24T05:46:00.216303+0000","last_active":"2024-04-24T05:46:06.058554+0000","last_peered":"2024-04-24T05:46:06.058554+0000","last_clean":"2024-04-24T05:46:06.058554+0000","last_became_active":"2024-04-24T05:46:00.215719+0000","last_became_peered":"2024-04-24T05:46:00.215719+0000","last_unstale":"2024-04-24T05:46:06.058554+0000","last_undegraded":"2024-04-24T05:46:06.058554+0000","last_fullsized":"2024-04-24T05:46:06.058554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:47:58.509148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558034+0000","last_change":"2024-04-24T05:45:59.218643+0000","last_active":"2024-04-24T05:46:05.558034+0000","last_peered":"2024-04-24T05:46:05.558034+0000","last_clean":"2024-04-24T05:46:05.558034+0000","last_became_active":"2024-04-24T05:45:59.218346+0000","last_became_peered":"2024-04-24T05:45:59.218346+0000","last_unstale":"2024-04-24T05:46:05.558034+0000","last_undegraded":"2024-04-24T05:46:05.558034+0000","last_fullsized":"2024-04-24T05:46:05.558034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:26.453566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063814+0000","last_change":"2024-04-24T05:46:04.527188+0000","last_active":"2024-04-24T05:46:06.063814+0000","last_peered":"2024-04-24T05:46:06.063814+0000","last_clean":"2024-04-24T05:46:06.063814+0000","last_became_active":"2024-04-24T05:46:04.526002+0000","last_became_peered":"2024-04-24T05:46:04.526002+0000","last_unstale":"2024-04-24T05:46:06.063814+0000","last_undegraded":"2024-04-24T05:46:06.063814+0000","last_fullsized":"2024-04-24T05:46:06.063814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:06:00.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060620+0000","last_change":"2024-04-24T05:46:00.224034+0000","last_active":"2024-04-24T05:46:06.060620+0000","last_peered":"2024-04-24T05:46:06.060620+0000","last_clean":"2024-04-24T05:46:06.060620+0000","last_became_active":"2024-04-24T05:46:00.223714+0000","last_became_peered":"2024-04-24T05:46:00.223714+0000","last_unstale":"2024-04-24T05:46:06.060620+0000","last_undegraded":"2024-04-24T05:46:06.060620+0000","last_fullsized":"2024-04-24T05:46:06.060620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:25.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252259+0000","last_change":"2024-04-24T05:45:59.199095+0000","last_active":"2024-04-24T05:46:05.252259+0000","last_peered":"2024-04-24T05:46:05.252259+0000","last_clean":"2024-04-24T05:46:05.252259+0000","last_became_active":"2024-04-24T05:45:59.198772+0000","last_became_peered":"2024-04-24T05:45:59.198772+0000","last_unstale":"2024-04-24T05:46:05.252259+0000","last_undegraded":"2024-04-24T05:46:05.252259+0000","last_fullsized":"2024-04-24T05:46:05.252259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:11.561126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558345+0000","last_change":"2024-04-24T05:45:51.019903+0000","last_active":"2024-04-24T05:46:05.558345+0000","last_peered":"2024-04-24T05:46:05.558345+0000","last_clean":"2024-04-24T05:46:05.558345+0000","last_became_active":"2024-04-24T05:45:51.017323+0000","last_became_peered":"2024-04-24T05:45:51.017323+0000","last_unstale":"2024-04-24T05:46:05.558345+0000","last_undegraded":"2024-04-24T05:46:05.558345+0000","last_fullsized":"2024-04-24T05:46:05.558345+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:48.855871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:48.855871+0000","last_clean_scrub_stamp":"2024-04-24T05:45:48.855871+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:10:41.045203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058781+0000","last_change":"2024-04-24T05:46:04.525962+0000","last_active":"2024-04-24T05:46:06.058781+0000","last_peered":"2024-04-24T05:46:06.058781+0000","last_clean":"2024-04-24T05:46:06.058781+0000","last_became_active":"2024-04-24T05:46:04.524947+0000","last_became_peered":"2024-04-24T05:46:04.524947+0000","last_unstale":"2024-04-24T05:46:06.058781+0000","last_undegraded":"2024-04-24T05:46:06.058781+0000","last_fullsized":"2024-04-24T05:46:06.058781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:17.960048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557690+0000","last_change":"2024-04-24T05:46:00.226016+0000","last_active":"2024-04-24T05:46:05.557690+0000","last_peered":"2024-04-24T05:46:05.557690+0000","last_clean":"2024-04-24T05:46:05.557690+0000","last_became_active":"2024-04-24T05:46:00.225693+0000","last_became_peered":"2024-04-24T05:46:00.225693+0000","last_unstale":"2024-04-24T05:46:05.557690+0000","last_undegraded":"2024-04-24T05:46:05.557690+0000","last_fullsized":"2024-04-24T05:46:05.557690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:13.907547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059847+0000","last_change":"2024-04-24T05:45:59.203021+0000","last_active":"2024-04-24T05:46:06.059847+0000","last_peered":"2024-04-24T05:46:06.059847+0000","last_clean":"2024-04-24T05:46:06.059847+0000","last_became_active":"2024-04-24T05:45:59.202733+0000","last_became_peered":"2024-04-24T05:45:59.202733+0000","last_unstale":"2024-04-24T05:46:06.059847+0000","last_undegraded":"2024-04-24T05:46:06.059847+0000","last_fullsized":"2024-04-24T05:46:06.059847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058297+0000","last_change":"2024-04-24T05:46:04.521125+0000","last_active":"2024-04-24T05:46:06.058297+0000","last_peered":"2024-04-24T05:46:06.058297+0000","last_clean":"2024-04-24T05:46:06.058297+0000","last_became_active":"2024-04-24T05:46:04.519878+0000","last_became_peered":"2024-04-24T05:46:04.519878+0000","last_unstale":"2024-04-24T05:46:06.058297+0000","last_undegraded":"2024-04-24T05:46:06.058297+0000","last_fullsized":"2024-04-24T05:46:06.058297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:34.349539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252701+0000","last_change":"2024-04-24T05:46:00.214353+0000","last_active":"2024-04-24T05:46:05.252701+0000","last_peered":"2024-04-24T05:46:05.252701+0000","last_clean":"2024-04-24T05:46:05.252701+0000","last_became_active":"2024-04-24T05:46:00.213791+0000","last_became_peered":"2024-04-24T05:46:00.213791+0000","last_unstale":"2024-04-24T05:46:05.252701+0000","last_undegraded":"2024-04-24T05:46:05.252701+0000","last_fullsized":"2024-04-24T05:46:05.252701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:30.297037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608191+0000","last_change":"2024-04-24T05:45:59.216779+0000","last_active":"2024-04-24T05:46:05.608191+0000","last_peered":"2024-04-24T05:46:05.608191+0000","last_clean":"2024-04-24T05:46:05.608191+0000","last_became_active":"2024-04-24T05:45:59.216507+0000","last_became_peered":"2024-04-24T05:45:59.216507+0000","last_unstale":"2024-04-24T05:46:05.608191+0000","last_undegraded":"2024-04-24T05:46:05.608191+0000","last_fullsized":"2024-04-24T05:46:05.608191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062332+0000","last_change":"2024-04-24T05:46:04.268777+0000","last_active":"2024-04-24T05:46:06.062332+0000","last_peered":"2024-04-24T05:46:06.062332+0000","last_clean":"2024-04-24T05:46:06.062332+0000","last_became_active":"2024-04-24T05:46:04.268416+0000","last_became_peered":"2024-04-24T05:46:04.268416+0000","last_unstale":"2024-04-24T05:46:06.062332+0000","last_undegraded":"2024-04-24T05:46:06.062332+0000","last_fullsized":"2024-04-24T05:46:06.062332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:13.252851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558634+0000","last_change":"2024-04-24T05:46:00.227851+0000","last_active":"2024-04-24T05:46:05.558634+0000","last_peered":"2024-04-24T05:46:05.558634+0000","last_clean":"2024-04-24T05:46:05.558634+0000","last_became_active":"2024-04-24T05:46:00.220088+0000","last_became_peered":"2024-04-24T05:46:00.220088+0000","last_unstale":"2024-04-24T05:46:05.558634+0000","last_undegraded":"2024-04-24T05:46:05.558634+0000","last_fullsized":"2024-04-24T05:46:05.558634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:40.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059276+0000","last_change":"2024-04-24T05:45:59.196319+0000","last_active":"2024-04-24T05:46:06.059276+0000","last_peered":"2024-04-24T05:46:06.059276+0000","last_clean":"2024-04-24T05:46:06.059276+0000","last_became_active":"2024-04-24T05:45:59.196029+0000","last_became_peered":"2024-04-24T05:45:59.196029+0000","last_unstale":"2024-04-24T05:46:06.059276+0000","last_undegraded":"2024-04-24T05:46:06.059276+0000","last_fullsized":"2024-04-24T05:46:06.059276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.250996+0000","last_change":"2024-04-24T05:46:04.527040+0000","last_active":"2024-04-24T05:46:05.250996+0000","last_peered":"2024-04-24T05:46:05.250996+0000","last_clean":"2024-04-24T05:46:05.250996+0000","last_became_active":"2024-04-24T05:46:04.522889+0000","last_became_peered":"2024-04-24T05:46:04.522889+0000","last_unstale":"2024-04-24T05:46:05.250996+0000","last_undegraded":"2024-04-24T05:46:05.250996+0000","last_fullsized":"2024-04-24T05:46:05.250996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:06:00.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558335+0000","last_change":"2024-04-24T05:46:00.225225+0000","last_active":"2024-04-24T05:46:05.558335+0000","last_peered":"2024-04-24T05:46:05.558335+0000","last_clean":"2024-04-24T05:46:05.558335+0000","last_became_active":"2024-04-24T05:46:00.224932+0000","last_became_peered":"2024-04-24T05:46:00.224932+0000","last_unstale":"2024-04-24T05:46:05.558335+0000","last_undegraded":"2024-04-24T05:46:05.558335+0000","last_fullsized":"2024-04-24T05:46:05.558335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:41.938677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248435+0000","last_change":"2024-04-24T05:45:59.214909+0000","last_active":"2024-04-24T05:46:05.248435+0000","last_peered":"2024-04-24T05:46:05.248435+0000","last_clean":"2024-04-24T05:46:05.248435+0000","last_became_active":"2024-04-24T05:45:59.213540+0000","last_became_peered":"2024-04-24T05:45:59.213540+0000","last_unstale":"2024-04-24T05:46:05.248435+0000","last_undegraded":"2024-04-24T05:46:05.248435+0000","last_fullsized":"2024-04-24T05:46:05.248435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:00.340470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.246897+0000","last_change":"2024-04-24T05:46:04.275887+0000","last_active":"2024-04-24T05:46:05.246897+0000","last_peered":"2024-04-24T05:46:05.246897+0000","last_clean":"2024-04-24T05:46:05.246897+0000","last_became_active":"2024-04-24T05:46:04.275455+0000","last_became_peered":"2024-04-24T05:46:04.275455+0000","last_unstale":"2024-04-24T05:46:05.246897+0000","last_undegraded":"2024-04-24T05:46:05.246897+0000","last_fullsized":"2024-04-24T05:46:05.246897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:29.075099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557429+0000","last_change":"2024-04-24T05:46:00.217599+0000","last_active":"2024-04-24T05:46:05.557429+0000","last_peered":"2024-04-24T05:46:05.557429+0000","last_clean":"2024-04-24T05:46:05.557429+0000","last_became_active":"2024-04-24T05:46:00.217383+0000","last_became_peered":"2024-04-24T05:46:00.217383+0000","last_unstale":"2024-04-24T05:46:05.557429+0000","last_undegraded":"2024-04-24T05:46:05.557429+0000","last_fullsized":"2024-04-24T05:46:05.557429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:38.107685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.057867+0000","last_change":"2024-04-24T05:45:59.196826+0000","last_active":"2024-04-24T05:46:06.057867+0000","last_peered":"2024-04-24T05:46:06.057867+0000","last_clean":"2024-04-24T05:46:06.057867+0000","last_became_active":"2024-04-24T05:45:59.195023+0000","last_became_peered":"2024-04-24T05:45:59.195023+0000","last_unstale":"2024-04-24T05:46:06.057867+0000","last_undegraded":"2024-04-24T05:46:06.057867+0000","last_fullsized":"2024-04-24T05:46:06.057867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063777+0000","last_change":"2024-04-24T05:46:04.265949+0000","last_active":"2024-04-24T05:46:06.063777+0000","last_peered":"2024-04-24T05:46:06.063777+0000","last_clean":"2024-04-24T05:46:06.063777+0000","last_became_active":"2024-04-24T05:46:04.265546+0000","last_became_peered":"2024-04-24T05:46:04.265546+0000","last_unstale":"2024-04-24T05:46:06.063777+0000","last_undegraded":"2024-04-24T05:46:06.063777+0000","last_fullsized":"2024-04-24T05:46:06.063777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:44.521141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058813+0000","last_change":"2024-04-24T05:46:00.221203+0000","last_active":"2024-04-24T05:46:06.058813+0000","last_peered":"2024-04-24T05:46:06.058813+0000","last_clean":"2024-04-24T05:46:06.058813+0000","last_became_active":"2024-04-24T05:46:00.220709+0000","last_became_peered":"2024-04-24T05:46:00.220709+0000","last_unstale":"2024-04-24T05:46:06.058813+0000","last_undegraded":"2024-04-24T05:46:06.058813+0000","last_fullsized":"2024-04-24T05:46:06.058813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059170+0000","last_change":"2024-04-24T05:45:59.217694+0000","last_active":"2024-04-24T05:46:06.059170+0000","last_peered":"2024-04-24T05:46:06.059170+0000","last_clean":"2024-04-24T05:46:06.059170+0000","last_became_active":"2024-04-24T05:45:59.217385+0000","last_became_peered":"2024-04-24T05:45:59.217385+0000","last_unstale":"2024-04-24T05:46:06.059170+0000","last_undegraded":"2024-04-24T05:46:06.059170+0000","last_fullsized":"2024-04-24T05:46:06.059170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063100+0000","last_change":"2024-04-24T05:46:04.529839+0000","last_active":"2024-04-24T05:46:06.063100+0000","last_peered":"2024-04-24T05:46:06.063100+0000","last_clean":"2024-04-24T05:46:06.063100+0000","last_became_active":"2024-04-24T05:46:04.524578+0000","last_became_peered":"2024-04-24T05:46:04.524578+0000","last_unstale":"2024-04-24T05:46:06.063100+0000","last_undegraded":"2024-04-24T05:46:06.063100+0000","last_fullsized":"2024-04-24T05:46:06.063100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:26.692990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557529+0000","last_change":"2024-04-24T05:46:00.226661+0000","last_active":"2024-04-24T05:46:05.557529+0000","last_peered":"2024-04-24T05:46:05.557529+0000","last_clean":"2024-04-24T05:46:05.557529+0000","last_became_active":"2024-04-24T05:46:00.226357+0000","last_became_peered":"2024-04-24T05:46:00.226357+0000","last_unstale":"2024-04-24T05:46:05.557529+0000","last_undegraded":"2024-04-24T05:46:05.557529+0000","last_fullsized":"2024-04-24T05:46:05.557529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:05:06.286589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252051+0000","last_change":"2024-04-24T05:45:59.206276+0000","last_active":"2024-04-24T05:46:05.252051+0000","last_peered":"2024-04-24T05:46:05.252051+0000","last_clean":"2024-04-24T05:46:05.252051+0000","last_became_active":"2024-04-24T05:45:59.205926+0000","last_became_peered":"2024-04-24T05:45:59.205926+0000","last_unstale":"2024-04-24T05:46:05.252051+0000","last_undegraded":"2024-04-24T05:46:05.252051+0000","last_fullsized":"2024-04-24T05:46:05.252051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062045+0000","last_change":"2024-04-24T05:46:04.515144+0000","last_active":"2024-04-24T05:46:06.062045+0000","last_peered":"2024-04-24T05:46:06.062045+0000","last_clean":"2024-04-24T05:46:06.062045+0000","last_became_active":"2024-04-24T05:46:04.514326+0000","last_became_peered":"2024-04-24T05:46:04.514326+0000","last_unstale":"2024-04-24T05:46:06.062045+0000","last_undegraded":"2024-04-24T05:46:06.062045+0000","last_fullsized":"2024-04-24T05:46:06.062045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:45.847497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060827+0000","last_change":"2024-04-24T05:46:00.214864+0000","last_active":"2024-04-24T05:46:06.060827+0000","last_peered":"2024-04-24T05:46:06.060827+0000","last_clean":"2024-04-24T05:46:06.060827+0000","last_became_active":"2024-04-24T05:46:00.214051+0000","last_became_peered":"2024-04-24T05:46:00.214051+0000","last_unstale":"2024-04-24T05:46:06.060827+0000","last_undegraded":"2024-04-24T05:46:06.060827+0000","last_fullsized":"2024-04-24T05:46:06.060827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:41:43.988057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062825+0000","last_change":"2024-04-24T05:45:59.213057+0000","last_active":"2024-04-24T05:46:06.062825+0000","last_peered":"2024-04-24T05:46:06.062825+0000","last_clean":"2024-04-24T05:46:06.062825+0000","last_became_active":"2024-04-24T05:45:59.212716+0000","last_became_peered":"2024-04-24T05:45:59.212716+0000","last_unstale":"2024-04-24T05:46:06.062825+0000","last_undegraded":"2024-04-24T05:46:06.062825+0000","last_fullsized":"2024-04-24T05:46:06.062825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:53.435411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252861+0000","last_change":"2024-04-24T05:46:04.516510+0000","last_active":"2024-04-24T05:46:05.252861+0000","last_peered":"2024-04-24T05:46:05.252861+0000","last_clean":"2024-04-24T05:46:05.252861+0000","last_became_active":"2024-04-24T05:46:04.515458+0000","last_became_peered":"2024-04-24T05:46:04.515458+0000","last_unstale":"2024-04-24T05:46:05.252861+0000","last_undegraded":"2024-04-24T05:46:05.252861+0000","last_fullsized":"2024-04-24T05:46:05.252861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:16:08.677684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557591+0000","last_change":"2024-04-24T05:46:00.215087+0000","last_active":"2024-04-24T05:46:05.557591+0000","last_peered":"2024-04-24T05:46:05.557591+0000","last_clean":"2024-04-24T05:46:05.557591+0000","last_became_active":"2024-04-24T05:46:00.213783+0000","last_became_peered":"2024-04-24T05:46:00.213783+0000","last_unstale":"2024-04-24T05:46:05.557591+0000","last_undegraded":"2024-04-24T05:46:05.557591+0000","last_fullsized":"2024-04-24T05:46:05.557591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:30.297037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:17.734750+0000","last_change":"2024-04-24T05:45:59.211881+0000","last_active":"2024-04-24T05:46:17.734750+0000","last_peered":"2024-04-24T05:46:17.734750+0000","last_clean":"2024-04-24T05:46:17.734750+0000","last_became_active":"2024-04-24T05:45:59.211566+0000","last_became_peered":"2024-04-24T05:45:59.211566+0000","last_unstale":"2024-04-24T05:46:17.734750+0000","last_undegraded":"2024-04-24T05:46:17.734750+0000","last_fullsized":"2024-04-24T05:46:17.734750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607263+0000","last_change":"2024-04-24T05:46:04.264433+0000","last_active":"2024-04-24T05:46:05.607263+0000","last_peered":"2024-04-24T05:46:05.607263+0000","last_clean":"2024-04-24T05:46:05.607263+0000","last_became_active":"2024-04-24T05:46:04.263926+0000","last_became_peered":"2024-04-24T05:46:04.263926+0000","last_unstale":"2024-04-24T05:46:05.607263+0000","last_undegraded":"2024-04-24T05:46:05.607263+0000","last_fullsized":"2024-04-24T05:46:05.607263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:13.252851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248914+0000","last_change":"2024-04-24T05:46:00.218334+0000","last_active":"2024-04-24T05:46:05.248914+0000","last_peered":"2024-04-24T05:46:05.248914+0000","last_clean":"2024-04-24T05:46:05.248914+0000","last_became_active":"2024-04-24T05:46:00.213738+0000","last_became_peered":"2024-04-24T05:46:00.213738+0000","last_unstale":"2024-04-24T05:46:05.248914+0000","last_undegraded":"2024-04-24T05:46:05.248914+0000","last_fullsized":"2024-04-24T05:46:05.248914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:27.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608102+0000","last_change":"2024-04-24T05:45:59.206057+0000","last_active":"2024-04-24T05:46:05.608102+0000","last_peered":"2024-04-24T05:46:05.608102+0000","last_clean":"2024-04-24T05:46:05.608102+0000","last_became_active":"2024-04-24T05:45:59.205560+0000","last_became_peered":"2024-04-24T05:45:59.205560+0000","last_unstale":"2024-04-24T05:46:05.608102+0000","last_undegraded":"2024-04-24T05:46:05.608102+0000","last_fullsized":"2024-04-24T05:46:05.608102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248223+0000","last_change":"2024-04-24T05:46:04.274265+0000","last_active":"2024-04-24T05:46:05.248223+0000","last_peered":"2024-04-24T05:46:05.248223+0000","last_clean":"2024-04-24T05:46:05.248223+0000","last_became_active":"2024-04-24T05:46:04.273775+0000","last_became_peered":"2024-04-24T05:46:04.273775+0000","last_unstale":"2024-04-24T05:46:05.248223+0000","last_undegraded":"2024-04-24T05:46:05.248223+0000","last_fullsized":"2024-04-24T05:46:05.248223+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:49.092042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063970+0000","last_change":"2024-04-24T05:46:00.222245+0000","last_active":"2024-04-24T05:46:06.063970+0000","last_peered":"2024-04-24T05:46:06.063970+0000","last_clean":"2024-04-24T05:46:06.063970+0000","last_became_active":"2024-04-24T05:46:00.220540+0000","last_became_peered":"2024-04-24T05:46:00.220540+0000","last_unstale":"2024-04-24T05:46:06.063970+0000","last_undegraded":"2024-04-24T05:46:06.063970+0000","last_fullsized":"2024-04-24T05:46:06.063970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253552+0000","last_change":"2024-04-24T05:45:59.203808+0000","last_active":"2024-04-24T05:46:05.253552+0000","last_peered":"2024-04-24T05:46:05.253552+0000","last_clean":"2024-04-24T05:46:05.253552+0000","last_became_active":"2024-04-24T05:45:59.203023+0000","last_became_peered":"2024-04-24T05:45:59.203023+0000","last_unstale":"2024-04-24T05:46:05.253552+0000","last_undegraded":"2024-04-24T05:46:05.253552+0000","last_fullsized":"2024-04-24T05:46:05.253552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059474+0000","last_change":"2024-04-24T05:46:04.274610+0000","last_active":"2024-04-24T05:46:06.059474+0000","last_peered":"2024-04-24T05:46:06.059474+0000","last_clean":"2024-04-24T05:46:06.059474+0000","last_became_active":"2024-04-24T05:46:04.274246+0000","last_became_peered":"2024-04-24T05:46:04.274246+0000","last_unstale":"2024-04-24T05:46:06.059474+0000","last_undegraded":"2024-04-24T05:46:06.059474+0000","last_fullsized":"2024-04-24T05:46:06.059474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:31:17.536498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253481+0000","last_change":"2024-04-24T05:46:00.207959+0000","last_active":"2024-04-24T05:46:05.253481+0000","last_peered":"2024-04-24T05:46:05.253481+0000","last_clean":"2024-04-24T05:46:05.253481+0000","last_became_active":"2024-04-24T05:46:00.205225+0000","last_became_peered":"2024-04-24T05:46:00.205225+0000","last_unstale":"2024-04-24T05:46:05.253481+0000","last_undegraded":"2024-04-24T05:46:05.253481+0000","last_fullsized":"2024-04-24T05:46:05.253481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:55:08.484471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059232+0000","last_change":"2024-04-24T05:45:59.212223+0000","last_active":"2024-04-24T05:46:06.059232+0000","last_peered":"2024-04-24T05:46:06.059232+0000","last_clean":"2024-04-24T05:46:06.059232+0000","last_became_active":"2024-04-24T05:45:59.211908+0000","last_became_peered":"2024-04-24T05:45:59.211908+0000","last_unstale":"2024-04-24T05:46:06.059232+0000","last_undegraded":"2024-04-24T05:46:06.059232+0000","last_fullsized":"2024-04-24T05:46:06.059232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060243+0000","last_change":"2024-04-24T05:46:04.529148+0000","last_active":"2024-04-24T05:46:06.060243+0000","last_peered":"2024-04-24T05:46:06.060243+0000","last_clean":"2024-04-24T05:46:06.060243+0000","last_became_active":"2024-04-24T05:46:04.524775+0000","last_became_peered":"2024-04-24T05:46:04.524775+0000","last_unstale":"2024-04-24T05:46:06.060243+0000","last_undegraded":"2024-04-24T05:46:06.060243+0000","last_fullsized":"2024-04-24T05:46:06.060243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:58:44.607716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063332+0000","last_change":"2024-04-24T05:46:04.524596+0000","last_active":"2024-04-24T05:46:06.063332+0000","last_peered":"2024-04-24T05:46:06.063332+0000","last_clean":"2024-04-24T05:46:06.063332+0000","last_became_active":"2024-04-24T05:46:04.524013+0000","last_became_peered":"2024-04-24T05:46:04.524013+0000","last_unstale":"2024-04-24T05:46:06.063332+0000","last_undegraded":"2024-04-24T05:46:06.063332+0000","last_fullsized":"2024-04-24T05:46:06.063332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:45.991179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606337+0000","last_change":"2024-04-24T05:46:00.222945+0000","last_active":"2024-04-24T05:46:05.606337+0000","last_peered":"2024-04-24T05:46:05.606337+0000","last_clean":"2024-04-24T05:46:05.606337+0000","last_became_active":"2024-04-24T05:46:00.222698+0000","last_became_peered":"2024-04-24T05:46:00.222698+0000","last_unstale":"2024-04-24T05:46:05.606337+0000","last_undegraded":"2024-04-24T05:46:05.606337+0000","last_fullsized":"2024-04-24T05:46:05.606337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251487+0000","last_change":"2024-04-24T05:45:59.211291+0000","last_active":"2024-04-24T05:46:05.251487+0000","last_peered":"2024-04-24T05:46:05.251487+0000","last_clean":"2024-04-24T05:46:05.251487+0000","last_became_active":"2024-04-24T05:45:59.209448+0000","last_became_peered":"2024-04-24T05:45:59.209448+0000","last_unstale":"2024-04-24T05:46:05.251487+0000","last_undegraded":"2024-04-24T05:46:05.251487+0000","last_fullsized":"2024-04-24T05:46:05.251487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058917+0000","last_change":"2024-04-24T05:46:04.274043+0000","last_active":"2024-04-24T05:46:06.058917+0000","last_peered":"2024-04-24T05:46:06.058917+0000","last_clean":"2024-04-24T05:46:06.058917+0000","last_became_active":"2024-04-24T05:46:04.273504+0000","last_became_peered":"2024-04-24T05:46:04.273504+0000","last_unstale":"2024-04-24T05:46:06.058917+0000","last_undegraded":"2024-04-24T05:46:06.058917+0000","last_fullsized":"2024-04-24T05:46:06.058917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:55:12.536973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059394+0000","last_change":"2024-04-24T05:46:00.216386+0000","last_active":"2024-04-24T05:46:06.059394+0000","last_peered":"2024-04-24T05:46:06.059394+0000","last_clean":"2024-04-24T05:46:06.059394+0000","last_became_active":"2024-04-24T05:46:00.216078+0000","last_became_peered":"2024-04-24T05:46:00.216078+0000","last_unstale":"2024-04-24T05:46:06.059394+0000","last_undegraded":"2024-04-24T05:46:06.059394+0000","last_fullsized":"2024-04-24T05:46:06.059394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060995+0000","last_change":"2024-04-24T05:45:59.206266+0000","last_active":"2024-04-24T05:46:06.060995+0000","last_peered":"2024-04-24T05:46:06.060995+0000","last_clean":"2024-04-24T05:46:06.060995+0000","last_became_active":"2024-04-24T05:45:59.205881+0000","last_became_peered":"2024-04-24T05:45:59.205881+0000","last_unstale":"2024-04-24T05:46:06.060995+0000","last_undegraded":"2024-04-24T05:46:06.060995+0000","last_fullsized":"2024-04-24T05:46:06.060995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:53.435411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058603+0000","last_change":"2024-04-24T05:45:59.217059+0000","last_active":"2024-04-24T05:46:06.058603+0000","last_peered":"2024-04-24T05:46:06.058603+0000","last_clean":"2024-04-24T05:46:06.058603+0000","last_became_active":"2024-04-24T05:45:59.216742+0000","last_became_peered":"2024-04-24T05:45:59.216742+0000","last_unstale":"2024-04-24T05:46:06.058603+0000","last_undegraded":"2024-04-24T05:46:06.058603+0000","last_fullsized":"2024-04-24T05:46:06.058603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062704+0000","last_change":"2024-04-24T05:46:00.215564+0000","last_active":"2024-04-24T05:46:06.062704+0000","last_peered":"2024-04-24T05:46:06.062704+0000","last_clean":"2024-04-24T05:46:06.062704+0000","last_became_active":"2024-04-24T05:46:00.214442+0000","last_became_peered":"2024-04-24T05:46:00.214442+0000","last_unstale":"2024-04-24T05:46:06.062704+0000","last_undegraded":"2024-04-24T05:46:06.062704+0000","last_fullsized":"2024-04-24T05:46:06.062704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059540+0000","last_change":"2024-04-24T05:46:04.272081+0000","last_active":"2024-04-24T05:46:06.059540+0000","last_peered":"2024-04-24T05:46:06.059540+0000","last_clean":"2024-04-24T05:46:06.059540+0000","last_became_active":"2024-04-24T05:46:04.269391+0000","last_became_peered":"2024-04-24T05:46:04.269391+0000","last_unstale":"2024-04-24T05:46:06.059540+0000","last_undegraded":"2024-04-24T05:46:06.059540+0000","last_fullsized":"2024-04-24T05:46:06.059540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:24.966843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061465+0000","last_change":"2024-04-24T05:45:59.210758+0000","last_active":"2024-04-24T05:46:06.061465+0000","last_peered":"2024-04-24T05:46:06.061465+0000","last_clean":"2024-04-24T05:46:06.061465+0000","last_became_active":"2024-04-24T05:45:59.210533+0000","last_became_peered":"2024-04-24T05:45:59.210533+0000","last_unstale":"2024-04-24T05:46:06.061465+0000","last_undegraded":"2024-04-24T05:46:06.061465+0000","last_fullsized":"2024-04-24T05:46:06.061465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:11.561126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.247851+0000","last_change":"2024-04-24T05:46:00.211972+0000","last_active":"2024-04-24T05:46:05.247851+0000","last_peered":"2024-04-24T05:46:05.247851+0000","last_clean":"2024-04-24T05:46:05.247851+0000","last_became_active":"2024-04-24T05:46:00.210764+0000","last_became_peered":"2024-04-24T05:46:00.210764+0000","last_unstale":"2024-04-24T05:46:05.247851+0000","last_undegraded":"2024-04-24T05:46:05.247851+0000","last_fullsized":"2024-04-24T05:46:05.247851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:32.173672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062871+0000","last_change":"2024-04-24T05:46:04.271064+0000","last_active":"2024-04-24T05:46:06.062871+0000","last_peered":"2024-04-24T05:46:06.062871+0000","last_clean":"2024-04-24T05:46:06.062871+0000","last_became_active":"2024-04-24T05:46:04.270662+0000","last_became_peered":"2024-04-24T05:46:04.270662+0000","last_unstale":"2024-04-24T05:46:06.062871+0000","last_undegraded":"2024-04-24T05:46:06.062871+0000","last_fullsized":"2024-04-24T05:46:06.062871+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:18.795481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058290+0000","last_change":"2024-04-24T05:46:00.221213+0000","last_active":"2024-04-24T05:46:06.058290+0000","last_peered":"2024-04-24T05:46:06.058290+0000","last_clean":"2024-04-24T05:46:06.058290+0000","last_became_active":"2024-04-24T05:46:00.220647+0000","last_became_peered":"2024-04-24T05:46:00.220647+0000","last_unstale":"2024-04-24T05:46:06.058290+0000","last_undegraded":"2024-04-24T05:46:06.058290+0000","last_fullsized":"2024-04-24T05:46:06.058290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058501+0000","last_change":"2024-04-24T05:46:00.199902+0000","last_active":"2024-04-24T05:46:06.058501+0000","last_peered":"2024-04-24T05:46:06.058501+0000","last_clean":"2024-04-24T05:46:06.058501+0000","last_became_active":"2024-04-24T05:46:00.199314+0000","last_became_peered":"2024-04-24T05:46:00.199314+0000","last_unstale":"2024-04-24T05:46:06.058501+0000","last_undegraded":"2024-04-24T05:46:06.058501+0000","last_fullsized":"2024-04-24T05:46:06.058501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:27.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251956+0000","last_change":"2024-04-24T05:46:00.212233+0000","last_active":"2024-04-24T05:46:05.251956+0000","last_peered":"2024-04-24T05:46:05.251956+0000","last_clean":"2024-04-24T05:46:05.251956+0000","last_became_active":"2024-04-24T05:46:00.209914+0000","last_became_peered":"2024-04-24T05:46:00.209914+0000","last_unstale":"2024-04-24T05:46:05.251956+0000","last_undegraded":"2024-04-24T05:46:05.251956+0000","last_fullsized":"2024-04-24T05:46:05.251956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:45.039541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062577+0000","last_change":"2024-04-24T05:46:00.216833+0000","last_active":"2024-04-24T05:46:06.062577+0000","last_peered":"2024-04-24T05:46:06.062577+0000","last_clean":"2024-04-24T05:46:06.062577+0000","last_became_active":"2024-04-24T05:46:00.216494+0000","last_became_peered":"2024-04-24T05:46:00.216494+0000","last_unstale":"2024-04-24T05:46:06.062577+0000","last_undegraded":"2024-04-24T05:46:06.062577+0000","last_fullsized":"2024-04-24T05:46:06.062577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062178+0000","last_change":"2024-04-24T05:46:00.213642+0000","last_active":"2024-04-24T05:46:06.062178+0000","last_peered":"2024-04-24T05:46:06.062178+0000","last_clean":"2024-04-24T05:46:06.062178+0000","last_became_active":"2024-04-24T05:46:00.210164+0000","last_became_peered":"2024-04-24T05:46:00.210164+0000","last_unstale":"2024-04-24T05:46:06.062178+0000","last_undegraded":"2024-04-24T05:46:06.062178+0000","last_fullsized":"2024-04-24T05:46:06.062178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:41.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607788+0000","last_change":"2024-04-24T05:46:00.210082+0000","last_active":"2024-04-24T05:46:05.607788+0000","last_peered":"2024-04-24T05:46:05.607788+0000","last_clean":"2024-04-24T05:46:05.607788+0000","last_became_active":"2024-04-24T05:46:00.209799+0000","last_became_peered":"2024-04-24T05:46:00.209799+0000","last_unstale":"2024-04-24T05:46:05.607788+0000","last_undegraded":"2024-04-24T05:46:05.607788+0000","last_fullsized":"2024-04-24T05:46:05.607788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059611+0000","last_change":"2024-04-24T05:46:00.219304+0000","last_active":"2024-04-24T05:46:06.059611+0000","last_peered":"2024-04-24T05:46:06.059611+0000","last_clean":"2024-04-24T05:46:06.059611+0000","last_became_active":"2024-04-24T05:46:00.218953+0000","last_became_peered":"2024-04-24T05:46:00.218953+0000","last_unstale":"2024-04-24T05:46:06.059611+0000","last_undegraded":"2024-04-24T05:46:06.059611+0000","last_fullsized":"2024-04-24T05:46:06.059611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:41.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063234+0000","last_change":"2024-04-24T05:46:00.217971+0000","last_active":"2024-04-24T05:46:06.063234+0000","last_peered":"2024-04-24T05:46:06.063234+0000","last_clean":"2024-04-24T05:46:06.063234+0000","last_became_active":"2024-04-24T05:46:00.217789+0000","last_became_peered":"2024-04-24T05:46:00.217789+0000","last_unstale":"2024-04-24T05:46:06.063234+0000","last_undegraded":"2024-04-24T05:46:06.063234+0000","last_fullsized":"2024-04-24T05:46:06.063234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:38.107685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251295+0000","last_change":"2024-04-24T05:46:00.212887+0000","last_active":"2024-04-24T05:46:05.251295+0000","last_peered":"2024-04-24T05:46:05.251295+0000","last_clean":"2024-04-24T05:46:05.251295+0000","last_became_active":"2024-04-24T05:46:00.212550+0000","last_became_peered":"2024-04-24T05:46:00.212550+0000","last_unstale":"2024-04-24T05:46:05.251295+0000","last_undegraded":"2024-04-24T05:46:05.251295+0000","last_fullsized":"2024-04-24T05:46:05.251295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:25.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060618+0000","last_change":"2024-04-24T05:46:00.219721+0000","last_active":"2024-04-24T05:46:06.060618+0000","last_peered":"2024-04-24T05:46:06.060618+0000","last_clean":"2024-04-24T05:46:06.060618+0000","last_became_active":"2024-04-24T05:46:00.219429+0000","last_became_peered":"2024-04-24T05:46:00.219429+0000","last_unstale":"2024-04-24T05:46:06.060618+0000","last_undegraded":"2024-04-24T05:46:06.060618+0000","last_fullsized":"2024-04-24T05:46:06.060618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:27.080921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251686+0000","last_change":"2024-04-24T05:46:00.215082+0000","last_active":"2024-04-24T05:46:05.251686+0000","last_peered":"2024-04-24T05:46:05.251686+0000","last_clean":"2024-04-24T05:46:05.251686+0000","last_became_active":"2024-04-24T05:46:00.214719+0000","last_became_peered":"2024-04-24T05:46:00.214719+0000","last_unstale":"2024-04-24T05:46:05.251686+0000","last_undegraded":"2024-04-24T05:46:05.251686+0000","last_fullsized":"2024-04-24T05:46:05.251686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:42.688050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061826+0000","last_change":"2024-04-24T05:46:00.204113+0000","last_active":"2024-04-24T05:46:06.061826+0000","last_peered":"2024-04-24T05:46:06.061826+0000","last_clean":"2024-04-24T05:46:06.061826+0000","last_became_active":"2024-04-24T05:46:00.203764+0000","last_became_peered":"2024-04-24T05:46:00.203764+0000","last_unstale":"2024-04-24T05:46:06.061826+0000","last_undegraded":"2024-04-24T05:46:06.061826+0000","last_fullsized":"2024-04-24T05:46:06.061826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059326+0000","last_change":"2024-04-24T05:46:00.223248+0000","last_active":"2024-04-24T05:46:06.059326+0000","last_peered":"2024-04-24T05:46:06.059326+0000","last_clean":"2024-04-24T05:46:06.059326+0000","last_became_active":"2024-04-24T05:46:00.222366+0000","last_became_peered":"2024-04-24T05:46:00.222366+0000","last_unstale":"2024-04-24T05:46:06.059326+0000","last_undegraded":"2024-04-24T05:46:06.059326+0000","last_fullsized":"2024-04-24T05:46:06.059326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558036+0000","last_change":"2024-04-24T05:46:00.217085+0000","last_active":"2024-04-24T05:46:05.558036+0000","last_peered":"2024-04-24T05:46:05.558036+0000","last_clean":"2024-04-24T05:46:05.558036+0000","last_became_active":"2024-04-24T05:46:00.216799+0000","last_became_peered":"2024-04-24T05:46:00.216799+0000","last_unstale":"2024-04-24T05:46:05.558036+0000","last_undegraded":"2024-04-24T05:46:05.558036+0000","last_fullsized":"2024-04-24T05:46:05.558036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:27.050101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059180+0000","last_change":"2024-04-24T05:46:00.221238+0000","last_active":"2024-04-24T05:46:06.059180+0000","last_peered":"2024-04-24T05:46:06.059180+0000","last_clean":"2024-04-24T05:46:06.059180+0000","last_became_active":"2024-04-24T05:46:00.220945+0000","last_became_peered":"2024-04-24T05:46:00.220945+0000","last_unstale":"2024-04-24T05:46:06.059180+0000","last_undegraded":"2024-04-24T05:46:06.059180+0000","last_fullsized":"2024-04-24T05:46:06.059180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:21:57.974456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.254113+0000","last_change":"2024-04-24T05:46:00.206311+0000","last_active":"2024-04-24T05:46:05.254113+0000","last_peered":"2024-04-24T05:46:05.254113+0000","last_clean":"2024-04-24T05:46:05.254113+0000","last_became_active":"2024-04-24T05:46:00.205935+0000","last_became_peered":"2024-04-24T05:46:00.205935+0000","last_unstale":"2024-04-24T05:46:05.254113+0000","last_undegraded":"2024-04-24T05:46:05.254113+0000","last_fullsized":"2024-04-24T05:46:05.254113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:09.200350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251128+0000","last_change":"2024-04-24T05:46:00.202613+0000","last_active":"2024-04-24T05:46:05.251128+0000","last_peered":"2024-04-24T05:46:05.251128+0000","last_clean":"2024-04-24T05:46:05.251128+0000","last_became_active":"2024-04-24T05:46:00.202286+0000","last_became_peered":"2024-04-24T05:46:00.202286+0000","last_unstale":"2024-04-24T05:46:05.251128+0000","last_undegraded":"2024-04-24T05:46:05.251128+0000","last_fullsized":"2024-04-24T05:46:05.251128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606554+0000","last_change":"2024-04-24T05:46:00.222335+0000","last_active":"2024-04-24T05:46:05.606554+0000","last_peered":"2024-04-24T05:46:05.606554+0000","last_clean":"2024-04-24T05:46:05.606554+0000","last_became_active":"2024-04-24T05:46:00.222066+0000","last_became_peered":"2024-04-24T05:46:00.222066+0000","last_unstale":"2024-04-24T05:46:05.606554+0000","last_undegraded":"2024-04-24T05:46:05.606554+0000","last_fullsized":"2024-04-24T05:46:05.606554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060597+0000","last_change":"2024-04-24T05:46:00.213735+0000","last_active":"2024-04-24T05:46:06.060597+0000","last_peered":"2024-04-24T05:46:06.060597+0000","last_clean":"2024-04-24T05:46:06.060597+0000","last_became_active":"2024-04-24T05:46:00.213390+0000","last_became_peered":"2024-04-24T05:46:00.213390+0000","last_unstale":"2024-04-24T05:46:06.060597+0000","last_undegraded":"2024-04-24T05:46:06.060597+0000","last_fullsized":"2024-04-24T05:46:06.060597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:20.914341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061232+0000","last_change":"2024-04-24T05:46:00.219073+0000","last_active":"2024-04-24T05:46:06.061232+0000","last_peered":"2024-04-24T05:46:06.061232+0000","last_clean":"2024-04-24T05:46:06.061232+0000","last_became_active":"2024-04-24T05:46:00.218577+0000","last_became_peered":"2024-04-24T05:46:00.218577+0000","last_unstale":"2024-04-24T05:46:06.061232+0000","last_undegraded":"2024-04-24T05:46:06.061232+0000","last_fullsized":"2024-04-24T05:46:06.061232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248861+0000","last_change":"2024-04-24T05:46:00.219097+0000","last_active":"2024-04-24T05:46:05.248861+0000","last_peered":"2024-04-24T05:46:05.248861+0000","last_clean":"2024-04-24T05:46:05.248861+0000","last_became_active":"2024-04-24T05:46:00.218772+0000","last_became_peered":"2024-04-24T05:46:00.218772+0000","last_unstale":"2024-04-24T05:46:05.248861+0000","last_undegraded":"2024-04-24T05:46:05.248861+0000","last_fullsized":"2024-04-24T05:46:05.248861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:35.088347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059066+0000","last_change":"2024-04-24T05:46:00.221300+0000","last_active":"2024-04-24T05:46:06.059066+0000","last_peered":"2024-04-24T05:46:06.059066+0000","last_clean":"2024-04-24T05:46:06.059066+0000","last_became_active":"2024-04-24T05:46:00.220921+0000","last_became_peered":"2024-04-24T05:46:00.220921+0000","last_unstale":"2024-04-24T05:46:06.059066+0000","last_undegraded":"2024-04-24T05:46:06.059066+0000","last_fullsized":"2024-04-24T05:46:06.059066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059208+0000","last_change":"2024-04-24T05:46:00.221735+0000","last_active":"2024-04-24T05:46:06.059208+0000","last_peered":"2024-04-24T05:46:06.059208+0000","last_clean":"2024-04-24T05:46:06.059208+0000","last_became_active":"2024-04-24T05:46:00.220859+0000","last_became_peered":"2024-04-24T05:46:00.220859+0000","last_unstale":"2024-04-24T05:46:06.059208+0000","last_undegraded":"2024-04-24T05:46:06.059208+0000","last_fullsized":"2024-04-24T05:46:06.059208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:25.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558492+0000","last_change":"2024-04-24T05:46:00.227925+0000","last_active":"2024-04-24T05:46:05.558492+0000","last_peered":"2024-04-24T05:46:05.558492+0000","last_clean":"2024-04-24T05:46:05.558492+0000","last_became_active":"2024-04-24T05:46:00.227553+0000","last_became_peered":"2024-04-24T05:46:00.227553+0000","last_unstale":"2024-04-24T05:46:05.558492+0000","last_undegraded":"2024-04-24T05:46:05.558492+0000","last_fullsized":"2024-04-24T05:46:05.558492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:09.200350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058874+0000","last_change":"2024-04-24T05:46:00.225244+0000","last_active":"2024-04-24T05:46:06.058874+0000","last_peered":"2024-04-24T05:46:06.058874+0000","last_clean":"2024-04-24T05:46:06.058874+0000","last_became_active":"2024-04-24T05:46:00.224935+0000","last_became_peered":"2024-04-24T05:46:00.224935+0000","last_unstale":"2024-04-24T05:46:06.058874+0000","last_undegraded":"2024-04-24T05:46:06.058874+0000","last_fullsized":"2024-04-24T05:46:06.058874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557220+0000","last_change":"2024-04-24T05:46:00.225238+0000","last_active":"2024-04-24T05:46:05.557220+0000","last_peered":"2024-04-24T05:46:05.557220+0000","last_clean":"2024-04-24T05:46:05.557220+0000","last_became_active":"2024-04-24T05:46:00.224903+0000","last_became_peered":"2024-04-24T05:46:00.224903+0000","last_unstale":"2024-04-24T05:46:05.557220+0000","last_undegraded":"2024-04-24T05:46:05.557220+0000","last_fullsized":"2024-04-24T05:46:05.557220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:42.688050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251674+0000","last_change":"2024-04-24T05:46:00.206725+0000","last_active":"2024-04-24T05:46:05.251674+0000","last_peered":"2024-04-24T05:46:05.251674+0000","last_clean":"2024-04-24T05:46:05.251674+0000","last_became_active":"2024-04-24T05:46:00.206266+0000","last_became_peered":"2024-04-24T05:46:00.206266+0000","last_unstale":"2024-04-24T05:46:05.251674+0000","last_undegraded":"2024-04-24T05:46:05.251674+0000","last_fullsized":"2024-04-24T05:46:05.251674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:41.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557088+0000","last_change":"2024-04-24T05:46:00.219593+0000","last_active":"2024-04-24T05:46:05.557088+0000","last_peered":"2024-04-24T05:46:05.557088+0000","last_clean":"2024-04-24T05:46:05.557088+0000","last_became_active":"2024-04-24T05:46:00.219250+0000","last_became_peered":"2024-04-24T05:46:00.219250+0000","last_unstale":"2024-04-24T05:46:05.557088+0000","last_undegraded":"2024-04-24T05:46:05.557088+0000","last_fullsized":"2024-04-24T05:46:05.557088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:45.039541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252270+0000","last_change":"2024-04-24T05:46:00.213476+0000","last_active":"2024-04-24T05:46:05.252270+0000","last_peered":"2024-04-24T05:46:05.252270+0000","last_clean":"2024-04-24T05:46:05.252270+0000","last_became_active":"2024-04-24T05:46:00.213065+0000","last_became_peered":"2024-04-24T05:46:00.213065+0000","last_unstale":"2024-04-24T05:46:05.252270+0000","last_undegraded":"2024-04-24T05:46:05.252270+0000","last_fullsized":"2024-04-24T05:46:05.252270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:38.107685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063065+0000","last_change":"2024-04-24T05:46:00.221236+0000","last_active":"2024-04-24T05:46:06.063065+0000","last_peered":"2024-04-24T05:46:06.063065+0000","last_clean":"2024-04-24T05:46:06.063065+0000","last_became_active":"2024-04-24T05:46:00.220917+0000","last_became_peered":"2024-04-24T05:46:00.220917+0000","last_unstale":"2024-04-24T05:46:06.063065+0000","last_undegraded":"2024-04-24T05:46:06.063065+0000","last_fullsized":"2024-04-24T05:46:06.063065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:38.445039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608230+0000","last_change":"2024-04-24T05:46:00.219494+0000","last_active":"2024-04-24T05:46:05.608230+0000","last_peered":"2024-04-24T05:46:05.608230+0000","last_clean":"2024-04-24T05:46:05.608230+0000","last_became_active":"2024-04-24T05:46:00.219219+0000","last_became_peered":"2024-04-24T05:46:00.219219+0000","last_unstale":"2024-04-24T05:46:05.608230+0000","last_undegraded":"2024-04-24T05:46:05.608230+0000","last_fullsized":"2024-04-24T05:46:05.608230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558194+0000","last_change":"2024-04-24T05:46:00.221425+0000","last_active":"2024-04-24T05:46:05.558194+0000","last_peered":"2024-04-24T05:46:05.558194+0000","last_clean":"2024-04-24T05:46:05.558194+0000","last_became_active":"2024-04-24T05:46:00.219794+0000","last_became_peered":"2024-04-24T05:46:00.219794+0000","last_unstale":"2024-04-24T05:46:05.558194+0000","last_undegraded":"2024-04-24T05:46:05.558194+0000","last_fullsized":"2024-04-24T05:46:05.558194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:55:08.484471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":696263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":109155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":105439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":109155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":105313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":105473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":696319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T05:46:27.186 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T05:46:27.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.417+0000 7f2b4b200640 1 Processor -- start 2024-04-24T05:46:27.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.417+0000 7f2b4b200640 1 -- start start 2024-04-24T05:46:27.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.419+0000 7f2b4b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2b4c15b5c0 0x7f2b4c0cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:27.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.419+0000 7f2b4b200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2b4c06dbe0 0x7f2b4c06e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:27.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.419+0000 7f2b4b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c0acf40 0x7f2b4c0ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.419+0000 7f2b4b200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2b4c06e580 con 0x7f2b4c06dbe0 2024-04-24T05:46:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.419+0000 7f2b4b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2b4c0ce4b0 con 0x7f2b4c15b5c0 2024-04-24T05:46:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.419+0000 7f2b4b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4c0b3000 con 0x7f2b4c0acf40 2024-04-24T05:46:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.419+0000 7f2b49400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2b4c06dbe0 0x7f2b4c06e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.419+0000 7f2b49400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2b4c06dbe0 0x7f2b4c06e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56840/0 (socket says 172.21.15.123:56840) 2024-04-24T05:46:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.419+0000 7f2b49400640 1 -- 172.21.15.123:0/1687111337 learned_addr learned my addr 172.21.15.123:0/1687111337 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.420+0000 7f2b49e00640 1 --2- 172.21.15.123:0/1687111337 >> v2:172.21.15.155:3300/0 conn(0x7f2b4c15b5c0 0x7f2b4c0cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.420+0000 7f2b4a800640 1 --2- 172.21.15.123:0/1687111337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c0acf40 0x7f2b4c0ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.420+0000 7f2b49400640 1 -- 172.21.15.123:0/1687111337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c0acf40 msgr2=0x7f2b4c0ad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.420+0000 7f2b49400640 1 --2- 172.21.15.123:0/1687111337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c0acf40 0x7f2b4c0ad3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.420+0000 7f2b49400640 1 -- 172.21.15.123:0/1687111337 >> v2:172.21.15.155:3300/0 conn(0x7f2b4c15b5c0 msgr2=0x7f2b4c0cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:27.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.420+0000 7f2b49400640 1 --2- 172.21.15.123:0/1687111337 >> v2:172.21.15.155:3300/0 conn(0x7f2b4c15b5c0 0x7f2b4c0cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.420+0000 7f2b49400640 1 -- 172.21.15.123:0/1687111337 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c06d0c0 con 0x7f2b4c06dbe0 2024-04-24T05:46:27.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.420+0000 7f2b4a800640 1 --2- 172.21.15.123:0/1687111337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c0acf40 0x7f2b4c0ad3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:27.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.421+0000 7f2b49400640 1 --2- 172.21.15.123:0/1687111337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b4c06dbe0 0x7f2b4c06e040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c050650 tx=0x7f2b3802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52d32ed6022b620 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:27.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.426+0000 7f2b48a00640 1 -- 172.21.15.123:0/1687111337 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38037070 con 0x7f2b4c06dbe0 2024-04-24T05:46:27.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.426+0000 7f2b48a00640 1 -- 172.21.15.123:0/1687111337 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b38032440 con 0x7f2b4c06dbe0 2024-04-24T05:46:27.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.426+0000 7f2b48a00640 1 -- 172.21.15.123:0/1687111337 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38032810 con 0x7f2b4c06dbe0 2024-04-24T05:46:27.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.427+0000 7f2b4b200640 1 -- 172.21.15.123:0/1687111337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b4c06dbe0 msgr2=0x7f2b4c06e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:27.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.427+0000 7f2b4b200640 1 --2- 172.21.15.123:0/1687111337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b4c06dbe0 0x7f2b4c06e040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c050650 tx=0x7f2b3802a1d0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.427+0000 7f2b4b200640 1 -- 172.21.15.123:0/1687111337 shutdown_connections 2024-04-24T05:46:27.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.427+0000 7f2b4b200640 1 --2- 172.21.15.123:0/1687111337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c0acf40 0x7f2b4c0ad3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.427+0000 7f2b4b200640 1 --2- 172.21.15.123:0/1687111337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b4c06dbe0 0x7f2b4c06e040 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.427+0000 7f2b4b200640 1 --2- 172.21.15.123:0/1687111337 >> v2:172.21.15.155:3300/0 conn(0x7f2b4c15b5c0 0x7f2b4c0cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.427+0000 7f2b4b200640 1 -- 172.21.15.123:0/1687111337 >> 172.21.15.123:0/1687111337 conn(0x7f2b4c0d0070 msgr2=0x7f2b4c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:27.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.427+0000 7f2b4b200640 1 -- 172.21.15.123:0/1687111337 shutdown_connections 2024-04-24T05:46:27.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.427+0000 7f2b4b200640 1 -- 172.21.15.123:0/1687111337 wait complete. 2024-04-24T05:46:27.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.429+0000 7f2b4b200640 1 Processor -- start 2024-04-24T05:46:27.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.429+0000 7f2b4b200640 1 -- start start 2024-04-24T05:46:27.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.431+0000 7f2b4b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06dbe0 0x7f2b4c14fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:27.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.431+0000 7f2b4b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c0acf40 0x7f2b4c09b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:27.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.431+0000 7f2b4b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b4c15b5c0 0x7f2b4c072bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:27.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.431+0000 7f2b4b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c06cd20 con 0x7f2b4c15b5c0 2024-04-24T05:46:27.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.431+0000 7f2b4b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c0ae510 con 0x7f2b4c0acf40 2024-04-24T05:46:27.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.431+0000 7f2b4b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4c06cea0 con 0x7f2b4c06dbe0 2024-04-24T05:46:27.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.431+0000 7f2b4a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b4c15b5c0 0x7f2b4c072bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:27.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.431+0000 7f2b4a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b4c15b5c0 0x7f2b4c072bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56844/0 (socket says 172.21.15.123:56844) 2024-04-24T05:46:27.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.431+0000 7f2b4a800640 1 -- 172.21.15.123:0/3853238015 learned_addr learned my addr 172.21.15.123:0/3853238015 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:27.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.431+0000 7f2b49e00640 1 --2- 172.21.15.123:0/3853238015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06dbe0 0x7f2b4c14fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:27.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.432+0000 7f2b4a800640 1 -- 172.21.15.123:0/3853238015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06dbe0 msgr2=0x7f2b4c14fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:27.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.432+0000 7f2b4a800640 1 --2- 172.21.15.123:0/3853238015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06dbe0 0x7f2b4c14fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.432+0000 7f2b4a800640 1 -- 172.21.15.123:0/3853238015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c0acf40 msgr2=0x7f2b4c09b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:27.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.432+0000 7f2b4a800640 1 --2- 172.21.15.123:0/3853238015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c0acf40 0x7f2b4c09b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.432+0000 7f2b4a800640 1 -- 172.21.15.123:0/3853238015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c09afb0 con 0x7f2b4c15b5c0 2024-04-24T05:46:27.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.432+0000 7f2b4a800640 1 --2- 172.21.15.123:0/3853238015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b4c15b5c0 0x7f2b4c072bd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400cef0 tx=0x7f2b4400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:27.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.432+0000 7f2b42000640 1 -- 172.21.15.123:0/3853238015 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4401a070 con 0x7f2b4c15b5c0 2024-04-24T05:46:27.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.432+0000 7f2b42000640 1 -- 172.21.15.123:0/3853238015 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b4400d860 con 0x7f2b4c15b5c0 2024-04-24T05:46:27.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.432+0000 7f2b4b200640 1 -- 172.21.15.123:0/3853238015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4c09d340 con 0x7f2b4c15b5c0 2024-04-24T05:46:27.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.432+0000 7f2b42000640 1 -- 172.21.15.123:0/3853238015 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4400dc50 con 0x7f2b4c15b5c0 2024-04-24T05:46:27.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.433+0000 7f2b4b200640 1 -- 172.21.15.123:0/3853238015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4c0c9e20 con 0x7f2b4c15b5c0 2024-04-24T05:46:27.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.434+0000 7f2b4b200640 1 -- 172.21.15.123:0/3853238015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b4c075b70 con 0x7f2b4c15b5c0 2024-04-24T05:46:27.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.434+0000 7f2b42000640 1 -- 172.21.15.123:0/3853238015 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b4401d070 con 0x7f2b4c15b5c0 2024-04-24T05:46:27.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.436+0000 7f2b42000640 1 --2- 172.21.15.123:0/3853238015 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2b2c06c8f0 0x7f2b2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:27.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.436+0000 7f2b42000640 1 -- 172.21.15.123:0/3853238015 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2b4408c6d0 con 0x7f2b4c15b5c0 2024-04-24T05:46:27.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.436+0000 7f2b49e00640 1 --2- 172.21.15.123:0/3853238015 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2b2c06c8f0 0x7f2b2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:27.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.437+0000 7f2b49e00640 1 --2- 172.21.15.123:0/3853238015 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2b2c06c8f0 0x7f2b2c06edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c006fd0 tx=0x7f2b3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:27.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.441+0000 7f2b42000640 1 -- 172.21.15.123:0/3853238015 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b4c09afb0 con 0x7f2b4c15b5c0 2024-04-24T05:46:27.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.623+0000 7f2b4b200640 1 -- 172.21.15.123:0/3853238015 --> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b4c06c6d0 con 0x7f2b2c06c8f0 2024-04-24T05:46:27.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.646+0000 7f2b42000640 1 -- 172.21.15.123:0/3853238015 <== mgr.4101 v2:172.21.15.123:6832/1503718323 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7f2b4c06c6d0 con 0x7f2b2c06c8f0 2024-04-24T05:46:27.651 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:27.654 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-24T05:46:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.653+0000 7f2b4b200640 1 -- 172.21.15.123:0/3853238015 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2b2c06c8f0 msgr2=0x7f2b2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.653+0000 7f2b4b200640 1 --2- 172.21.15.123:0/3853238015 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2b2c06c8f0 0x7f2b2c06edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c006fd0 tx=0x7f2b3c008040 comp rx=0 tx=0).stop 2024-04-24T05:46:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.653+0000 7f2b4b200640 1 -- 172.21.15.123:0/3853238015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b4c15b5c0 msgr2=0x7f2b4c072bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.653+0000 7f2b4b200640 1 --2- 172.21.15.123:0/3853238015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b4c15b5c0 0x7f2b4c072bd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400cef0 tx=0x7f2b4400a710 comp rx=0 tx=0).stop 2024-04-24T05:46:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.654+0000 7f2b4b200640 1 -- 172.21.15.123:0/3853238015 shutdown_connections 2024-04-24T05:46:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.654+0000 7f2b4b200640 1 --2- 172.21.15.123:0/3853238015 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f2b2c06c8f0 0x7f2b2c06edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.654+0000 7f2b4b200640 1 --2- 172.21.15.123:0/3853238015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b4c15b5c0 0x7f2b4c072bd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.654+0000 7f2b4b200640 1 --2- 172.21.15.123:0/3853238015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c0acf40 0x7f2b4c09b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.654+0000 7f2b4b200640 1 --2- 172.21.15.123:0/3853238015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b4c06dbe0 0x7f2b4c14fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.654+0000 7f2b4b200640 1 -- 172.21.15.123:0/3853238015 >> 172.21.15.123:0/3853238015 conn(0x7f2b4c0d0070 msgr2=0x7f2b4c0b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:27.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.654+0000 7f2b4b200640 1 -- 172.21.15.123:0/3853238015 shutdown_connections 2024-04-24T05:46:27.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.654+0000 7f2b4b200640 1 -- 172.21.15.123:0/3853238015 wait complete. 2024-04-24T05:46:27.672 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-24T05:46:26.523325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219476,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755244,"statfs":{"total":773094113280,"available":772869369856,"internally_reserved":0,"allocated":3383296,"data_stored":2032520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007124"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063974+0000","last_change":"2024-04-24T05:45:59.220218+0000","last_active":"2024-04-24T05:46:06.063974+0000","last_peered":"2024-04-24T05:46:06.063974+0000","last_clean":"2024-04-24T05:46:06.063974+0000","last_became_active":"2024-04-24T05:45:59.219888+0000","last_became_peered":"2024-04-24T05:45:59.219888+0000","last_unstale":"2024-04-24T05:46:06.063974+0000","last_undegraded":"2024-04-24T05:46:06.063974+0000","last_fullsized":"2024-04-24T05:46:06.063974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062489+0000","last_change":"2024-04-24T05:46:00.218964+0000","last_active":"2024-04-24T05:46:06.062489+0000","last_peered":"2024-04-24T05:46:06.062489+0000","last_clean":"2024-04-24T05:46:06.062489+0000","last_became_active":"2024-04-24T05:46:00.218589+0000","last_became_peered":"2024-04-24T05:46:00.218589+0000","last_unstale":"2024-04-24T05:46:06.062489+0000","last_undegraded":"2024-04-24T05:46:06.062489+0000","last_fullsized":"2024-04-24T05:46:06.062489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:42.688050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059820+0000","last_change":"2024-04-24T05:46:04.525402+0000","last_active":"2024-04-24T05:46:06.059820+0000","last_peered":"2024-04-24T05:46:06.059820+0000","last_clean":"2024-04-24T05:46:06.059820+0000","last_became_active":"2024-04-24T05:46:04.524690+0000","last_became_peered":"2024-04-24T05:46:04.524690+0000","last_unstale":"2024-04-24T05:46:06.059820+0000","last_undegraded":"2024-04-24T05:46:06.059820+0000","last_fullsized":"2024-04-24T05:46:06.059820+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:15:19.979319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062738+0000","last_change":"2024-04-24T05:45:59.203824+0000","last_active":"2024-04-24T05:46:06.062738+0000","last_peered":"2024-04-24T05:46:06.062738+0000","last_clean":"2024-04-24T05:46:06.062738+0000","last_became_active":"2024-04-24T05:45:59.203396+0000","last_became_peered":"2024-04-24T05:45:59.203396+0000","last_unstale":"2024-04-24T05:46:06.062738+0000","last_undegraded":"2024-04-24T05:46:06.062738+0000","last_fullsized":"2024-04-24T05:46:06.062738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:53:31.596793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063701+0000","last_change":"2024-04-24T05:46:00.221369+0000","last_active":"2024-04-24T05:46:06.063701+0000","last_peered":"2024-04-24T05:46:06.063701+0000","last_clean":"2024-04-24T05:46:06.063701+0000","last_became_active":"2024-04-24T05:46:00.220890+0000","last_became_peered":"2024-04-24T05:46:00.220890+0000","last_unstale":"2024-04-24T05:46:06.063701+0000","last_undegraded":"2024-04-24T05:46:06.063701+0000","last_fullsized":"2024-04-24T05:46:06.063701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:27.080921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.246725+0000","last_change":"2024-04-24T05:46:04.274351+0000","last_active":"2024-04-24T05:46:05.246725+0000","last_peered":"2024-04-24T05:46:05.246725+0000","last_clean":"2024-04-24T05:46:05.246725+0000","last_became_active":"2024-04-24T05:46:04.273892+0000","last_became_peered":"2024-04-24T05:46:04.273892+0000","last_unstale":"2024-04-24T05:46:05.246725+0000","last_undegraded":"2024-04-24T05:46:05.246725+0000","last_fullsized":"2024-04-24T05:46:05.246725+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:39.015857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557980+0000","last_change":"2024-04-24T05:45:59.221936+0000","last_active":"2024-04-24T05:46:05.557980+0000","last_peered":"2024-04-24T05:46:05.557980+0000","last_clean":"2024-04-24T05:46:05.557980+0000","last_became_active":"2024-04-24T05:45:59.221563+0000","last_became_peered":"2024-04-24T05:45:59.221563+0000","last_unstale":"2024-04-24T05:46:05.557980+0000","last_undegraded":"2024-04-24T05:46:05.557980+0000","last_fullsized":"2024-04-24T05:46:05.557980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:34.072620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252559+0000","last_change":"2024-04-24T05:46:00.204200+0000","last_active":"2024-04-24T05:46:05.252559+0000","last_peered":"2024-04-24T05:46:05.252559+0000","last_clean":"2024-04-24T05:46:05.252559+0000","last_became_active":"2024-04-24T05:46:00.203884+0000","last_became_peered":"2024-04-24T05:46:00.203884+0000","last_unstale":"2024-04-24T05:46:05.252559+0000","last_undegraded":"2024-04-24T05:46:05.252559+0000","last_fullsized":"2024-04-24T05:46:05.252559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:05:06.286589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248046+0000","last_change":"2024-04-24T05:46:04.279449+0000","last_active":"2024-04-24T05:46:05.248046+0000","last_peered":"2024-04-24T05:46:05.248046+0000","last_clean":"2024-04-24T05:46:05.248046+0000","last_became_active":"2024-04-24T05:46:04.278915+0000","last_became_peered":"2024-04-24T05:46:04.278915+0000","last_unstale":"2024-04-24T05:46:05.248046+0000","last_undegraded":"2024-04-24T05:46:05.248046+0000","last_fullsized":"2024-04-24T05:46:05.248046+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:42.160187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557287+0000","last_change":"2024-04-24T05:45:59.220243+0000","last_active":"2024-04-24T05:46:05.557287+0000","last_peered":"2024-04-24T05:46:05.557287+0000","last_clean":"2024-04-24T05:46:05.557287+0000","last_became_active":"2024-04-24T05:45:59.219288+0000","last_became_peered":"2024-04-24T05:45:59.219288+0000","last_unstale":"2024-04-24T05:46:05.557287+0000","last_undegraded":"2024-04-24T05:46:05.557287+0000","last_fullsized":"2024-04-24T05:46:05.557287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253052+0000","last_change":"2024-04-24T05:46:00.208516+0000","last_active":"2024-04-24T05:46:05.253052+0000","last_peered":"2024-04-24T05:46:05.253052+0000","last_clean":"2024-04-24T05:46:05.253052+0000","last_became_active":"2024-04-24T05:46:00.208217+0000","last_became_peered":"2024-04-24T05:46:00.208217+0000","last_unstale":"2024-04-24T05:46:05.253052+0000","last_undegraded":"2024-04-24T05:46:05.253052+0000","last_fullsized":"2024-04-24T05:46:05.253052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:13.907547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.247497+0000","last_change":"2024-04-24T05:46:04.274319+0000","last_active":"2024-04-24T05:46:05.247497+0000","last_peered":"2024-04-24T05:46:05.247497+0000","last_clean":"2024-04-24T05:46:05.247497+0000","last_became_active":"2024-04-24T05:46:04.273687+0000","last_became_peered":"2024-04-24T05:46:04.273687+0000","last_unstale":"2024-04-24T05:46:05.247497+0000","last_undegraded":"2024-04-24T05:46:05.247497+0000","last_fullsized":"2024-04-24T05:46:05.247497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:46.740551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058089+0000","last_change":"2024-04-24T05:46:00.222735+0000","last_active":"2024-04-24T05:46:06.058089+0000","last_peered":"2024-04-24T05:46:06.058089+0000","last_clean":"2024-04-24T05:46:06.058089+0000","last_became_active":"2024-04-24T05:46:00.222428+0000","last_became_peered":"2024-04-24T05:46:00.222428+0000","last_unstale":"2024-04-24T05:46:06.058089+0000","last_undegraded":"2024-04-24T05:46:06.058089+0000","last_fullsized":"2024-04-24T05:46:06.058089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557728+0000","last_change":"2024-04-24T05:45:59.220847+0000","last_active":"2024-04-24T05:46:05.557728+0000","last_peered":"2024-04-24T05:46:05.557728+0000","last_clean":"2024-04-24T05:46:05.557728+0000","last_became_active":"2024-04-24T05:45:59.220187+0000","last_became_peered":"2024-04-24T05:45:59.220187+0000","last_unstale":"2024-04-24T05:46:05.557728+0000","last_undegraded":"2024-04-24T05:46:05.557728+0000","last_fullsized":"2024-04-24T05:46:05.557728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:58:40.227841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.249739+0000","last_change":"2024-04-24T05:46:04.271803+0000","last_active":"2024-04-24T05:46:05.249739+0000","last_peered":"2024-04-24T05:46:05.249739+0000","last_clean":"2024-04-24T05:46:05.249739+0000","last_became_active":"2024-04-24T05:46:04.270863+0000","last_became_peered":"2024-04-24T05:46:04.270863+0000","last_unstale":"2024-04-24T05:46:05.249739+0000","last_undegraded":"2024-04-24T05:46:05.249739+0000","last_fullsized":"2024-04-24T05:46:05.249739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:34.349539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061349+0000","last_change":"2024-04-24T05:46:00.216572+0000","last_active":"2024-04-24T05:46:06.061349+0000","last_peered":"2024-04-24T05:46:06.061349+0000","last_clean":"2024-04-24T05:46:06.061349+0000","last_became_active":"2024-04-24T05:46:00.216226+0000","last_became_peered":"2024-04-24T05:46:00.216226+0000","last_unstale":"2024-04-24T05:46:06.061349+0000","last_undegraded":"2024-04-24T05:46:06.061349+0000","last_fullsized":"2024-04-24T05:46:06.061349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607418+0000","last_change":"2024-04-24T05:45:59.201895+0000","last_active":"2024-04-24T05:46:05.607418+0000","last_peered":"2024-04-24T05:46:05.607418+0000","last_clean":"2024-04-24T05:46:05.607418+0000","last_became_active":"2024-04-24T05:45:59.201064+0000","last_became_peered":"2024-04-24T05:45:59.201064+0000","last_unstale":"2024-04-24T05:46:05.607418+0000","last_undegraded":"2024-04-24T05:46:05.607418+0000","last_fullsized":"2024-04-24T05:46:05.607418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248655+0000","last_change":"2024-04-24T05:46:04.278089+0000","last_active":"2024-04-24T05:46:05.248655+0000","last_peered":"2024-04-24T05:46:05.248655+0000","last_clean":"2024-04-24T05:46:05.248655+0000","last_became_active":"2024-04-24T05:46:04.277685+0000","last_became_peered":"2024-04-24T05:46:04.277685+0000","last_unstale":"2024-04-24T05:46:05.248655+0000","last_undegraded":"2024-04-24T05:46:05.248655+0000","last_fullsized":"2024-04-24T05:46:05.248655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:05:10.339090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059639+0000","last_change":"2024-04-24T05:46:00.210339+0000","last_active":"2024-04-24T05:46:06.059639+0000","last_peered":"2024-04-24T05:46:06.059639+0000","last_clean":"2024-04-24T05:46:06.059639+0000","last_became_active":"2024-04-24T05:46:00.209637+0000","last_became_peered":"2024-04-24T05:46:00.209637+0000","last_unstale":"2024-04-24T05:46:06.059639+0000","last_undegraded":"2024-04-24T05:46:06.059639+0000","last_fullsized":"2024-04-24T05:46:06.059639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606639+0000","last_change":"2024-04-24T05:45:59.204528+0000","last_active":"2024-04-24T05:46:05.606639+0000","last_peered":"2024-04-24T05:46:05.606639+0000","last_clean":"2024-04-24T05:46:05.606639+0000","last_became_active":"2024-04-24T05:45:59.204200+0000","last_became_peered":"2024-04-24T05:45:59.204200+0000","last_unstale":"2024-04-24T05:46:05.606639+0000","last_undegraded":"2024-04-24T05:46:05.606639+0000","last_fullsized":"2024-04-24T05:46:05.606639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:26.453566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608562+0000","last_change":"2024-04-24T05:46:04.521949+0000","last_active":"2024-04-24T05:46:05.608562+0000","last_peered":"2024-04-24T05:46:05.608562+0000","last_clean":"2024-04-24T05:46:05.608562+0000","last_became_active":"2024-04-24T05:46:04.521423+0000","last_became_peered":"2024-04-24T05:46:04.521423+0000","last_unstale":"2024-04-24T05:46:05.608562+0000","last_undegraded":"2024-04-24T05:46:05.608562+0000","last_fullsized":"2024-04-24T05:46:05.608562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:44.521141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248230+0000","last_change":"2024-04-24T05:46:00.209888+0000","last_active":"2024-04-24T05:46:05.248230+0000","last_peered":"2024-04-24T05:46:05.248230+0000","last_clean":"2024-04-24T05:46:05.248230+0000","last_became_active":"2024-04-24T05:46:00.209568+0000","last_became_peered":"2024-04-24T05:46:00.209568+0000","last_unstale":"2024-04-24T05:46:05.248230+0000","last_undegraded":"2024-04-24T05:46:05.248230+0000","last_fullsized":"2024-04-24T05:46:05.248230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:21.844840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557732+0000","last_change":"2024-04-24T05:45:59.218584+0000","last_active":"2024-04-24T05:46:05.557732+0000","last_peered":"2024-04-24T05:46:05.557732+0000","last_clean":"2024-04-24T05:46:05.557732+0000","last_became_active":"2024-04-24T05:45:59.218316+0000","last_became_peered":"2024-04-24T05:45:59.218316+0000","last_unstale":"2024-04-24T05:46:05.557732+0000","last_undegraded":"2024-04-24T05:46:05.557732+0000","last_fullsized":"2024-04-24T05:46:05.557732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.249754+0000","last_change":"2024-04-24T05:46:04.281339+0000","last_active":"2024-04-24T05:46:05.249754+0000","last_peered":"2024-04-24T05:46:05.249754+0000","last_clean":"2024-04-24T05:46:05.249754+0000","last_became_active":"2024-04-24T05:46:04.280972+0000","last_became_peered":"2024-04-24T05:46:04.280972+0000","last_unstale":"2024-04-24T05:46:05.249754+0000","last_undegraded":"2024-04-24T05:46:05.249754+0000","last_fullsized":"2024-04-24T05:46:05.249754+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:17.960048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.064250+0000","last_change":"2024-04-24T05:46:00.210893+0000","last_active":"2024-04-24T05:46:06.064250+0000","last_peered":"2024-04-24T05:46:06.064250+0000","last_clean":"2024-04-24T05:46:06.064250+0000","last_became_active":"2024-04-24T05:46:00.210566+0000","last_became_peered":"2024-04-24T05:46:00.210566+0000","last_unstale":"2024-04-24T05:46:06.064250+0000","last_undegraded":"2024-04-24T05:46:06.064250+0000","last_fullsized":"2024-04-24T05:46:06.064250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059489+0000","last_change":"2024-04-24T05:45:59.213341+0000","last_active":"2024-04-24T05:46:06.059489+0000","last_peered":"2024-04-24T05:46:06.059489+0000","last_clean":"2024-04-24T05:46:06.059489+0000","last_became_active":"2024-04-24T05:45:59.212756+0000","last_became_peered":"2024-04-24T05:45:59.212756+0000","last_unstale":"2024-04-24T05:46:06.059489+0000","last_undegraded":"2024-04-24T05:46:06.059489+0000","last_fullsized":"2024-04-24T05:46:06.059489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:34.072620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.247506+0000","last_change":"2024-04-24T05:46:04.524925+0000","last_active":"2024-04-24T05:46:05.247506+0000","last_peered":"2024-04-24T05:46:05.247506+0000","last_clean":"2024-04-24T05:46:05.247506+0000","last_became_active":"2024-04-24T05:46:04.523828+0000","last_became_peered":"2024-04-24T05:46:04.523828+0000","last_unstale":"2024-04-24T05:46:05.247506+0000","last_undegraded":"2024-04-24T05:46:05.247506+0000","last_fullsized":"2024-04-24T05:46:05.247506+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:45.904568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063326+0000","last_change":"2024-04-24T05:46:00.227429+0000","last_active":"2024-04-24T05:46:06.063326+0000","last_peered":"2024-04-24T05:46:06.063326+0000","last_clean":"2024-04-24T05:46:06.063326+0000","last_became_active":"2024-04-24T05:46:00.226909+0000","last_became_peered":"2024-04-24T05:46:00.226909+0000","last_unstale":"2024-04-24T05:46:06.063326+0000","last_undegraded":"2024-04-24T05:46:06.063326+0000","last_fullsized":"2024-04-24T05:46:06.063326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:45.039541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608337+0000","last_change":"2024-04-24T05:45:59.203340+0000","last_active":"2024-04-24T05:46:05.608337+0000","last_peered":"2024-04-24T05:46:05.608337+0000","last_clean":"2024-04-24T05:46:05.608337+0000","last_became_active":"2024-04-24T05:45:59.203004+0000","last_became_peered":"2024-04-24T05:45:59.203004+0000","last_unstale":"2024-04-24T05:46:05.608337+0000","last_undegraded":"2024-04-24T05:46:05.608337+0000","last_fullsized":"2024-04-24T05:46:05.608337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:34.072620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059109+0000","last_change":"2024-04-24T05:46:04.528328+0000","last_active":"2024-04-24T05:46:06.059109+0000","last_peered":"2024-04-24T05:46:06.059109+0000","last_clean":"2024-04-24T05:46:06.059109+0000","last_became_active":"2024-04-24T05:46:04.527760+0000","last_became_peered":"2024-04-24T05:46:04.527760+0000","last_unstale":"2024-04-24T05:46:06.059109+0000","last_undegraded":"2024-04-24T05:46:06.059109+0000","last_fullsized":"2024-04-24T05:46:06.059109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:16:08.677684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607804+0000","last_change":"2024-04-24T05:46:00.215704+0000","last_active":"2024-04-24T05:46:05.607804+0000","last_peered":"2024-04-24T05:46:05.607804+0000","last_clean":"2024-04-24T05:46:05.607804+0000","last_became_active":"2024-04-24T05:46:00.214751+0000","last_became_peered":"2024-04-24T05:46:00.214751+0000","last_unstale":"2024-04-24T05:46:05.607804+0000","last_undegraded":"2024-04-24T05:46:05.607804+0000","last_fullsized":"2024-04-24T05:46:05.607804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558293+0000","last_change":"2024-04-24T05:45:59.219396+0000","last_active":"2024-04-24T05:46:05.558293+0000","last_peered":"2024-04-24T05:46:05.558293+0000","last_clean":"2024-04-24T05:46:05.558293+0000","last_became_active":"2024-04-24T05:45:59.217765+0000","last_became_peered":"2024-04-24T05:45:59.217765+0000","last_unstale":"2024-04-24T05:46:05.558293+0000","last_undegraded":"2024-04-24T05:46:05.558293+0000","last_fullsized":"2024-04-24T05:46:05.558293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:21:56.958729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061548+0000","last_change":"2024-04-24T05:46:04.518477+0000","last_active":"2024-04-24T05:46:06.061548+0000","last_peered":"2024-04-24T05:46:06.061548+0000","last_clean":"2024-04-24T05:46:06.061548+0000","last_became_active":"2024-04-24T05:46:04.515078+0000","last_became_peered":"2024-04-24T05:46:04.515078+0000","last_unstale":"2024-04-24T05:46:06.061548+0000","last_undegraded":"2024-04-24T05:46:06.061548+0000","last_fullsized":"2024-04-24T05:46:06.061548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:31.102602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253112+0000","last_change":"2024-04-24T05:46:00.212998+0000","last_active":"2024-04-24T05:46:05.253112+0000","last_peered":"2024-04-24T05:46:05.253112+0000","last_clean":"2024-04-24T05:46:05.253112+0000","last_became_active":"2024-04-24T05:46:00.212446+0000","last_became_peered":"2024-04-24T05:46:00.212446+0000","last_unstale":"2024-04-24T05:46:05.253112+0000","last_undegraded":"2024-04-24T05:46:05.253112+0000","last_fullsized":"2024-04-24T05:46:05.253112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:27.050101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607823+0000","last_change":"2024-04-24T05:45:59.197842+0000","last_active":"2024-04-24T05:46:05.607823+0000","last_peered":"2024-04-24T05:46:05.607823+0000","last_clean":"2024-04-24T05:46:05.607823+0000","last_became_active":"2024-04-24T05:45:59.197510+0000","last_became_peered":"2024-04-24T05:45:59.197510+0000","last_unstale":"2024-04-24T05:46:05.607823+0000","last_undegraded":"2024-04-24T05:46:05.607823+0000","last_fullsized":"2024-04-24T05:46:05.607823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606699+0000","last_change":"2024-04-24T05:46:04.270957+0000","last_active":"2024-04-24T05:46:05.606699+0000","last_peered":"2024-04-24T05:46:05.606699+0000","last_clean":"2024-04-24T05:46:05.606699+0000","last_became_active":"2024-04-24T05:46:04.268656+0000","last_became_peered":"2024-04-24T05:46:04.268656+0000","last_unstale":"2024-04-24T05:46:05.606699+0000","last_undegraded":"2024-04-24T05:46:05.606699+0000","last_fullsized":"2024-04-24T05:46:05.606699+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:31.102602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248701+0000","last_change":"2024-04-24T05:46:00.205128+0000","last_active":"2024-04-24T05:46:05.248701+0000","last_peered":"2024-04-24T05:46:05.248701+0000","last_clean":"2024-04-24T05:46:05.248701+0000","last_became_active":"2024-04-24T05:46:00.202044+0000","last_became_peered":"2024-04-24T05:46:00.202044+0000","last_unstale":"2024-04-24T05:46:05.248701+0000","last_undegraded":"2024-04-24T05:46:05.248701+0000","last_fullsized":"2024-04-24T05:46:05.248701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:18.772951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252226+0000","last_change":"2024-04-24T05:45:59.199320+0000","last_active":"2024-04-24T05:46:05.252226+0000","last_peered":"2024-04-24T05:46:05.252226+0000","last_clean":"2024-04-24T05:46:05.252226+0000","last_became_active":"2024-04-24T05:45:59.198382+0000","last_became_peered":"2024-04-24T05:45:59.198382+0000","last_unstale":"2024-04-24T05:46:05.252226+0000","last_undegraded":"2024-04-24T05:46:05.252226+0000","last_fullsized":"2024-04-24T05:46:05.252226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607713+0000","last_change":"2024-04-24T05:46:04.273947+0000","last_active":"2024-04-24T05:46:05.607713+0000","last_peered":"2024-04-24T05:46:05.607713+0000","last_clean":"2024-04-24T05:46:05.607713+0000","last_became_active":"2024-04-24T05:46:04.271583+0000","last_became_peered":"2024-04-24T05:46:04.271583+0000","last_unstale":"2024-04-24T05:46:05.607713+0000","last_undegraded":"2024-04-24T05:46:05.607713+0000","last_fullsized":"2024-04-24T05:46:05.607713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:45.991179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606401+0000","last_change":"2024-04-24T05:46:00.211997+0000","last_active":"2024-04-24T05:46:05.606401+0000","last_peered":"2024-04-24T05:46:05.606401+0000","last_clean":"2024-04-24T05:46:05.606401+0000","last_became_active":"2024-04-24T05:46:00.209526+0000","last_became_peered":"2024-04-24T05:46:00.209526+0000","last_unstale":"2024-04-24T05:46:05.606401+0000","last_undegraded":"2024-04-24T05:46:05.606401+0000","last_fullsized":"2024-04-24T05:46:05.606401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:27.080921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058600+0000","last_change":"2024-04-24T05:45:59.198738+0000","last_active":"2024-04-24T05:46:06.058600+0000","last_peered":"2024-04-24T05:46:06.058600+0000","last_clean":"2024-04-24T05:46:06.058600+0000","last_became_active":"2024-04-24T05:45:59.198458+0000","last_became_peered":"2024-04-24T05:45:59.198458+0000","last_unstale":"2024-04-24T05:46:06.058600+0000","last_undegraded":"2024-04-24T05:46:06.058600+0000","last_fullsized":"2024-04-24T05:46:06.058600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:11.561126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608670+0000","last_change":"2024-04-24T05:46:04.266757+0000","last_active":"2024-04-24T05:46:05.608670+0000","last_peered":"2024-04-24T05:46:05.608670+0000","last_clean":"2024-04-24T05:46:05.608670+0000","last_became_active":"2024-04-24T05:46:04.266279+0000","last_became_peered":"2024-04-24T05:46:04.266279+0000","last_unstale":"2024-04-24T05:46:05.608670+0000","last_undegraded":"2024-04-24T05:46:05.608670+0000","last_fullsized":"2024-04-24T05:46:05.608670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:06:00.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058554+0000","last_change":"2024-04-24T05:46:00.216303+0000","last_active":"2024-04-24T05:46:06.058554+0000","last_peered":"2024-04-24T05:46:06.058554+0000","last_clean":"2024-04-24T05:46:06.058554+0000","last_became_active":"2024-04-24T05:46:00.215719+0000","last_became_peered":"2024-04-24T05:46:00.215719+0000","last_unstale":"2024-04-24T05:46:06.058554+0000","last_undegraded":"2024-04-24T05:46:06.058554+0000","last_fullsized":"2024-04-24T05:46:06.058554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:47:58.509148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558034+0000","last_change":"2024-04-24T05:45:59.218643+0000","last_active":"2024-04-24T05:46:05.558034+0000","last_peered":"2024-04-24T05:46:05.558034+0000","last_clean":"2024-04-24T05:46:05.558034+0000","last_became_active":"2024-04-24T05:45:59.218346+0000","last_became_peered":"2024-04-24T05:45:59.218346+0000","last_unstale":"2024-04-24T05:46:05.558034+0000","last_undegraded":"2024-04-24T05:46:05.558034+0000","last_fullsized":"2024-04-24T05:46:05.558034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:26.453566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063814+0000","last_change":"2024-04-24T05:46:04.527188+0000","last_active":"2024-04-24T05:46:06.063814+0000","last_peered":"2024-04-24T05:46:06.063814+0000","last_clean":"2024-04-24T05:46:06.063814+0000","last_became_active":"2024-04-24T05:46:04.526002+0000","last_became_peered":"2024-04-24T05:46:04.526002+0000","last_unstale":"2024-04-24T05:46:06.063814+0000","last_undegraded":"2024-04-24T05:46:06.063814+0000","last_fullsized":"2024-04-24T05:46:06.063814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:06:00.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060620+0000","last_change":"2024-04-24T05:46:00.224034+0000","last_active":"2024-04-24T05:46:06.060620+0000","last_peered":"2024-04-24T05:46:06.060620+0000","last_clean":"2024-04-24T05:46:06.060620+0000","last_became_active":"2024-04-24T05:46:00.223714+0000","last_became_peered":"2024-04-24T05:46:00.223714+0000","last_unstale":"2024-04-24T05:46:06.060620+0000","last_undegraded":"2024-04-24T05:46:06.060620+0000","last_fullsized":"2024-04-24T05:46:06.060620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:25.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252259+0000","last_change":"2024-04-24T05:45:59.199095+0000","last_active":"2024-04-24T05:46:05.252259+0000","last_peered":"2024-04-24T05:46:05.252259+0000","last_clean":"2024-04-24T05:46:05.252259+0000","last_became_active":"2024-04-24T05:45:59.198772+0000","last_became_peered":"2024-04-24T05:45:59.198772+0000","last_unstale":"2024-04-24T05:46:05.252259+0000","last_undegraded":"2024-04-24T05:46:05.252259+0000","last_fullsized":"2024-04-24T05:46:05.252259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:11.561126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558345+0000","last_change":"2024-04-24T05:45:51.019903+0000","last_active":"2024-04-24T05:46:05.558345+0000","last_peered":"2024-04-24T05:46:05.558345+0000","last_clean":"2024-04-24T05:46:05.558345+0000","last_became_active":"2024-04-24T05:45:51.017323+0000","last_became_peered":"2024-04-24T05:45:51.017323+0000","last_unstale":"2024-04-24T05:46:05.558345+0000","last_undegraded":"2024-04-24T05:46:05.558345+0000","last_fullsized":"2024-04-24T05:46:05.558345+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:48.855871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:48.855871+0000","last_clean_scrub_stamp":"2024-04-24T05:45:48.855871+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:10:41.045203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058781+0000","last_change":"2024-04-24T05:46:04.525962+0000","last_active":"2024-04-24T05:46:06.058781+0000","last_peered":"2024-04-24T05:46:06.058781+0000","last_clean":"2024-04-24T05:46:06.058781+0000","last_became_active":"2024-04-24T05:46:04.524947+0000","last_became_peered":"2024-04-24T05:46:04.524947+0000","last_unstale":"2024-04-24T05:46:06.058781+0000","last_undegraded":"2024-04-24T05:46:06.058781+0000","last_fullsized":"2024-04-24T05:46:06.058781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:17.960048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557690+0000","last_change":"2024-04-24T05:46:00.226016+0000","last_active":"2024-04-24T05:46:05.557690+0000","last_peered":"2024-04-24T05:46:05.557690+0000","last_clean":"2024-04-24T05:46:05.557690+0000","last_became_active":"2024-04-24T05:46:00.225693+0000","last_became_peered":"2024-04-24T05:46:00.225693+0000","last_unstale":"2024-04-24T05:46:05.557690+0000","last_undegraded":"2024-04-24T05:46:05.557690+0000","last_fullsized":"2024-04-24T05:46:05.557690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:13.907547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059847+0000","last_change":"2024-04-24T05:45:59.203021+0000","last_active":"2024-04-24T05:46:06.059847+0000","last_peered":"2024-04-24T05:46:06.059847+0000","last_clean":"2024-04-24T05:46:06.059847+0000","last_became_active":"2024-04-24T05:45:59.202733+0000","last_became_peered":"2024-04-24T05:45:59.202733+0000","last_unstale":"2024-04-24T05:46:06.059847+0000","last_undegraded":"2024-04-24T05:46:06.059847+0000","last_fullsized":"2024-04-24T05:46:06.059847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058297+0000","last_change":"2024-04-24T05:46:04.521125+0000","last_active":"2024-04-24T05:46:06.058297+0000","last_peered":"2024-04-24T05:46:06.058297+0000","last_clean":"2024-04-24T05:46:06.058297+0000","last_became_active":"2024-04-24T05:46:04.519878+0000","last_became_peered":"2024-04-24T05:46:04.519878+0000","last_unstale":"2024-04-24T05:46:06.058297+0000","last_undegraded":"2024-04-24T05:46:06.058297+0000","last_fullsized":"2024-04-24T05:46:06.058297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:34.349539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252701+0000","last_change":"2024-04-24T05:46:00.214353+0000","last_active":"2024-04-24T05:46:05.252701+0000","last_peered":"2024-04-24T05:46:05.252701+0000","last_clean":"2024-04-24T05:46:05.252701+0000","last_became_active":"2024-04-24T05:46:00.213791+0000","last_became_peered":"2024-04-24T05:46:00.213791+0000","last_unstale":"2024-04-24T05:46:05.252701+0000","last_undegraded":"2024-04-24T05:46:05.252701+0000","last_fullsized":"2024-04-24T05:46:05.252701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:30.297037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608191+0000","last_change":"2024-04-24T05:45:59.216779+0000","last_active":"2024-04-24T05:46:05.608191+0000","last_peered":"2024-04-24T05:46:05.608191+0000","last_clean":"2024-04-24T05:46:05.608191+0000","last_became_active":"2024-04-24T05:45:59.216507+0000","last_became_peered":"2024-04-24T05:45:59.216507+0000","last_unstale":"2024-04-24T05:46:05.608191+0000","last_undegraded":"2024-04-24T05:46:05.608191+0000","last_fullsized":"2024-04-24T05:46:05.608191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062332+0000","last_change":"2024-04-24T05:46:04.268777+0000","last_active":"2024-04-24T05:46:06.062332+0000","last_peered":"2024-04-24T05:46:06.062332+0000","last_clean":"2024-04-24T05:46:06.062332+0000","last_became_active":"2024-04-24T05:46:04.268416+0000","last_became_peered":"2024-04-24T05:46:04.268416+0000","last_unstale":"2024-04-24T05:46:06.062332+0000","last_undegraded":"2024-04-24T05:46:06.062332+0000","last_fullsized":"2024-04-24T05:46:06.062332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:13.252851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558634+0000","last_change":"2024-04-24T05:46:00.227851+0000","last_active":"2024-04-24T05:46:05.558634+0000","last_peered":"2024-04-24T05:46:05.558634+0000","last_clean":"2024-04-24T05:46:05.558634+0000","last_became_active":"2024-04-24T05:46:00.220088+0000","last_became_peered":"2024-04-24T05:46:00.220088+0000","last_unstale":"2024-04-24T05:46:05.558634+0000","last_undegraded":"2024-04-24T05:46:05.558634+0000","last_fullsized":"2024-04-24T05:46:05.558634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:40.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059276+0000","last_change":"2024-04-24T05:45:59.196319+0000","last_active":"2024-04-24T05:46:06.059276+0000","last_peered":"2024-04-24T05:46:06.059276+0000","last_clean":"2024-04-24T05:46:06.059276+0000","last_became_active":"2024-04-24T05:45:59.196029+0000","last_became_peered":"2024-04-24T05:45:59.196029+0000","last_unstale":"2024-04-24T05:46:06.059276+0000","last_undegraded":"2024-04-24T05:46:06.059276+0000","last_fullsized":"2024-04-24T05:46:06.059276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.250996+0000","last_change":"2024-04-24T05:46:04.527040+0000","last_active":"2024-04-24T05:46:05.250996+0000","last_peered":"2024-04-24T05:46:05.250996+0000","last_clean":"2024-04-24T05:46:05.250996+0000","last_became_active":"2024-04-24T05:46:04.522889+0000","last_became_peered":"2024-04-24T05:46:04.522889+0000","last_unstale":"2024-04-24T05:46:05.250996+0000","last_undegraded":"2024-04-24T05:46:05.250996+0000","last_fullsized":"2024-04-24T05:46:05.250996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:06:00.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558335+0000","last_change":"2024-04-24T05:46:00.225225+0000","last_active":"2024-04-24T05:46:05.558335+0000","last_peered":"2024-04-24T05:46:05.558335+0000","last_clean":"2024-04-24T05:46:05.558335+0000","last_became_active":"2024-04-24T05:46:00.224932+0000","last_became_peered":"2024-04-24T05:46:00.224932+0000","last_unstale":"2024-04-24T05:46:05.558335+0000","last_undegraded":"2024-04-24T05:46:05.558335+0000","last_fullsized":"2024-04-24T05:46:05.558335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:41.938677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248435+0000","last_change":"2024-04-24T05:45:59.214909+0000","last_active":"2024-04-24T05:46:05.248435+0000","last_peered":"2024-04-24T05:46:05.248435+0000","last_clean":"2024-04-24T05:46:05.248435+0000","last_became_active":"2024-04-24T05:45:59.213540+0000","last_became_peered":"2024-04-24T05:45:59.213540+0000","last_unstale":"2024-04-24T05:46:05.248435+0000","last_undegraded":"2024-04-24T05:46:05.248435+0000","last_fullsized":"2024-04-24T05:46:05.248435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:00.340470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.246897+0000","last_change":"2024-04-24T05:46:04.275887+0000","last_active":"2024-04-24T05:46:05.246897+0000","last_peered":"2024-04-24T05:46:05.246897+0000","last_clean":"2024-04-24T05:46:05.246897+0000","last_became_active":"2024-04-24T05:46:04.275455+0000","last_became_peered":"2024-04-24T05:46:04.275455+0000","last_unstale":"2024-04-24T05:46:05.246897+0000","last_undegraded":"2024-04-24T05:46:05.246897+0000","last_fullsized":"2024-04-24T05:46:05.246897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:29.075099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557429+0000","last_change":"2024-04-24T05:46:00.217599+0000","last_active":"2024-04-24T05:46:05.557429+0000","last_peered":"2024-04-24T05:46:05.557429+0000","last_clean":"2024-04-24T05:46:05.557429+0000","last_became_active":"2024-04-24T05:46:00.217383+0000","last_became_peered":"2024-04-24T05:46:00.217383+0000","last_unstale":"2024-04-24T05:46:05.557429+0000","last_undegraded":"2024-04-24T05:46:05.557429+0000","last_fullsized":"2024-04-24T05:46:05.557429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:38.107685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.057867+0000","last_change":"2024-04-24T05:45:59.196826+0000","last_active":"2024-04-24T05:46:06.057867+0000","last_peered":"2024-04-24T05:46:06.057867+0000","last_clean":"2024-04-24T05:46:06.057867+0000","last_became_active":"2024-04-24T05:45:59.195023+0000","last_became_peered":"2024-04-24T05:45:59.195023+0000","last_unstale":"2024-04-24T05:46:06.057867+0000","last_undegraded":"2024-04-24T05:46:06.057867+0000","last_fullsized":"2024-04-24T05:46:06.057867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063777+0000","last_change":"2024-04-24T05:46:04.265949+0000","last_active":"2024-04-24T05:46:06.063777+0000","last_peered":"2024-04-24T05:46:06.063777+0000","last_clean":"2024-04-24T05:46:06.063777+0000","last_became_active":"2024-04-24T05:46:04.265546+0000","last_became_peered":"2024-04-24T05:46:04.265546+0000","last_unstale":"2024-04-24T05:46:06.063777+0000","last_undegraded":"2024-04-24T05:46:06.063777+0000","last_fullsized":"2024-04-24T05:46:06.063777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:01:44.521141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058813+0000","last_change":"2024-04-24T05:46:00.221203+0000","last_active":"2024-04-24T05:46:06.058813+0000","last_peered":"2024-04-24T05:46:06.058813+0000","last_clean":"2024-04-24T05:46:06.058813+0000","last_became_active":"2024-04-24T05:46:00.220709+0000","last_became_peered":"2024-04-24T05:46:00.220709+0000","last_unstale":"2024-04-24T05:46:06.058813+0000","last_undegraded":"2024-04-24T05:46:06.058813+0000","last_fullsized":"2024-04-24T05:46:06.058813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059170+0000","last_change":"2024-04-24T05:45:59.217694+0000","last_active":"2024-04-24T05:46:06.059170+0000","last_peered":"2024-04-24T05:46:06.059170+0000","last_clean":"2024-04-24T05:46:06.059170+0000","last_became_active":"2024-04-24T05:45:59.217385+0000","last_became_peered":"2024-04-24T05:45:59.217385+0000","last_unstale":"2024-04-24T05:46:06.059170+0000","last_undegraded":"2024-04-24T05:46:06.059170+0000","last_fullsized":"2024-04-24T05:46:06.059170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063100+0000","last_change":"2024-04-24T05:46:04.529839+0000","last_active":"2024-04-24T05:46:06.063100+0000","last_peered":"2024-04-24T05:46:06.063100+0000","last_clean":"2024-04-24T05:46:06.063100+0000","last_became_active":"2024-04-24T05:46:04.524578+0000","last_became_peered":"2024-04-24T05:46:04.524578+0000","last_unstale":"2024-04-24T05:46:06.063100+0000","last_undegraded":"2024-04-24T05:46:06.063100+0000","last_fullsized":"2024-04-24T05:46:06.063100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:26.692990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557529+0000","last_change":"2024-04-24T05:46:00.226661+0000","last_active":"2024-04-24T05:46:05.557529+0000","last_peered":"2024-04-24T05:46:05.557529+0000","last_clean":"2024-04-24T05:46:05.557529+0000","last_became_active":"2024-04-24T05:46:00.226357+0000","last_became_peered":"2024-04-24T05:46:00.226357+0000","last_unstale":"2024-04-24T05:46:05.557529+0000","last_undegraded":"2024-04-24T05:46:05.557529+0000","last_fullsized":"2024-04-24T05:46:05.557529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:05:06.286589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252051+0000","last_change":"2024-04-24T05:45:59.206276+0000","last_active":"2024-04-24T05:46:05.252051+0000","last_peered":"2024-04-24T05:46:05.252051+0000","last_clean":"2024-04-24T05:46:05.252051+0000","last_became_active":"2024-04-24T05:45:59.205926+0000","last_became_peered":"2024-04-24T05:45:59.205926+0000","last_unstale":"2024-04-24T05:46:05.252051+0000","last_undegraded":"2024-04-24T05:46:05.252051+0000","last_fullsized":"2024-04-24T05:46:05.252051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062045+0000","last_change":"2024-04-24T05:46:04.515144+0000","last_active":"2024-04-24T05:46:06.062045+0000","last_peered":"2024-04-24T05:46:06.062045+0000","last_clean":"2024-04-24T05:46:06.062045+0000","last_became_active":"2024-04-24T05:46:04.514326+0000","last_became_peered":"2024-04-24T05:46:04.514326+0000","last_unstale":"2024-04-24T05:46:06.062045+0000","last_undegraded":"2024-04-24T05:46:06.062045+0000","last_fullsized":"2024-04-24T05:46:06.062045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:45.847497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060827+0000","last_change":"2024-04-24T05:46:00.214864+0000","last_active":"2024-04-24T05:46:06.060827+0000","last_peered":"2024-04-24T05:46:06.060827+0000","last_clean":"2024-04-24T05:46:06.060827+0000","last_became_active":"2024-04-24T05:46:00.214051+0000","last_became_peered":"2024-04-24T05:46:00.214051+0000","last_unstale":"2024-04-24T05:46:06.060827+0000","last_undegraded":"2024-04-24T05:46:06.060827+0000","last_fullsized":"2024-04-24T05:46:06.060827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:41:43.988057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062825+0000","last_change":"2024-04-24T05:45:59.213057+0000","last_active":"2024-04-24T05:46:06.062825+0000","last_peered":"2024-04-24T05:46:06.062825+0000","last_clean":"2024-04-24T05:46:06.062825+0000","last_became_active":"2024-04-24T05:45:59.212716+0000","last_became_peered":"2024-04-24T05:45:59.212716+0000","last_unstale":"2024-04-24T05:46:06.062825+0000","last_undegraded":"2024-04-24T05:46:06.062825+0000","last_fullsized":"2024-04-24T05:46:06.062825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:53.435411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252861+0000","last_change":"2024-04-24T05:46:04.516510+0000","last_active":"2024-04-24T05:46:05.252861+0000","last_peered":"2024-04-24T05:46:05.252861+0000","last_clean":"2024-04-24T05:46:05.252861+0000","last_became_active":"2024-04-24T05:46:04.515458+0000","last_became_peered":"2024-04-24T05:46:04.515458+0000","last_unstale":"2024-04-24T05:46:05.252861+0000","last_undegraded":"2024-04-24T05:46:05.252861+0000","last_fullsized":"2024-04-24T05:46:05.252861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:16:08.677684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557591+0000","last_change":"2024-04-24T05:46:00.215087+0000","last_active":"2024-04-24T05:46:05.557591+0000","last_peered":"2024-04-24T05:46:05.557591+0000","last_clean":"2024-04-24T05:46:05.557591+0000","last_became_active":"2024-04-24T05:46:00.213783+0000","last_became_peered":"2024-04-24T05:46:00.213783+0000","last_unstale":"2024-04-24T05:46:05.557591+0000","last_undegraded":"2024-04-24T05:46:05.557591+0000","last_fullsized":"2024-04-24T05:46:05.557591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:51:30.297037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:17.734750+0000","last_change":"2024-04-24T05:45:59.211881+0000","last_active":"2024-04-24T05:46:17.734750+0000","last_peered":"2024-04-24T05:46:17.734750+0000","last_clean":"2024-04-24T05:46:17.734750+0000","last_became_active":"2024-04-24T05:45:59.211566+0000","last_became_peered":"2024-04-24T05:45:59.211566+0000","last_unstale":"2024-04-24T05:46:17.734750+0000","last_undegraded":"2024-04-24T05:46:17.734750+0000","last_fullsized":"2024-04-24T05:46:17.734750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607263+0000","last_change":"2024-04-24T05:46:04.264433+0000","last_active":"2024-04-24T05:46:05.607263+0000","last_peered":"2024-04-24T05:46:05.607263+0000","last_clean":"2024-04-24T05:46:05.607263+0000","last_became_active":"2024-04-24T05:46:04.263926+0000","last_became_peered":"2024-04-24T05:46:04.263926+0000","last_unstale":"2024-04-24T05:46:05.607263+0000","last_undegraded":"2024-04-24T05:46:05.607263+0000","last_fullsized":"2024-04-24T05:46:05.607263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:13.252851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248914+0000","last_change":"2024-04-24T05:46:00.218334+0000","last_active":"2024-04-24T05:46:05.248914+0000","last_peered":"2024-04-24T05:46:05.248914+0000","last_clean":"2024-04-24T05:46:05.248914+0000","last_became_active":"2024-04-24T05:46:00.213738+0000","last_became_peered":"2024-04-24T05:46:00.213738+0000","last_unstale":"2024-04-24T05:46:05.248914+0000","last_undegraded":"2024-04-24T05:46:05.248914+0000","last_fullsized":"2024-04-24T05:46:05.248914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:27.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608102+0000","last_change":"2024-04-24T05:45:59.206057+0000","last_active":"2024-04-24T05:46:05.608102+0000","last_peered":"2024-04-24T05:46:05.608102+0000","last_clean":"2024-04-24T05:46:05.608102+0000","last_became_active":"2024-04-24T05:45:59.205560+0000","last_became_peered":"2024-04-24T05:45:59.205560+0000","last_unstale":"2024-04-24T05:46:05.608102+0000","last_undegraded":"2024-04-24T05:46:05.608102+0000","last_fullsized":"2024-04-24T05:46:05.608102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:31.157944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248223+0000","last_change":"2024-04-24T05:46:04.274265+0000","last_active":"2024-04-24T05:46:05.248223+0000","last_peered":"2024-04-24T05:46:05.248223+0000","last_clean":"2024-04-24T05:46:05.248223+0000","last_became_active":"2024-04-24T05:46:04.273775+0000","last_became_peered":"2024-04-24T05:46:04.273775+0000","last_unstale":"2024-04-24T05:46:05.248223+0000","last_undegraded":"2024-04-24T05:46:05.248223+0000","last_fullsized":"2024-04-24T05:46:05.248223+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:49.092042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063970+0000","last_change":"2024-04-24T05:46:00.222245+0000","last_active":"2024-04-24T05:46:06.063970+0000","last_peered":"2024-04-24T05:46:06.063970+0000","last_clean":"2024-04-24T05:46:06.063970+0000","last_became_active":"2024-04-24T05:46:00.220540+0000","last_became_peered":"2024-04-24T05:46:00.220540+0000","last_unstale":"2024-04-24T05:46:06.063970+0000","last_undegraded":"2024-04-24T05:46:06.063970+0000","last_fullsized":"2024-04-24T05:46:06.063970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:55.348055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253552+0000","last_change":"2024-04-24T05:45:59.203808+0000","last_active":"2024-04-24T05:46:05.253552+0000","last_peered":"2024-04-24T05:46:05.253552+0000","last_clean":"2024-04-24T05:46:05.253552+0000","last_became_active":"2024-04-24T05:45:59.203023+0000","last_became_peered":"2024-04-24T05:45:59.203023+0000","last_unstale":"2024-04-24T05:46:05.253552+0000","last_undegraded":"2024-04-24T05:46:05.253552+0000","last_fullsized":"2024-04-24T05:46:05.253552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059474+0000","last_change":"2024-04-24T05:46:04.274610+0000","last_active":"2024-04-24T05:46:06.059474+0000","last_peered":"2024-04-24T05:46:06.059474+0000","last_clean":"2024-04-24T05:46:06.059474+0000","last_became_active":"2024-04-24T05:46:04.274246+0000","last_became_peered":"2024-04-24T05:46:04.274246+0000","last_unstale":"2024-04-24T05:46:06.059474+0000","last_undegraded":"2024-04-24T05:46:06.059474+0000","last_fullsized":"2024-04-24T05:46:06.059474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:31:17.536498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.253481+0000","last_change":"2024-04-24T05:46:00.207959+0000","last_active":"2024-04-24T05:46:05.253481+0000","last_peered":"2024-04-24T05:46:05.253481+0000","last_clean":"2024-04-24T05:46:05.253481+0000","last_became_active":"2024-04-24T05:46:00.205225+0000","last_became_peered":"2024-04-24T05:46:00.205225+0000","last_unstale":"2024-04-24T05:46:05.253481+0000","last_undegraded":"2024-04-24T05:46:05.253481+0000","last_fullsized":"2024-04-24T05:46:05.253481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:55:08.484471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059232+0000","last_change":"2024-04-24T05:45:59.212223+0000","last_active":"2024-04-24T05:46:06.059232+0000","last_peered":"2024-04-24T05:46:06.059232+0000","last_clean":"2024-04-24T05:46:06.059232+0000","last_became_active":"2024-04-24T05:45:59.211908+0000","last_became_peered":"2024-04-24T05:45:59.211908+0000","last_unstale":"2024-04-24T05:46:06.059232+0000","last_undegraded":"2024-04-24T05:46:06.059232+0000","last_fullsized":"2024-04-24T05:46:06.059232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:55:17.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060243+0000","last_change":"2024-04-24T05:46:04.529148+0000","last_active":"2024-04-24T05:46:06.060243+0000","last_peered":"2024-04-24T05:46:06.060243+0000","last_clean":"2024-04-24T05:46:06.060243+0000","last_became_active":"2024-04-24T05:46:04.524775+0000","last_became_peered":"2024-04-24T05:46:04.524775+0000","last_unstale":"2024-04-24T05:46:06.060243+0000","last_undegraded":"2024-04-24T05:46:06.060243+0000","last_fullsized":"2024-04-24T05:46:06.060243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:58:44.607716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063332+0000","last_change":"2024-04-24T05:46:04.524596+0000","last_active":"2024-04-24T05:46:06.063332+0000","last_peered":"2024-04-24T05:46:06.063332+0000","last_clean":"2024-04-24T05:46:06.063332+0000","last_became_active":"2024-04-24T05:46:04.524013+0000","last_became_peered":"2024-04-24T05:46:04.524013+0000","last_unstale":"2024-04-24T05:46:06.063332+0000","last_undegraded":"2024-04-24T05:46:06.063332+0000","last_fullsized":"2024-04-24T05:46:06.063332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:45.991179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606337+0000","last_change":"2024-04-24T05:46:00.222945+0000","last_active":"2024-04-24T05:46:05.606337+0000","last_peered":"2024-04-24T05:46:05.606337+0000","last_clean":"2024-04-24T05:46:05.606337+0000","last_became_active":"2024-04-24T05:46:00.222698+0000","last_became_peered":"2024-04-24T05:46:00.222698+0000","last_unstale":"2024-04-24T05:46:05.606337+0000","last_undegraded":"2024-04-24T05:46:05.606337+0000","last_fullsized":"2024-04-24T05:46:05.606337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251487+0000","last_change":"2024-04-24T05:45:59.211291+0000","last_active":"2024-04-24T05:46:05.251487+0000","last_peered":"2024-04-24T05:46:05.251487+0000","last_clean":"2024-04-24T05:46:05.251487+0000","last_became_active":"2024-04-24T05:45:59.209448+0000","last_became_peered":"2024-04-24T05:45:59.209448+0000","last_unstale":"2024-04-24T05:46:05.251487+0000","last_undegraded":"2024-04-24T05:46:05.251487+0000","last_fullsized":"2024-04-24T05:46:05.251487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:45.254420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058917+0000","last_change":"2024-04-24T05:46:04.274043+0000","last_active":"2024-04-24T05:46:06.058917+0000","last_peered":"2024-04-24T05:46:06.058917+0000","last_clean":"2024-04-24T05:46:06.058917+0000","last_became_active":"2024-04-24T05:46:04.273504+0000","last_became_peered":"2024-04-24T05:46:04.273504+0000","last_unstale":"2024-04-24T05:46:06.058917+0000","last_undegraded":"2024-04-24T05:46:06.058917+0000","last_fullsized":"2024-04-24T05:46:06.058917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:55:12.536973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059394+0000","last_change":"2024-04-24T05:46:00.216386+0000","last_active":"2024-04-24T05:46:06.059394+0000","last_peered":"2024-04-24T05:46:06.059394+0000","last_clean":"2024-04-24T05:46:06.059394+0000","last_became_active":"2024-04-24T05:46:00.216078+0000","last_became_peered":"2024-04-24T05:46:00.216078+0000","last_unstale":"2024-04-24T05:46:06.059394+0000","last_undegraded":"2024-04-24T05:46:06.059394+0000","last_fullsized":"2024-04-24T05:46:06.059394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060995+0000","last_change":"2024-04-24T05:45:59.206266+0000","last_active":"2024-04-24T05:46:06.060995+0000","last_peered":"2024-04-24T05:46:06.060995+0000","last_clean":"2024-04-24T05:46:06.060995+0000","last_became_active":"2024-04-24T05:45:59.205881+0000","last_became_peered":"2024-04-24T05:45:59.205881+0000","last_unstale":"2024-04-24T05:46:06.060995+0000","last_undegraded":"2024-04-24T05:46:06.060995+0000","last_fullsized":"2024-04-24T05:46:06.060995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:22:53.435411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058603+0000","last_change":"2024-04-24T05:45:59.217059+0000","last_active":"2024-04-24T05:46:06.058603+0000","last_peered":"2024-04-24T05:46:06.058603+0000","last_clean":"2024-04-24T05:46:06.058603+0000","last_became_active":"2024-04-24T05:45:59.216742+0000","last_became_peered":"2024-04-24T05:45:59.216742+0000","last_unstale":"2024-04-24T05:46:06.058603+0000","last_undegraded":"2024-04-24T05:46:06.058603+0000","last_fullsized":"2024-04-24T05:46:06.058603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:59:20.829113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062704+0000","last_change":"2024-04-24T05:46:00.215564+0000","last_active":"2024-04-24T05:46:06.062704+0000","last_peered":"2024-04-24T05:46:06.062704+0000","last_clean":"2024-04-24T05:46:06.062704+0000","last_became_active":"2024-04-24T05:46:00.214442+0000","last_became_peered":"2024-04-24T05:46:00.214442+0000","last_unstale":"2024-04-24T05:46:06.062704+0000","last_undegraded":"2024-04-24T05:46:06.062704+0000","last_fullsized":"2024-04-24T05:46:06.062704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059540+0000","last_change":"2024-04-24T05:46:04.272081+0000","last_active":"2024-04-24T05:46:06.059540+0000","last_peered":"2024-04-24T05:46:06.059540+0000","last_clean":"2024-04-24T05:46:06.059540+0000","last_became_active":"2024-04-24T05:46:04.269391+0000","last_became_peered":"2024-04-24T05:46:04.269391+0000","last_unstale":"2024-04-24T05:46:06.059540+0000","last_undegraded":"2024-04-24T05:46:06.059540+0000","last_fullsized":"2024-04-24T05:46:06.059540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:24.966843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061465+0000","last_change":"2024-04-24T05:45:59.210758+0000","last_active":"2024-04-24T05:46:06.061465+0000","last_peered":"2024-04-24T05:46:06.061465+0000","last_clean":"2024-04-24T05:46:06.061465+0000","last_became_active":"2024-04-24T05:45:59.210533+0000","last_became_peered":"2024-04-24T05:45:59.210533+0000","last_unstale":"2024-04-24T05:46:06.061465+0000","last_undegraded":"2024-04-24T05:46:06.061465+0000","last_fullsized":"2024-04-24T05:46:06.061465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:58.142587+0000","last_clean_scrub_stamp":"2024-04-24T05:45:58.142587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:11.561126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.247851+0000","last_change":"2024-04-24T05:46:00.211972+0000","last_active":"2024-04-24T05:46:05.247851+0000","last_peered":"2024-04-24T05:46:05.247851+0000","last_clean":"2024-04-24T05:46:05.247851+0000","last_became_active":"2024-04-24T05:46:00.210764+0000","last_became_peered":"2024-04-24T05:46:00.210764+0000","last_unstale":"2024-04-24T05:46:05.247851+0000","last_undegraded":"2024-04-24T05:46:05.247851+0000","last_fullsized":"2024-04-24T05:46:05.247851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:32.173672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062871+0000","last_change":"2024-04-24T05:46:04.271064+0000","last_active":"2024-04-24T05:46:06.062871+0000","last_peered":"2024-04-24T05:46:06.062871+0000","last_clean":"2024-04-24T05:46:06.062871+0000","last_became_active":"2024-04-24T05:46:04.270662+0000","last_became_peered":"2024-04-24T05:46:04.270662+0000","last_unstale":"2024-04-24T05:46:06.062871+0000","last_undegraded":"2024-04-24T05:46:06.062871+0000","last_fullsized":"2024-04-24T05:46:06.062871+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:46:03.210816+0000","last_clean_scrub_stamp":"2024-04-24T05:46:03.210816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:51:18.795481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058290+0000","last_change":"2024-04-24T05:46:00.221213+0000","last_active":"2024-04-24T05:46:06.058290+0000","last_peered":"2024-04-24T05:46:06.058290+0000","last_clean":"2024-04-24T05:46:06.058290+0000","last_became_active":"2024-04-24T05:46:00.220647+0000","last_became_peered":"2024-04-24T05:46:00.220647+0000","last_unstale":"2024-04-24T05:46:06.058290+0000","last_undegraded":"2024-04-24T05:46:06.058290+0000","last_fullsized":"2024-04-24T05:46:06.058290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058501+0000","last_change":"2024-04-24T05:46:00.199902+0000","last_active":"2024-04-24T05:46:06.058501+0000","last_peered":"2024-04-24T05:46:06.058501+0000","last_clean":"2024-04-24T05:46:06.058501+0000","last_became_active":"2024-04-24T05:46:00.199314+0000","last_became_peered":"2024-04-24T05:46:00.199314+0000","last_unstale":"2024-04-24T05:46:06.058501+0000","last_undegraded":"2024-04-24T05:46:06.058501+0000","last_fullsized":"2024-04-24T05:46:06.058501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:27.469294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251956+0000","last_change":"2024-04-24T05:46:00.212233+0000","last_active":"2024-04-24T05:46:05.251956+0000","last_peered":"2024-04-24T05:46:05.251956+0000","last_clean":"2024-04-24T05:46:05.251956+0000","last_became_active":"2024-04-24T05:46:00.209914+0000","last_became_peered":"2024-04-24T05:46:00.209914+0000","last_unstale":"2024-04-24T05:46:05.251956+0000","last_undegraded":"2024-04-24T05:46:05.251956+0000","last_fullsized":"2024-04-24T05:46:05.251956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:45.039541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062577+0000","last_change":"2024-04-24T05:46:00.216833+0000","last_active":"2024-04-24T05:46:06.062577+0000","last_peered":"2024-04-24T05:46:06.062577+0000","last_clean":"2024-04-24T05:46:06.062577+0000","last_became_active":"2024-04-24T05:46:00.216494+0000","last_became_peered":"2024-04-24T05:46:00.216494+0000","last_unstale":"2024-04-24T05:46:06.062577+0000","last_undegraded":"2024-04-24T05:46:06.062577+0000","last_fullsized":"2024-04-24T05:46:06.062577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.062178+0000","last_change":"2024-04-24T05:46:00.213642+0000","last_active":"2024-04-24T05:46:06.062178+0000","last_peered":"2024-04-24T05:46:06.062178+0000","last_clean":"2024-04-24T05:46:06.062178+0000","last_became_active":"2024-04-24T05:46:00.210164+0000","last_became_peered":"2024-04-24T05:46:00.210164+0000","last_unstale":"2024-04-24T05:46:06.062178+0000","last_undegraded":"2024-04-24T05:46:06.062178+0000","last_fullsized":"2024-04-24T05:46:06.062178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:41.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.607788+0000","last_change":"2024-04-24T05:46:00.210082+0000","last_active":"2024-04-24T05:46:05.607788+0000","last_peered":"2024-04-24T05:46:05.607788+0000","last_clean":"2024-04-24T05:46:05.607788+0000","last_became_active":"2024-04-24T05:46:00.209799+0000","last_became_peered":"2024-04-24T05:46:00.209799+0000","last_unstale":"2024-04-24T05:46:05.607788+0000","last_undegraded":"2024-04-24T05:46:05.607788+0000","last_fullsized":"2024-04-24T05:46:05.607788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059611+0000","last_change":"2024-04-24T05:46:00.219304+0000","last_active":"2024-04-24T05:46:06.059611+0000","last_peered":"2024-04-24T05:46:06.059611+0000","last_clean":"2024-04-24T05:46:06.059611+0000","last_became_active":"2024-04-24T05:46:00.218953+0000","last_became_peered":"2024-04-24T05:46:00.218953+0000","last_unstale":"2024-04-24T05:46:06.059611+0000","last_undegraded":"2024-04-24T05:46:06.059611+0000","last_fullsized":"2024-04-24T05:46:06.059611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:41.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063234+0000","last_change":"2024-04-24T05:46:00.217971+0000","last_active":"2024-04-24T05:46:06.063234+0000","last_peered":"2024-04-24T05:46:06.063234+0000","last_clean":"2024-04-24T05:46:06.063234+0000","last_became_active":"2024-04-24T05:46:00.217789+0000","last_became_peered":"2024-04-24T05:46:00.217789+0000","last_unstale":"2024-04-24T05:46:06.063234+0000","last_undegraded":"2024-04-24T05:46:06.063234+0000","last_fullsized":"2024-04-24T05:46:06.063234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:38.107685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251295+0000","last_change":"2024-04-24T05:46:00.212887+0000","last_active":"2024-04-24T05:46:05.251295+0000","last_peered":"2024-04-24T05:46:05.251295+0000","last_clean":"2024-04-24T05:46:05.251295+0000","last_became_active":"2024-04-24T05:46:00.212550+0000","last_became_peered":"2024-04-24T05:46:00.212550+0000","last_unstale":"2024-04-24T05:46:05.251295+0000","last_undegraded":"2024-04-24T05:46:05.251295+0000","last_fullsized":"2024-04-24T05:46:05.251295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:25.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060618+0000","last_change":"2024-04-24T05:46:00.219721+0000","last_active":"2024-04-24T05:46:06.060618+0000","last_peered":"2024-04-24T05:46:06.060618+0000","last_clean":"2024-04-24T05:46:06.060618+0000","last_became_active":"2024-04-24T05:46:00.219429+0000","last_became_peered":"2024-04-24T05:46:00.219429+0000","last_unstale":"2024-04-24T05:46:06.060618+0000","last_undegraded":"2024-04-24T05:46:06.060618+0000","last_fullsized":"2024-04-24T05:46:06.060618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:27.080921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251686+0000","last_change":"2024-04-24T05:46:00.215082+0000","last_active":"2024-04-24T05:46:05.251686+0000","last_peered":"2024-04-24T05:46:05.251686+0000","last_clean":"2024-04-24T05:46:05.251686+0000","last_became_active":"2024-04-24T05:46:00.214719+0000","last_became_peered":"2024-04-24T05:46:00.214719+0000","last_unstale":"2024-04-24T05:46:05.251686+0000","last_undegraded":"2024-04-24T05:46:05.251686+0000","last_fullsized":"2024-04-24T05:46:05.251686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:42.688050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061826+0000","last_change":"2024-04-24T05:46:00.204113+0000","last_active":"2024-04-24T05:46:06.061826+0000","last_peered":"2024-04-24T05:46:06.061826+0000","last_clean":"2024-04-24T05:46:06.061826+0000","last_became_active":"2024-04-24T05:46:00.203764+0000","last_became_peered":"2024-04-24T05:46:00.203764+0000","last_unstale":"2024-04-24T05:46:06.061826+0000","last_undegraded":"2024-04-24T05:46:06.061826+0000","last_fullsized":"2024-04-24T05:46:06.061826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059326+0000","last_change":"2024-04-24T05:46:00.223248+0000","last_active":"2024-04-24T05:46:06.059326+0000","last_peered":"2024-04-24T05:46:06.059326+0000","last_clean":"2024-04-24T05:46:06.059326+0000","last_became_active":"2024-04-24T05:46:00.222366+0000","last_became_peered":"2024-04-24T05:46:00.222366+0000","last_unstale":"2024-04-24T05:46:06.059326+0000","last_undegraded":"2024-04-24T05:46:06.059326+0000","last_fullsized":"2024-04-24T05:46:06.059326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558036+0000","last_change":"2024-04-24T05:46:00.217085+0000","last_active":"2024-04-24T05:46:05.558036+0000","last_peered":"2024-04-24T05:46:05.558036+0000","last_clean":"2024-04-24T05:46:05.558036+0000","last_became_active":"2024-04-24T05:46:00.216799+0000","last_became_peered":"2024-04-24T05:46:00.216799+0000","last_unstale":"2024-04-24T05:46:05.558036+0000","last_undegraded":"2024-04-24T05:46:05.558036+0000","last_fullsized":"2024-04-24T05:46:05.558036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:02:27.050101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059180+0000","last_change":"2024-04-24T05:46:00.221238+0000","last_active":"2024-04-24T05:46:06.059180+0000","last_peered":"2024-04-24T05:46:06.059180+0000","last_clean":"2024-04-24T05:46:06.059180+0000","last_became_active":"2024-04-24T05:46:00.220945+0000","last_became_peered":"2024-04-24T05:46:00.220945+0000","last_unstale":"2024-04-24T05:46:06.059180+0000","last_undegraded":"2024-04-24T05:46:06.059180+0000","last_fullsized":"2024-04-24T05:46:06.059180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:21:57.974456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.254113+0000","last_change":"2024-04-24T05:46:00.206311+0000","last_active":"2024-04-24T05:46:05.254113+0000","last_peered":"2024-04-24T05:46:05.254113+0000","last_clean":"2024-04-24T05:46:05.254113+0000","last_became_active":"2024-04-24T05:46:00.205935+0000","last_became_peered":"2024-04-24T05:46:00.205935+0000","last_unstale":"2024-04-24T05:46:05.254113+0000","last_undegraded":"2024-04-24T05:46:05.254113+0000","last_fullsized":"2024-04-24T05:46:05.254113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:09.200350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251128+0000","last_change":"2024-04-24T05:46:00.202613+0000","last_active":"2024-04-24T05:46:05.251128+0000","last_peered":"2024-04-24T05:46:05.251128+0000","last_clean":"2024-04-24T05:46:05.251128+0000","last_became_active":"2024-04-24T05:46:00.202286+0000","last_became_peered":"2024-04-24T05:46:00.202286+0000","last_unstale":"2024-04-24T05:46:05.251128+0000","last_undegraded":"2024-04-24T05:46:05.251128+0000","last_fullsized":"2024-04-24T05:46:05.251128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.606554+0000","last_change":"2024-04-24T05:46:00.222335+0000","last_active":"2024-04-24T05:46:05.606554+0000","last_peered":"2024-04-24T05:46:05.606554+0000","last_clean":"2024-04-24T05:46:05.606554+0000","last_became_active":"2024-04-24T05:46:00.222066+0000","last_became_peered":"2024-04-24T05:46:00.222066+0000","last_unstale":"2024-04-24T05:46:05.606554+0000","last_undegraded":"2024-04-24T05:46:05.606554+0000","last_fullsized":"2024-04-24T05:46:05.606554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:18:27.829784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.060597+0000","last_change":"2024-04-24T05:46:00.213735+0000","last_active":"2024-04-24T05:46:06.060597+0000","last_peered":"2024-04-24T05:46:06.060597+0000","last_clean":"2024-04-24T05:46:06.060597+0000","last_became_active":"2024-04-24T05:46:00.213390+0000","last_became_peered":"2024-04-24T05:46:00.213390+0000","last_unstale":"2024-04-24T05:46:06.060597+0000","last_undegraded":"2024-04-24T05:46:06.060597+0000","last_fullsized":"2024-04-24T05:46:06.060597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:27:20.914341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.061232+0000","last_change":"2024-04-24T05:46:00.219073+0000","last_active":"2024-04-24T05:46:06.061232+0000","last_peered":"2024-04-24T05:46:06.061232+0000","last_clean":"2024-04-24T05:46:06.061232+0000","last_became_active":"2024-04-24T05:46:00.218577+0000","last_became_peered":"2024-04-24T05:46:00.218577+0000","last_unstale":"2024-04-24T05:46:06.061232+0000","last_undegraded":"2024-04-24T05:46:06.061232+0000","last_fullsized":"2024-04-24T05:46:06.061232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.248861+0000","last_change":"2024-04-24T05:46:00.219097+0000","last_active":"2024-04-24T05:46:05.248861+0000","last_peered":"2024-04-24T05:46:05.248861+0000","last_clean":"2024-04-24T05:46:05.248861+0000","last_became_active":"2024-04-24T05:46:00.218772+0000","last_became_peered":"2024-04-24T05:46:00.218772+0000","last_unstale":"2024-04-24T05:46:05.248861+0000","last_undegraded":"2024-04-24T05:46:05.248861+0000","last_fullsized":"2024-04-24T05:46:05.248861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:28:35.088347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059066+0000","last_change":"2024-04-24T05:46:00.221300+0000","last_active":"2024-04-24T05:46:06.059066+0000","last_peered":"2024-04-24T05:46:06.059066+0000","last_clean":"2024-04-24T05:46:06.059066+0000","last_became_active":"2024-04-24T05:46:00.220921+0000","last_became_peered":"2024-04-24T05:46:00.220921+0000","last_unstale":"2024-04-24T05:46:06.059066+0000","last_undegraded":"2024-04-24T05:46:06.059066+0000","last_fullsized":"2024-04-24T05:46:06.059066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:00:02.054620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.059208+0000","last_change":"2024-04-24T05:46:00.221735+0000","last_active":"2024-04-24T05:46:06.059208+0000","last_peered":"2024-04-24T05:46:06.059208+0000","last_clean":"2024-04-24T05:46:06.059208+0000","last_became_active":"2024-04-24T05:46:00.220859+0000","last_became_peered":"2024-04-24T05:46:00.220859+0000","last_unstale":"2024-04-24T05:46:06.059208+0000","last_undegraded":"2024-04-24T05:46:06.059208+0000","last_fullsized":"2024-04-24T05:46:06.059208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:25.022598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558492+0000","last_change":"2024-04-24T05:46:00.227925+0000","last_active":"2024-04-24T05:46:05.558492+0000","last_peered":"2024-04-24T05:46:05.558492+0000","last_clean":"2024-04-24T05:46:05.558492+0000","last_became_active":"2024-04-24T05:46:00.227553+0000","last_became_peered":"2024-04-24T05:46:00.227553+0000","last_unstale":"2024-04-24T05:46:05.558492+0000","last_undegraded":"2024-04-24T05:46:05.558492+0000","last_fullsized":"2024-04-24T05:46:05.558492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:29:09.200350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.058874+0000","last_change":"2024-04-24T05:46:00.225244+0000","last_active":"2024-04-24T05:46:06.058874+0000","last_peered":"2024-04-24T05:46:06.058874+0000","last_clean":"2024-04-24T05:46:06.058874+0000","last_became_active":"2024-04-24T05:46:00.224935+0000","last_became_peered":"2024-04-24T05:46:00.224935+0000","last_unstale":"2024-04-24T05:46:06.058874+0000","last_undegraded":"2024-04-24T05:46:06.058874+0000","last_fullsized":"2024-04-24T05:46:06.058874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:57.671822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557220+0000","last_change":"2024-04-24T05:46:00.225238+0000","last_active":"2024-04-24T05:46:05.557220+0000","last_peered":"2024-04-24T05:46:05.557220+0000","last_clean":"2024-04-24T05:46:05.557220+0000","last_became_active":"2024-04-24T05:46:00.224903+0000","last_became_peered":"2024-04-24T05:46:00.224903+0000","last_unstale":"2024-04-24T05:46:05.557220+0000","last_undegraded":"2024-04-24T05:46:05.557220+0000","last_fullsized":"2024-04-24T05:46:05.557220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:02:42.688050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.251674+0000","last_change":"2024-04-24T05:46:00.206725+0000","last_active":"2024-04-24T05:46:05.251674+0000","last_peered":"2024-04-24T05:46:05.251674+0000","last_clean":"2024-04-24T05:46:05.251674+0000","last_became_active":"2024-04-24T05:46:00.206266+0000","last_became_peered":"2024-04-24T05:46:00.206266+0000","last_unstale":"2024-04-24T05:46:05.251674+0000","last_undegraded":"2024-04-24T05:46:05.251674+0000","last_fullsized":"2024-04-24T05:46:05.251674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:28:41.852067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.557088+0000","last_change":"2024-04-24T05:46:00.219593+0000","last_active":"2024-04-24T05:46:05.557088+0000","last_peered":"2024-04-24T05:46:05.557088+0000","last_clean":"2024-04-24T05:46:05.557088+0000","last_became_active":"2024-04-24T05:46:00.219250+0000","last_became_peered":"2024-04-24T05:46:00.219250+0000","last_unstale":"2024-04-24T05:46:05.557088+0000","last_undegraded":"2024-04-24T05:46:05.557088+0000","last_fullsized":"2024-04-24T05:46:05.557088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:45.039541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.252270+0000","last_change":"2024-04-24T05:46:00.213476+0000","last_active":"2024-04-24T05:46:05.252270+0000","last_peered":"2024-04-24T05:46:05.252270+0000","last_clean":"2024-04-24T05:46:05.252270+0000","last_became_active":"2024-04-24T05:46:00.213065+0000","last_became_peered":"2024-04-24T05:46:00.213065+0000","last_unstale":"2024-04-24T05:46:05.252270+0000","last_undegraded":"2024-04-24T05:46:05.252270+0000","last_fullsized":"2024-04-24T05:46:05.252270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:38.107685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:06.063065+0000","last_change":"2024-04-24T05:46:00.221236+0000","last_active":"2024-04-24T05:46:06.063065+0000","last_peered":"2024-04-24T05:46:06.063065+0000","last_clean":"2024-04-24T05:46:06.063065+0000","last_became_active":"2024-04-24T05:46:00.220917+0000","last_became_peered":"2024-04-24T05:46:00.220917+0000","last_unstale":"2024-04-24T05:46:06.063065+0000","last_undegraded":"2024-04-24T05:46:06.063065+0000","last_fullsized":"2024-04-24T05:46:06.063065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:38.445039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.608230+0000","last_change":"2024-04-24T05:46:00.219494+0000","last_active":"2024-04-24T05:46:05.608230+0000","last_peered":"2024-04-24T05:46:05.608230+0000","last_clean":"2024-04-24T05:46:05.608230+0000","last_became_active":"2024-04-24T05:46:00.219219+0000","last_became_peered":"2024-04-24T05:46:00.219219+0000","last_unstale":"2024-04-24T05:46:05.608230+0000","last_undegraded":"2024-04-24T05:46:05.608230+0000","last_fullsized":"2024-04-24T05:46:05.608230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:39:34.963356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-24T05:46:05.558194+0000","last_change":"2024-04-24T05:46:00.221425+0000","last_active":"2024-04-24T05:46:05.558194+0000","last_peered":"2024-04-24T05:46:05.558194+0000","last_clean":"2024-04-24T05:46:05.558194+0000","last_became_active":"2024-04-24T05:46:00.219794+0000","last_became_peered":"2024-04-24T05:46:00.219794+0000","last_unstale":"2024-04-24T05:46:05.558194+0000","last_undegraded":"2024-04-24T05:46:05.558194+0000","last_fullsized":"2024-04-24T05:46:05.558194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:59.158315+0000","last_clean_scrub_stamp":"2024-04-24T05:45:59.158315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:55:08.484471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":696263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":109155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":105439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":109155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":105313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":105473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":696319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T05:46:27.677 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T05:46:27.677 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T05:46:27.677 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-24T05:46:27.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T05:46:27.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.906+0000 7fba67200640 1 Processor -- start 2024-04-24T05:46:27.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.907+0000 7fba67200640 1 -- start start 2024-04-24T05:46:27.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.908+0000 7fba67200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fba6806e660 0x7fba6806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:27.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.908+0000 7fba67200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fba680cf7b0 0x7fba6806c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:27.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.908+0000 7fba67200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba680d0de0 con 0x7fba680ad950 2024-04-24T05:46:27.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.908+0000 7fba67200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fba6815ec00 con 0x7fba6806e660 2024-04-24T05:46:27.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.908+0000 7fba67200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fba680ade50 con 0x7fba680cf7b0 2024-04-24T05:46:27.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.909+0000 7fba65e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fba6806e660 0x7fba6806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:27.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.909+0000 7fba65e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fba6806e660 0x7fba6806ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56854/0 (socket says 172.21.15.123:56854) 2024-04-24T05:46:27.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.909+0000 7fba65e00640 1 -- 172.21.15.123:0/929853055 learned_addr learned my addr 172.21.15.123:0/929853055 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:27.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.909+0000 7fba66800640 1 --2- 172.21.15.123:0/929853055 >> v2:172.21.15.155:3300/0 conn(0x7fba680cf7b0 0x7fba6806c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:27.911 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.909+0000 7fba65e00640 1 -- 172.21.15.123:0/929853055 >> v2:172.21.15.155:3300/0 conn(0x7fba680cf7b0 msgr2=0x7fba6806c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:27.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.909+0000 7fba65e00640 1 --2- 172.21.15.123:0/929853055 >> v2:172.21.15.155:3300/0 conn(0x7fba680cf7b0 0x7fba6806c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.909+0000 7fba65e00640 1 -- 172.21.15.123:0/929853055 >> v1:172.21.15.155:6789/0 conn(0x7fba680ad950 legacy=0x7fba6806ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:27.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.909+0000 7fba65e00640 1 -- 172.21.15.123:0/929853055 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6806dc80 con 0x7fba6806e660 2024-04-24T05:46:27.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.911+0000 7fba65e00640 1 --2- 172.21.15.123:0/929853055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba6806e660 0x7fba6806ea40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fba680b1bf0 tx=0x7fba5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2affb2bcc660cf64 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:27.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.911+0000 7fba65400640 1 -- 172.21.15.123:0/929853055 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba58037070 con 0x7fba6806e660 2024-04-24T05:46:27.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.911+0000 7fba65400640 1 -- 172.21.15.123:0/929853055 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba58032440 con 0x7fba6806e660 2024-04-24T05:46:27.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.911+0000 7fba65400640 1 -- 172.21.15.123:0/929853055 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba58032810 con 0x7fba6806e660 2024-04-24T05:46:27.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.912+0000 7fba67200640 1 -- 172.21.15.123:0/929853055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba6806e660 msgr2=0x7fba6806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:27.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.912+0000 7fba67200640 1 --2- 172.21.15.123:0/929853055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba6806e660 0x7fba6806ea40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fba680b1bf0 tx=0x7fba5802a270 comp rx=0 tx=0).stop 2024-04-24T05:46:27.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.912+0000 7fba67200640 1 -- 172.21.15.123:0/929853055 shutdown_connections 2024-04-24T05:46:27.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.912+0000 7fba67200640 1 --2- 172.21.15.123:0/929853055 >> v2:172.21.15.155:3300/0 conn(0x7fba680cf7b0 0x7fba6806c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.912+0000 7fba67200640 1 --2- 172.21.15.123:0/929853055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba6806e660 0x7fba6806ea40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.912+0000 7fba67200640 1 -- 172.21.15.123:0/929853055 >> 172.21.15.123:0/929853055 conn(0x7fba680d0070 msgr2=0x7fba680d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:27.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.912+0000 7fba67200640 1 -- 172.21.15.123:0/929853055 shutdown_connections 2024-04-24T05:46:27.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.912+0000 7fba67200640 1 -- 172.21.15.123:0/929853055 wait complete. 2024-04-24T05:46:27.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.914+0000 7fba67200640 1 Processor -- start 2024-04-24T05:46:27.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.914+0000 7fba67200640 1 -- start start 2024-04-24T05:46:27.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.914+0000 7fba67200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba6806e660 0x7fba681578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba67200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba680ad950 0x7fba6809d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba67200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba680cf7b0 0x7fba6809b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba67200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba6809e0c0 con 0x7fba680ad950 2024-04-24T05:46:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba67200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fba6809df40 con 0x7fba680cf7b0 2024-04-24T05:46:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba67200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fba6806c650 con 0x7fba6806e660 2024-04-24T05:46:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba5fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba680ad950 0x7fba6809d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba5fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba680ad950 0x7fba6809d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56860/0 (socket says 172.21.15.123:56860) 2024-04-24T05:46:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba5fe00640 1 -- 172.21.15.123:0/2113002513 learned_addr learned my addr 172.21.15.123:0/2113002513 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba66800640 1 --2- 172.21.15.123:0/2113002513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba680cf7b0 0x7fba6809b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba65e00640 1 --2- 172.21.15.123:0/2113002513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba6806e660 0x7fba681578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba5fe00640 1 -- 172.21.15.123:0/2113002513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba6806e660 msgr2=0x7fba681578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba5fe00640 1 --2- 172.21.15.123:0/2113002513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba6806e660 0x7fba681578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba5fe00640 1 -- 172.21.15.123:0/2113002513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba680cf7b0 msgr2=0x7fba6809b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:27.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba5fe00640 1 --2- 172.21.15.123:0/2113002513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba680cf7b0 0x7fba6809b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.915+0000 7fba5fe00640 1 -- 172.21.15.123:0/2113002513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba680ca210 con 0x7fba680ad950 2024-04-24T05:46:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.916+0000 7fba5fe00640 1 --2- 172.21.15.123:0/2113002513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba680ad950 0x7fba6809d060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fba5000c400 tx=0x7fba5000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.916+0000 7fba5e000640 1 -- 172.21.15.123:0/2113002513 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba500052d0 con 0x7fba680ad950 2024-04-24T05:46:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.916+0000 7fba5e000640 1 -- 172.21.15.123:0/2113002513 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba50005490 con 0x7fba680ad950 2024-04-24T05:46:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.916+0000 7fba67200640 1 -- 172.21.15.123:0/2113002513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba680ca480 con 0x7fba680ad950 2024-04-24T05:46:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.916+0000 7fba5e000640 1 -- 172.21.15.123:0/2113002513 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50007260 con 0x7fba680ad950 2024-04-24T05:46:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.916+0000 7fba67200640 1 -- 172.21.15.123:0/2113002513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba6809dc90 con 0x7fba680ad950 2024-04-24T05:46:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.917+0000 7fba67200640 1 -- 172.21.15.123:0/2113002513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba680cc480 con 0x7fba680ad950 2024-04-24T05:46:27.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.918+0000 7fba5e000640 1 -- 172.21.15.123:0/2113002513 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba50007400 con 0x7fba680ad950 2024-04-24T05:46:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.919+0000 7fba5e000640 1 --2- 172.21.15.123:0/2113002513 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fba4406c8f0 0x7fba4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.919+0000 7fba5e000640 1 -- 172.21.15.123:0/2113002513 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fba5008b4c0 con 0x7fba680ad950 2024-04-24T05:46:27.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.919+0000 7fba65e00640 1 --2- 172.21.15.123:0/2113002513 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fba4406c8f0 0x7fba4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:27.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.920+0000 7fba65e00640 1 --2- 172.21.15.123:0/2113002513 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fba4406c8f0 0x7fba4406edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba58034000 tx=0x7fba5802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:27.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:27.923+0000 7fba5e000640 1 -- 172.21.15.123:0/2113002513 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba680cc480 con 0x7fba680ad950 2024-04-24T05:46:28.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.164+0000 7fba67200640 1 -- 172.21.15.123:0/2113002513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fba68075b70 con 0x7fba680ad950 2024-04-24T05:46:28.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.164+0000 7fba5e000640 1 -- 172.21.15.123:0/2113002513 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fba50059510 con 0x7fba680ad950 2024-04-24T05:46:28.166 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:28.166 INFO:teuthology.orchestra.run.smithi123.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T05:46:28.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.167+0000 7fba67200640 1 -- 172.21.15.123:0/2113002513 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fba4406c8f0 msgr2=0x7fba4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:28.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.167+0000 7fba67200640 1 --2- 172.21.15.123:0/2113002513 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fba4406c8f0 0x7fba4406edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba58034000 tx=0x7fba5802e000 comp rx=0 tx=0).stop 2024-04-24T05:46:28.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.167+0000 7fba67200640 1 -- 172.21.15.123:0/2113002513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba680ad950 msgr2=0x7fba6809d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:28.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.167+0000 7fba67200640 1 --2- 172.21.15.123:0/2113002513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba680ad950 0x7fba6809d060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fba5000c400 tx=0x7fba5000c8c0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.167+0000 7fba67200640 1 -- 172.21.15.123:0/2113002513 shutdown_connections 2024-04-24T05:46:28.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.167+0000 7fba67200640 1 --2- 172.21.15.123:0/2113002513 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fba4406c8f0 0x7fba4406edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.167+0000 7fba67200640 1 --2- 172.21.15.123:0/2113002513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba680cf7b0 0x7fba6809b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.167+0000 7fba67200640 1 --2- 172.21.15.123:0/2113002513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba680ad950 0x7fba6809d060 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.167+0000 7fba67200640 1 --2- 172.21.15.123:0/2113002513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba6806e660 0x7fba681578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.168+0000 7fba67200640 1 -- 172.21.15.123:0/2113002513 >> 172.21.15.123:0/2113002513 conn(0x7fba680d0070 msgr2=0x7fba680730a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:28.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.168+0000 7fba67200640 1 -- 172.21.15.123:0/2113002513 shutdown_connections 2024-04-24T05:46:28.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.168+0000 7fba67200640 1 -- 172.21.15.123:0/2113002513 wait complete. 2024-04-24T05:46:28.189 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-24T05:46:28.189 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:28.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.394+0000 7fc3b2e00640 1 Processor -- start 2024-04-24T05:46:28.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.394+0000 7fc3b2e00640 1 -- start start 2024-04-24T05:46:28.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.395+0000 7fc3b2e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3ac15a2a0 0x7fc3ac0ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.396+0000 7fc3b2e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc3ac0af950 0x7fc3ac06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.396+0000 7fc3b2e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3ac15cdf0 con 0x7fc3ac0ad690 2024-04-24T05:46:28.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.396+0000 7fc3b2e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc3ac0adaf0 con 0x7fc3ac0af950 2024-04-24T05:46:28.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.396+0000 7fc3b2e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ac15a680 con 0x7fc3ac15a2a0 2024-04-24T05:46:28.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.396+0000 7fc3b0e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc3ac0af950 0x7fc3ac06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:28.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.396+0000 7fc3b0e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc3ac0af950 0x7fc3ac06bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56876/0 (socket says 172.21.15.123:56876) 2024-04-24T05:46:28.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.396+0000 7fc3b0e00640 1 -- 172.21.15.123:0/3715396859 learned_addr learned my addr 172.21.15.123:0/3715396859 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:28.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.396+0000 7fc3abe00640 1 --2- 172.21.15.123:0/3715396859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3ac15a2a0 0x7fc3ac0ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:28.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.396+0000 7fc3b0e00640 1 -- 172.21.15.123:0/3715396859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3ac15a2a0 msgr2=0x7fc3ac0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:28.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.396+0000 7fc3b0e00640 1 --2- 172.21.15.123:0/3715396859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3ac15a2a0 0x7fc3ac0ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.396+0000 7fc3b0e00640 1 -- 172.21.15.123:0/3715396859 >> v1:172.21.15.155:6789/0 conn(0x7fc3ac0ad690 legacy=0x7fc3ac0af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:28.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.396+0000 7fc3b0e00640 1 -- 172.21.15.123:0/3715396859 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac15d0d0 con 0x7fc3ac0af950 2024-04-24T05:46:28.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.397+0000 7fc3b0e00640 1 --2- 172.21.15.123:0/3715396859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3ac0af950 0x7fc3ac06bb00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009e50 tx=0x7fc3a402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8b508c4a7def84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:28.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.398+0000 7fc3aaa00640 1 -- 172.21.15.123:0/3715396859 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4036070 con 0x7fc3ac0af950 2024-04-24T05:46:28.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.398+0000 7fc3aaa00640 1 -- 172.21.15.123:0/3715396859 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a4031600 con 0x7fc3ac0af950 2024-04-24T05:46:28.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.398+0000 7fc3aaa00640 1 -- 172.21.15.123:0/3715396859 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40319d0 con 0x7fc3ac0af950 2024-04-24T05:46:28.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.398+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3715396859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3ac0af950 msgr2=0x7fc3ac06bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:28.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.398+0000 7fc3b2e00640 1 --2- 172.21.15.123:0/3715396859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3ac0af950 0x7fc3ac06bb00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009e50 tx=0x7fc3a402a630 comp rx=0 tx=0).stop 2024-04-24T05:46:28.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.398+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3715396859 shutdown_connections 2024-04-24T05:46:28.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.398+0000 7fc3b2e00640 1 --2- 172.21.15.123:0/3715396859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3ac0af950 0x7fc3ac06bb00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.398+0000 7fc3b2e00640 1 --2- 172.21.15.123:0/3715396859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3ac15a2a0 0x7fc3ac0ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.398+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3715396859 >> 172.21.15.123:0/3715396859 conn(0x7fc3ac0b1e70 msgr2=0x7fc3ac0b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:28.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.398+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3715396859 shutdown_connections 2024-04-24T05:46:28.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.399+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3715396859 wait complete. 2024-04-24T05:46:28.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.400+0000 7fc3b2e00640 1 Processor -- start 2024-04-24T05:46:28.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.400+0000 7fc3b2e00640 1 -- start start 2024-04-24T05:46:28.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.401+0000 7fc3b2e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3ac0ad690 0x7fc3ac14fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.401+0000 7fc3b2e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3ac0af950 0x7fc3ac06e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.401+0000 7fc3b2e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ac15a2a0 0x7fc3ac06eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.401+0000 7fc3b2e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac15d690 con 0x7fc3ac0af950 2024-04-24T05:46:28.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.401+0000 7fc3b2e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac15da00 con 0x7fc3ac15a2a0 2024-04-24T05:46:28.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.401+0000 7fc3b2e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ac06c040 con 0x7fc3ac0ad690 2024-04-24T05:46:28.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.402+0000 7fc3ab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3ac0af950 0x7fc3ac06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:28.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.402+0000 7fc3ab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3ac0af950 0x7fc3ac06e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56878/0 (socket says 172.21.15.123:56878) 2024-04-24T05:46:28.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.402+0000 7fc3ab400640 1 -- 172.21.15.123:0/3660789929 learned_addr learned my addr 172.21.15.123:0/3660789929 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:28.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.402+0000 7fc3b0e00640 1 --2- 172.21.15.123:0/3660789929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ac15a2a0 0x7fc3ac06eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:28.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.402+0000 7fc3abe00640 1 --2- 172.21.15.123:0/3660789929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3ac0ad690 0x7fc3ac14fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:28.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.402+0000 7fc3ab400640 1 -- 172.21.15.123:0/3660789929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3ac0ad690 msgr2=0x7fc3ac14fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:28.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.402+0000 7fc3ab400640 1 --2- 172.21.15.123:0/3660789929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3ac0ad690 0x7fc3ac14fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.402+0000 7fc3ab400640 1 -- 172.21.15.123:0/3660789929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ac15a2a0 msgr2=0x7fc3ac06eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:28.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.402+0000 7fc3ab400640 1 --2- 172.21.15.123:0/3660789929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ac15a2a0 0x7fc3ac06eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.402+0000 7fc3ab400640 1 -- 172.21.15.123:0/3660789929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac1f4d90 con 0x7fc3ac0af950 2024-04-24T05:46:28.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.402+0000 7fc3ab400640 1 --2- 172.21.15.123:0/3660789929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3ac0af950 0x7fc3ac06e630 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc39800c460 tx=0x7fc39800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:28.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.403+0000 7fc3a3e00640 1 -- 172.21.15.123:0/3660789929 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398016020 con 0x7fc3ac0af950 2024-04-24T05:46:28.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.403+0000 7fc3a3e00640 1 -- 172.21.15.123:0/3660789929 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc398012870 con 0x7fc3ac0af950 2024-04-24T05:46:28.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.403+0000 7fc3a3e00640 1 -- 172.21.15.123:0/3660789929 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398012c60 con 0x7fc3ac0af950 2024-04-24T05:46:28.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.403+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3660789929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ac1f5070 con 0x7fc3ac0af950 2024-04-24T05:46:28.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.403+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3660789929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ac1f5630 con 0x7fc3ac0af950 2024-04-24T05:46:28.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.404+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3660789929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ac068a20 con 0x7fc3ac0af950 2024-04-24T05:46:28.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.405+0000 7fc3a3e00640 1 -- 172.21.15.123:0/3660789929 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc398019020 con 0x7fc3ac0af950 2024-04-24T05:46:28.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.406+0000 7fc3a3e00640 1 --2- 172.21.15.123:0/3660789929 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc38c06c8f0 0x7fc38c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.407+0000 7fc3a3e00640 1 -- 172.21.15.123:0/3660789929 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc39808bda0 con 0x7fc3ac0af950 2024-04-24T05:46:28.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.407+0000 7fc3abe00640 1 --2- 172.21.15.123:0/3660789929 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc38c06c8f0 0x7fc38c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:28.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.407+0000 7fc3abe00640 1 --2- 172.21.15.123:0/3660789929 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc38c06c8f0 0x7fc38c06edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc39c006fd0 tx=0x7fc39c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:28.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.411+0000 7fc3a3e00640 1 -- 172.21.15.123:0/3660789929 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc398020020 con 0x7fc3ac0af950 2024-04-24T05:46:28.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.622+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3660789929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3ac14c310 con 0x7fc3ac0af950 2024-04-24T05:46:28.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.625+0000 7fc3a3e00640 1 -- 172.21.15.123:0/3660789929 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7fc398059f20 con 0x7fc3ac0af950 2024-04-24T05:46:28.627 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:28.628 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:04.715857+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.628+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3660789929 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc38c06c8f0 msgr2=0x7fc38c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.628+0000 7fc3b2e00640 1 --2- 172.21.15.123:0/3660789929 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc38c06c8f0 0x7fc38c06edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc39c006fd0 tx=0x7fc39c008040 comp rx=0 tx=0).stop 2024-04-24T05:46:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.628+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3660789929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3ac0af950 msgr2=0x7fc3ac06e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.628+0000 7fc3b2e00640 1 --2- 172.21.15.123:0/3660789929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3ac0af950 0x7fc3ac06e630 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc39800c460 tx=0x7fc39800c920 comp rx=0 tx=0).stop 2024-04-24T05:46:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.628+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3660789929 shutdown_connections 2024-04-24T05:46:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.628+0000 7fc3b2e00640 1 --2- 172.21.15.123:0/3660789929 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc38c06c8f0 0x7fc38c06edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.628+0000 7fc3b2e00640 1 --2- 172.21.15.123:0/3660789929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3ac15a2a0 0x7fc3ac06eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.628+0000 7fc3b2e00640 1 --2- 172.21.15.123:0/3660789929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3ac0af950 0x7fc3ac06e630 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.628+0000 7fc3b2e00640 1 --2- 172.21.15.123:0/3660789929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3ac0ad690 0x7fc3ac14fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.628+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3660789929 >> 172.21.15.123:0/3660789929 conn(0x7fc3ac0b1e70 msgr2=0x7fc3ac14cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:28.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.629+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3660789929 shutdown_connections 2024-04-24T05:46:28.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.629+0000 7fc3b2e00640 1 -- 172.21.15.123:0/3660789929 wait complete. 2024-04-24T05:46:28.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:46:28.648 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:46:28.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.851+0000 7f44f7200640 1 Processor -- start 2024-04-24T05:46:28.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.851+0000 7f44f7200640 1 -- start start 2024-04-24T05:46:28.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.853+0000 7f44f7200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f015c2f0 0x7f44f0141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.853+0000 7f44f7200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f44f00ad6e0 0x7f44f006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.853+0000 7f44f7200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44f006ba00 con 0x7f44f00cc5d0 2024-04-24T05:46:28.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.853+0000 7f44f7200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f44f006bd30 con 0x7f44f00ad6e0 2024-04-24T05:46:28.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.853+0000 7f44f7200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f44f0142530 con 0x7f44f015c2f0 2024-04-24T05:46:28.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.853+0000 7f44f5400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f44f00cc5d0 0x7f44f00acfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49270/0 (socket says 172.21.15.123:49270) 2024-04-24T05:46:28.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.853+0000 7f44f5400640 1 -- 172.21.15.123:0/1795260847 learned_addr learned my addr 172.21.15.123:0/1795260847 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:28.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.853+0000 7f44f6800640 1 --2- 172.21.15.123:0/1795260847 >> v2:172.21.15.155:3300/0 conn(0x7f44f00ad6e0 0x7f44f006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:28.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.853+0000 7f44f5e00640 1 --2- 172.21.15.123:0/1795260847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f015c2f0 0x7f44f0141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:28.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.854+0000 7f44f4a00640 1 -- 172.21.15.123:0/1795260847 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1118337721 0 0) 0x7f44f006ba00 con 0x7f44f00cc5d0 2024-04-24T05:46:28.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.854+0000 7f44f4a00640 1 -- 172.21.15.123:0/1795260847 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44d8003520 con 0x7f44f00cc5d0 2024-04-24T05:46:28.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.854+0000 7f44f4a00640 1 -- 172.21.15.123:0/1795260847 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 17014276 0 0) 0x7f44d8003520 con 0x7f44f00cc5d0 2024-04-24T05:46:28.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.854+0000 7f44f4a00640 1 -- 172.21.15.123:0/1795260847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f015c2f0 msgr2=0x7f44f0141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:28.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.854+0000 7f44f4a00640 1 --2- 172.21.15.123:0/1795260847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f015c2f0 0x7f44f0141ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.854+0000 7f44f4a00640 1 -- 172.21.15.123:0/1795260847 >> v2:172.21.15.155:3300/0 conn(0x7f44f00ad6e0 msgr2=0x7f44f006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:28.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.854+0000 7f44f4a00640 1 --2- 172.21.15.123:0/1795260847 >> v2:172.21.15.155:3300/0 conn(0x7f44f00ad6e0 0x7f44f006c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.854+0000 7f44f4a00640 1 -- 172.21.15.123:0/1795260847 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f0142810 con 0x7f44f00cc5d0 2024-04-24T05:46:28.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.854+0000 7f44f4a00640 1 -- 172.21.15.123:0/1795260847 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f44dc001670 con 0x7f44f00cc5d0 2024-04-24T05:46:28.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.854+0000 7f44f4a00640 1 -- 172.21.15.123:0/1795260847 >> v1:172.21.15.123:6789/0 conn(0x7f44f00cc5d0 legacy=0x7f44f00acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:28.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.854+0000 7f44f4a00640 1 --2- 172.21.15.123:0/1795260847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d80039b0 0x7f44d8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.854+0000 7f44f4a00640 1 --2- 172.21.15.123:0/1795260847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f015c2f0 0x7f44d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.854+0000 7f44f4a00640 1 --2- 172.21.15.123:0/1795260847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d8004d40 0x7f44d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.854+0000 7f44f4a00640 1 -- 172.21.15.123:0/1795260847 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44f006bd30 con 0x7f44d80039b0 2024-04-24T05:46:28.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.855+0000 7f44f4a00640 1 -- 172.21.15.123:0/1795260847 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f44f0142530 con 0x7f44d8004d40 2024-04-24T05:46:28.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.855+0000 7f44f4a00640 1 -- 172.21.15.123:0/1795260847 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f44d800b9c0 con 0x7f44f015c2f0 2024-04-24T05:46:28.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.855+0000 7f44f5e00640 1 --2- 172.21.15.123:0/1795260847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d80039b0 0x7f44d8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:28.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.855+0000 7f44f5e00640 1 -- 172.21.15.123:0/1795260847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f015c2f0 msgr2=0x7f44d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:28.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.855+0000 7f44f5e00640 1 --2- 172.21.15.123:0/1795260847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f015c2f0 0x7f44d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.855+0000 7f44f5e00640 1 -- 172.21.15.123:0/1795260847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d8004d40 msgr2=0x7f44d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:28.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.855+0000 7f44f5e00640 1 --2- 172.21.15.123:0/1795260847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44d8004d40 0x7f44d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.855+0000 7f44f5e00640 1 -- 172.21.15.123:0/1795260847 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d8003520 con 0x7f44d80039b0 2024-04-24T05:46:28.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.856+0000 7f44f5e00640 1 --2- 172.21.15.123:0/1795260847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d80039b0 0x7f44d8003df0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009ac0 tx=0x7f44d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948c177620828e06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:28.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.856+0000 7f44f4a00640 1 -- 172.21.15.123:0/1795260847 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4036070 con 0x7f44d80039b0 2024-04-24T05:46:28.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.856+0000 7f44f4a00640 1 -- 172.21.15.123:0/1795260847 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e4031400 con 0x7f44d80039b0 2024-04-24T05:46:28.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.856+0000 7f44f4a00640 1 -- 172.21.15.123:0/1795260847 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4009690 con 0x7f44d80039b0 2024-04-24T05:46:28.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.857+0000 7f44f7200640 1 -- 172.21.15.123:0/1795260847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d80039b0 msgr2=0x7f44d8003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:28.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.857+0000 7f44f7200640 1 --2- 172.21.15.123:0/1795260847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d80039b0 0x7f44d8003df0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009ac0 tx=0x7f44d8005830 comp rx=0 tx=0).stop 2024-04-24T05:46:28.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.857+0000 7f44f6800640 1 -- 172.21.15.123:0/1795260847 reap_dead start 2024-04-24T05:46:28.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.857+0000 7f44f7200640 1 -- 172.21.15.123:0/1795260847 shutdown_connections 2024-04-24T05:46:28.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.857+0000 7f44f7200640 1 -- 172.21.15.123:0/1795260847 >> 172.21.15.123:0/1795260847 conn(0x7f44f00b1cf0 msgr2=0x7f44f00b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:28.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.857+0000 7f44f7200640 1 -- 172.21.15.123:0/1795260847 shutdown_connections 2024-04-24T05:46:28.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.857+0000 7f44f7200640 1 -- 172.21.15.123:0/1795260847 wait complete. 2024-04-24T05:46:28.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.859+0000 7f44f7200640 1 Processor -- start 2024-04-24T05:46:28.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.859+0000 7f44f7200640 1 -- start start 2024-04-24T05:46:28.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.860+0000 7f44f7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d80039b0 0x7f44f0156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.860+0000 7f44f7200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44f00cb800 0x7f44f006c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.860+0000 7f44f7200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f00cbc80 0x7f44f006c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.860+0000 7f44f7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44f015c850 con 0x7f44d80039b0 2024-04-24T05:46:28.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.860+0000 7f44f7200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f44f015c6d0 con 0x7f44f00cb800 2024-04-24T05:46:28.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.860+0000 7f44f7200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f44f0141ce0 con 0x7f44f00cbc80 2024-04-24T05:46:28.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.860+0000 7f44f5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d80039b0 0x7f44f0156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:28.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.860+0000 7f44f5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d80039b0 0x7f44f0156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50192/0 (socket says 172.21.15.123:50192) 2024-04-24T05:46:28.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.860+0000 7f44f5e00640 1 -- 172.21.15.123:0/345852975 learned_addr learned my addr 172.21.15.123:0/345852975 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:28.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.861+0000 7f44f5400640 1 --2- 172.21.15.123:0/345852975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44f00cb800 0x7f44f006c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:28.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.861+0000 7f44f5e00640 1 -- 172.21.15.123:0/345852975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f00cbc80 msgr2=0x7f44f006c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:28.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.861+0000 7f44f6800640 1 --2- 172.21.15.123:0/345852975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f00cbc80 0x7f44f006c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:28.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.861+0000 7f44f5e00640 1 --2- 172.21.15.123:0/345852975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f00cbc80 0x7f44f006c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.861+0000 7f44f5e00640 1 -- 172.21.15.123:0/345852975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44f00cb800 msgr2=0x7f44f006c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:28.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.861+0000 7f44f5e00640 1 --2- 172.21.15.123:0/345852975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44f00cb800 0x7f44f006c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:28.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.861+0000 7f44f5e00640 1 -- 172.21.15.123:0/345852975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f0156650 con 0x7f44d80039b0 2024-04-24T05:46:28.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.861+0000 7f44f5e00640 1 --2- 172.21.15.123:0/345852975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d80039b0 0x7f44f0156110 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009be0 tx=0x7f44e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:28.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.861+0000 7f44ea000640 1 -- 172.21.15.123:0/345852975 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4040020 con 0x7f44d80039b0 2024-04-24T05:46:28.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.861+0000 7f44ea000640 1 -- 172.21.15.123:0/345852975 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e4003bc0 con 0x7f44d80039b0 2024-04-24T05:46:28.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.861+0000 7f44ea000640 1 -- 172.21.15.123:0/345852975 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4036040 con 0x7f44d80039b0 2024-04-24T05:46:28.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.861+0000 7f44f7200640 1 -- 172.21.15.123:0/345852975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f00cdf90 con 0x7f44d80039b0 2024-04-24T05:46:28.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.861+0000 7f44f7200640 1 -- 172.21.15.123:0/345852975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f00ce550 con 0x7f44d80039b0 2024-04-24T05:46:28.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.863+0000 7f44f7200640 1 -- 172.21.15.123:0/345852975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f01420c0 con 0x7f44d80039b0 2024-04-24T05:46:28.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.863+0000 7f44ea000640 1 -- 172.21.15.123:0/345852975 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44e4043020 con 0x7f44d80039b0 2024-04-24T05:46:28.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.864+0000 7f44ea000640 1 --2- 172.21.15.123:0/345852975 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f44d006c8f0 0x7f44d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:28.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.864+0000 7f44ea000640 1 -- 172.21.15.123:0/345852975 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f44e40abff0 con 0x7f44d80039b0 2024-04-24T05:46:28.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.865+0000 7f44f5400640 1 --2- 172.21.15.123:0/345852975 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f44d006c8f0 0x7f44d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:28.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.865+0000 7f44f5400640 1 --2- 172.21.15.123:0/345852975 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f44d006c8f0 0x7f44d006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f44dc000f60 tx=0x7f44dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:28.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:28.869+0000 7f44ea000640 1 -- 172.21.15.123:0/345852975 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44f01420c0 con 0x7f44d80039b0 2024-04-24T05:46:29.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.067+0000 7f44f7200640 1 -- 172.21.15.123:0/345852975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f44f0068a20 con 0x7f44d80039b0 2024-04-24T05:46:29.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.068+0000 7f44ea000640 1 -- 172.21.15.123:0/345852975 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f44e404a070 con 0x7f44d80039b0 2024-04-24T05:46:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:46:29.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.071+0000 7f44f7200640 1 -- 172.21.15.123:0/345852975 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f44d006c8f0 msgr2=0x7f44d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:29.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.071+0000 7f44f7200640 1 --2- 172.21.15.123:0/345852975 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f44d006c8f0 0x7f44d006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f44dc000f60 tx=0x7f44dc009040 comp rx=0 tx=0).stop 2024-04-24T05:46:29.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.071+0000 7f44f7200640 1 -- 172.21.15.123:0/345852975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d80039b0 msgr2=0x7f44f0156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:29.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.071+0000 7f44f7200640 1 --2- 172.21.15.123:0/345852975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d80039b0 0x7f44f0156110 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009be0 tx=0x7f44e4004730 comp rx=0 tx=0).stop 2024-04-24T05:46:29.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.071+0000 7f44f7200640 1 -- 172.21.15.123:0/345852975 shutdown_connections 2024-04-24T05:46:29.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.071+0000 7f44f7200640 1 --2- 172.21.15.123:0/345852975 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f44d006c8f0 0x7f44d006edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.071+0000 7f44f7200640 1 --2- 172.21.15.123:0/345852975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44f00cbc80 0x7f44f006c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.071+0000 7f44f7200640 1 --2- 172.21.15.123:0/345852975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44f00cb800 0x7f44f006c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.071+0000 7f44f7200640 1 --2- 172.21.15.123:0/345852975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d80039b0 0x7f44f0156110 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.071+0000 7f44f7200640 1 -- 172.21.15.123:0/345852975 >> 172.21.15.123:0/345852975 conn(0x7f44f00b1cf0 msgr2=0x7f44f006ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:29.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.071+0000 7f44f7200640 1 -- 172.21.15.123:0/345852975 shutdown_connections 2024-04-24T05:46:29.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.071+0000 7f44f7200640 1 -- 172.21.15.123:0/345852975 wait complete. 2024-04-24T05:46:29.091 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:46:29.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.292+0000 7f4201a00640 1 Processor -- start 2024-04-24T05:46:29.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.292+0000 7f4201a00640 1 -- start start 2024-04-24T05:46:29.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.293+0000 7f4201a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f41fc0ae990 0x7f41fc0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:29.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.294+0000 7f4201a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f41fc0ce7e0 0x7f41fc06e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:29.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.294+0000 7f4201a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41fc0d0740 con 0x7f41fc06c410 2024-04-24T05:46:29.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.294+0000 7f4201a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f41fc06dcb0 con 0x7f41fc0ce7e0 2024-04-24T05:46:29.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.294+0000 7f4201a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f41fc06e270 con 0x7f41fc0ae990 2024-04-24T05:46:29.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.294+0000 7f41fb400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f41fc0ce7e0 0x7f41fc06e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:29.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.294+0000 7f41faa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f41fc0ae990 0x7f41fc0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:29.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.294+0000 7f41f2000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f41fc06c410 0x7f41fc0ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49276/0 (socket says 172.21.15.123:49276) 2024-04-24T05:46:29.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.294+0000 7f41faa00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f41fc0ae990 0x7f41fc0d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:58850/0 (socket says 172.21.15.123:58850) 2024-04-24T05:46:29.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.294+0000 7f41f2000640 1 -- 172.21.15.123:0/3915439922 learned_addr learned my addr 172.21.15.123:0/3915439922 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:29.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.294+0000 7f41fb400640 1 --2- 172.21.15.123:0/3915439922 >> v2:172.21.15.123:3300/0 conn(0x7f41fc0ce7e0 0x7f41fc06e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50208/0 (socket says 172.21.15.123:50208) 2024-04-24T05:46:29.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.295+0000 7f41faa00640 1 -- 172.21.15.123:0/3915439922 >> v2:172.21.15.123:3300/0 conn(0x7f41fc0ce7e0 msgr2=0x7f41fc06e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:29.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.295+0000 7f41faa00640 1 --2- 172.21.15.123:0/3915439922 >> v2:172.21.15.123:3300/0 conn(0x7f41fc0ce7e0 0x7f41fc06e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.295+0000 7f41faa00640 1 -- 172.21.15.123:0/3915439922 >> v1:172.21.15.123:6789/0 conn(0x7f41fc06c410 legacy=0x7f41fc0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:29.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.295+0000 7f41faa00640 1 -- 172.21.15.123:0/3915439922 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc0d0a20 con 0x7f41fc0ae990 2024-04-24T05:46:29.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.295+0000 7f41fb400640 1 --2- 172.21.15.123:0/3915439922 >> v2:172.21.15.123:3300/0 conn(0x7f41fc0ce7e0 0x7f41fc06e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:46:29.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.296+0000 7f41faa00640 1 --2- 172.21.15.123:0/3915439922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41fc0ae990 0x7f41fc0d0160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f41fc06ff40 tx=0x7f41ec02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cde46c7967b3a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:29.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.297+0000 7f41fa000640 1 -- 172.21.15.123:0/3915439922 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec007740 con 0x7f41fc0ae990 2024-04-24T05:46:29.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.297+0000 7f41fa000640 1 -- 172.21.15.123:0/3915439922 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ec0078e0 con 0x7f41fc0ae990 2024-04-24T05:46:29.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.297+0000 7f41fa000640 1 -- 172.21.15.123:0/3915439922 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec007cb0 con 0x7f41fc0ae990 2024-04-24T05:46:29.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.297+0000 7f4201a00640 1 -- 172.21.15.123:0/3915439922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41fc0ae990 msgr2=0x7f41fc0d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:29.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.297+0000 7f4201a00640 1 --2- 172.21.15.123:0/3915439922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41fc0ae990 0x7f41fc0d0160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f41fc06ff40 tx=0x7f41ec02a680 comp rx=0 tx=0).stop 2024-04-24T05:46:29.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.298+0000 7f4201a00640 1 -- 172.21.15.123:0/3915439922 shutdown_connections 2024-04-24T05:46:29.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.298+0000 7f4201a00640 1 --2- 172.21.15.123:0/3915439922 >> v2:172.21.15.123:3300/0 conn(0x7f41fc0ce7e0 0x7f41fc06e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.298+0000 7f4201a00640 1 --2- 172.21.15.123:0/3915439922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41fc0ae990 0x7f41fc0d0160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.298+0000 7f4201a00640 1 -- 172.21.15.123:0/3915439922 >> 172.21.15.123:0/3915439922 conn(0x7f41fc0b51f0 msgr2=0x7f41fc0b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:29.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.298+0000 7f4201a00640 1 -- 172.21.15.123:0/3915439922 shutdown_connections 2024-04-24T05:46:29.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.298+0000 7f4201a00640 1 -- 172.21.15.123:0/3915439922 wait complete. 2024-04-24T05:46:29.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.300+0000 7f4201a00640 1 Processor -- start 2024-04-24T05:46:29.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.300+0000 7f4201a00640 1 -- start start 2024-04-24T05:46:29.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.301+0000 7f4201a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41fc06c410 0x7f41fc1f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:29.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.301+0000 7f4201a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41fc0ae990 0x7f41fc0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:29.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.301+0000 7f4201a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41fc0ce7e0 0x7f41fc15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.301+0000 7f4201a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc06c870 con 0x7f41fc06c410 2024-04-24T05:46:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.301+0000 7f4201a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc0aed70 con 0x7f41fc0ae990 2024-04-24T05:46:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.301+0000 7f4201a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f41fc0b1cd0 con 0x7f41fc0ce7e0 2024-04-24T05:46:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.301+0000 7f41faa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41fc06c410 0x7f41fc1f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.301+0000 7f41faa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41fc06c410 0x7f41fc1f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50214/0 (socket says 172.21.15.123:50214) 2024-04-24T05:46:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.301+0000 7f41faa00640 1 -- 172.21.15.123:0/3192121120 learned_addr learned my addr 172.21.15.123:0/3192121120 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.302+0000 7f41f2000640 1 --2- 172.21.15.123:0/3192121120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41fc0ae990 0x7f41fc0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.302+0000 7f41faa00640 1 -- 172.21.15.123:0/3192121120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41fc0ce7e0 msgr2=0x7f41fc15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.302+0000 7f41faa00640 1 --2- 172.21.15.123:0/3192121120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41fc0ce7e0 0x7f41fc15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.302+0000 7f41faa00640 1 -- 172.21.15.123:0/3192121120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41fc0ae990 msgr2=0x7f41fc0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.302+0000 7f41faa00640 1 --2- 172.21.15.123:0/3192121120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41fc0ae990 0x7f41fc0c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.302+0000 7f41faa00640 1 -- 172.21.15.123:0/3192121120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc1f34c0 con 0x7f41fc06c410 2024-04-24T05:46:29.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.302+0000 7f41faa00640 1 --2- 172.21.15.123:0/3192121120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41fc06c410 0x7f41fc1f3c60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f41ec02ac70 tx=0x7f41ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:29.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.302+0000 7f41f3400640 1 -- 172.21.15.123:0/3192121120 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec040070 con 0x7f41fc06c410 2024-04-24T05:46:29.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.303+0000 7f41f3400640 1 -- 172.21.15.123:0/3192121120 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ec031c40 con 0x7f41fc06c410 2024-04-24T05:46:29.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.303+0000 7f4201a00640 1 -- 172.21.15.123:0/3192121120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41fc15c980 con 0x7f41fc06c410 2024-04-24T05:46:29.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.303+0000 7f41f3400640 1 -- 172.21.15.123:0/3192121120 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec009450 con 0x7f41fc06c410 2024-04-24T05:46:29.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.303+0000 7f4201a00640 1 -- 172.21.15.123:0/3192121120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41fc15cf40 con 0x7f41fc06c410 2024-04-24T05:46:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.304+0000 7f4201a00640 1 -- 172.21.15.123:0/3192121120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41fc068a20 con 0x7f41fc06c410 2024-04-24T05:46:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.305+0000 7f41f3400640 1 -- 172.21.15.123:0/3192121120 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41ec039040 con 0x7f41fc06c410 2024-04-24T05:46:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.306+0000 7f41f3400640 1 --2- 172.21.15.123:0/3192121120 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f41d806c8f0 0x7f41d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.306+0000 7f41f3400640 1 -- 172.21.15.123:0/3192121120 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f41ec0ad6a0 con 0x7f41fc06c410 2024-04-24T05:46:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.306+0000 7f41f2000640 1 --2- 172.21.15.123:0/3192121120 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f41d806c8f0 0x7f41d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:29.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.307+0000 7f41f2000640 1 --2- 172.21.15.123:0/3192121120 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f41d806c8f0 0x7f41d806edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f41e4005880 tx=0x7f41e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:29.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.310+0000 7f41f3400640 1 -- 172.21.15.123:0/3192121120 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f41ec07b940 con 0x7f41fc06c410 2024-04-24T05:46:29.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.514+0000 7f4201a00640 1 -- 172.21.15.123:0/3192121120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f41fc0008d0 con 0x7f41fc06c410 2024-04-24T05:46:29.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.514+0000 7f41f3400640 1 -- 172.21.15.123:0/3192121120 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f41ec07b240 con 0x7f41fc06c410 2024-04-24T05:46:29.516 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:29.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.517+0000 7f4201a00640 1 -- 172.21.15.123:0/3192121120 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f41d806c8f0 msgr2=0x7f41d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:29.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.517+0000 7f4201a00640 1 --2- 172.21.15.123:0/3192121120 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f41d806c8f0 0x7f41d806edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f41e4005880 tx=0x7f41e4009040 comp rx=0 tx=0).stop 2024-04-24T05:46:29.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.517+0000 7f4201a00640 1 -- 172.21.15.123:0/3192121120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41fc06c410 msgr2=0x7f41fc1f3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:29.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.517+0000 7f4201a00640 1 --2- 172.21.15.123:0/3192121120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41fc06c410 0x7f41fc1f3c60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f41ec02ac70 tx=0x7f41ec004a40 comp rx=0 tx=0).stop 2024-04-24T05:46:29.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.517+0000 7f4201a00640 1 -- 172.21.15.123:0/3192121120 shutdown_connections 2024-04-24T05:46:29.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.517+0000 7f4201a00640 1 --2- 172.21.15.123:0/3192121120 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f41d806c8f0 0x7f41d806edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.517+0000 7f4201a00640 1 --2- 172.21.15.123:0/3192121120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41fc0ce7e0 0x7f41fc15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.517+0000 7f4201a00640 1 --2- 172.21.15.123:0/3192121120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41fc0ae990 0x7f41fc0c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.517+0000 7f4201a00640 1 --2- 172.21.15.123:0/3192121120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41fc06c410 0x7f41fc1f3c60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.517+0000 7f4201a00640 1 -- 172.21.15.123:0/3192121120 >> 172.21.15.123:0/3192121120 conn(0x7f41fc0b51f0 msgr2=0x7f41fc0add40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:29.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.518+0000 7f4201a00640 1 -- 172.21.15.123:0/3192121120 shutdown_connections 2024-04-24T05:46:29.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.518+0000 7f4201a00640 1 -- 172.21.15.123:0/3192121120 wait complete. 2024-04-24T05:46:29.538 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:46:29.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.743+0000 7f4d89800640 1 Processor -- start 2024-04-24T05:46:29.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.743+0000 7f4d89800640 1 -- start start 2024-04-24T05:46:29.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.744+0000 7f4d89800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d84162ee0 0x7f4d840ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:29.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.744+0000 7f4d89800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4d8406dcb0 0x7f4d840c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:29.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.744+0000 7f4d89800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4d840c36e0 0x7f4d84166950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:29.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.744+0000 7f4d89800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4d840afa00 con 0x7f4d8406dcb0 2024-04-24T05:46:29.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.744+0000 7f4d89800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4d841632c0 con 0x7f4d840c36e0 2024-04-24T05:46:29.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.744+0000 7f4d89800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4d840ce810 con 0x7f4d84162ee0 2024-04-24T05:46:29.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.745+0000 7f4d7a000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4d8406dcb0 0x7f4d840c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:29.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.745+0000 7f4d7a000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4d8406dcb0 0x7f4d840c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50222/0 (socket says 172.21.15.123:50222) 2024-04-24T05:46:29.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.745+0000 7f4d7a000640 1 -- 172.21.15.123:0/2703424422 learned_addr learned my addr 172.21.15.123:0/2703424422 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:29.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.745+0000 7f4d82a00640 1 --2- 172.21.15.123:0/2703424422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d84162ee0 0x7f4d840ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:29.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.745+0000 7f4d83400640 1 --2- 172.21.15.123:0/2703424422 >> v2:172.21.15.155:3300/0 conn(0x7f4d840c36e0 0x7f4d84166950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:29.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.745+0000 7f4d7a000640 1 -- 172.21.15.123:0/2703424422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d84162ee0 msgr2=0x7f4d840ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:29.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.745+0000 7f4d7a000640 1 --2- 172.21.15.123:0/2703424422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d84162ee0 0x7f4d840ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.745+0000 7f4d7a000640 1 -- 172.21.15.123:0/2703424422 >> v2:172.21.15.155:3300/0 conn(0x7f4d840c36e0 msgr2=0x7f4d84166950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:29.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.745+0000 7f4d7a000640 1 --2- 172.21.15.123:0/2703424422 >> v2:172.21.15.155:3300/0 conn(0x7f4d840c36e0 0x7f4d84166950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.745+0000 7f4d7a000640 1 -- 172.21.15.123:0/2703424422 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d840cea60 con 0x7f4d8406dcb0 2024-04-24T05:46:29.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.745+0000 7f4d82a00640 1 --2- 172.21.15.123:0/2703424422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d84162ee0 0x7f4d840ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:29.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.746+0000 7f4d7a000640 1 --2- 172.21.15.123:0/2703424422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d8406dcb0 0x7f4d840c31a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4d8406e110 tx=0x7f4d6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0888a48465620f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:29.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.746+0000 7f4d82000640 1 -- 172.21.15.123:0/2703424422 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c037070 con 0x7f4d8406dcb0 2024-04-24T05:46:29.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.746+0000 7f4d82000640 1 -- 172.21.15.123:0/2703424422 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d6c032440 con 0x7f4d8406dcb0 2024-04-24T05:46:29.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.746+0000 7f4d82000640 1 -- 172.21.15.123:0/2703424422 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c032810 con 0x7f4d8406dcb0 2024-04-24T05:46:29.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.747+0000 7f4d89800640 1 -- 172.21.15.123:0/2703424422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d8406dcb0 msgr2=0x7f4d840c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:29.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.747+0000 7f4d89800640 1 --2- 172.21.15.123:0/2703424422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d8406dcb0 0x7f4d840c31a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4d8406e110 tx=0x7f4d6c02a1e0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.747+0000 7f4d89800640 1 -- 172.21.15.123:0/2703424422 shutdown_connections 2024-04-24T05:46:29.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.747+0000 7f4d89800640 1 --2- 172.21.15.123:0/2703424422 >> v2:172.21.15.155:3300/0 conn(0x7f4d840c36e0 0x7f4d84166950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.747+0000 7f4d89800640 1 --2- 172.21.15.123:0/2703424422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d8406dcb0 0x7f4d840c31a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.747+0000 7f4d89800640 1 --2- 172.21.15.123:0/2703424422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d84162ee0 0x7f4d840ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.747+0000 7f4d89800640 1 -- 172.21.15.123:0/2703424422 >> 172.21.15.123:0/2703424422 conn(0x7f4d840b51f0 msgr2=0x7f4d840b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:29.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.747+0000 7f4d89800640 1 -- 172.21.15.123:0/2703424422 shutdown_connections 2024-04-24T05:46:29.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.747+0000 7f4d89800640 1 -- 172.21.15.123:0/2703424422 wait complete. 2024-04-24T05:46:29.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.749+0000 7f4d89800640 1 Processor -- start 2024-04-24T05:46:29.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.749+0000 7f4d89800640 1 -- start start 2024-04-24T05:46:29.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.750+0000 7f4d89800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d8406dcb0 0x7f4d840c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:29.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.750+0000 7f4d89800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d840c36e0 0x7f4d840c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:29.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.750+0000 7f4d89800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d84162ee0 0x7f4d840c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:29.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.750+0000 7f4d89800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d840cf300 con 0x7f4d84162ee0 2024-04-24T05:46:29.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.750+0000 7f4d89800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4d840cf020 con 0x7f4d840c36e0 2024-04-24T05:46:29.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.750+0000 7f4d89800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4d840cf5e0 con 0x7f4d8406dcb0 2024-04-24T05:46:29.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.750+0000 7f4d83400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d84162ee0 0x7f4d840c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:29.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.750+0000 7f4d83400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d84162ee0 0x7f4d840c3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50228/0 (socket says 172.21.15.123:50228) 2024-04-24T05:46:29.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.750+0000 7f4d83400640 1 -- 172.21.15.123:0/4015841220 learned_addr learned my addr 172.21.15.123:0/4015841220 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:29.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.751+0000 7f4d7a000640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d840c36e0 0x7f4d840c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:29.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.751+0000 7f4d82a00640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d8406dcb0 0x7f4d840c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:29.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.751+0000 7f4d83400640 1 -- 172.21.15.123:0/4015841220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d8406dcb0 msgr2=0x7f4d840c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:29.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.751+0000 7f4d83400640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d8406dcb0 0x7f4d840c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.751+0000 7f4d83400640 1 -- 172.21.15.123:0/4015841220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d840c36e0 msgr2=0x7f4d840c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:29.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.751+0000 7f4d83400640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d840c36e0 0x7f4d840c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.751+0000 7f4d83400640 1 -- 172.21.15.123:0/4015841220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d840c8980 con 0x7f4d84162ee0 2024-04-24T05:46:29.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.751+0000 7f4d7a000640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d840c36e0 0x7f4d840c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:29.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.751+0000 7f4d83400640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d84162ee0 0x7f4d840c3d00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00ccf0 tx=0x7f4d7c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:29.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.751+0000 7f4d7b400640 1 -- 172.21.15.123:0/4015841220 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c003b90 con 0x7f4d84162ee0 2024-04-24T05:46:29.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.751+0000 7f4d7b400640 1 -- 172.21.15.123:0/4015841220 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c003d30 con 0x7f4d84162ee0 2024-04-24T05:46:29.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.751+0000 7f4d7b400640 1 -- 172.21.15.123:0/4015841220 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c004240 con 0x7f4d84162ee0 2024-04-24T05:46:29.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.752+0000 7f4d89800640 1 -- 172.21.15.123:0/4015841220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d840c8f10 con 0x7f4d84162ee0 2024-04-24T05:46:29.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.752+0000 7f4d89800640 1 -- 172.21.15.123:0/4015841220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d840c94d0 con 0x7f4d84162ee0 2024-04-24T05:46:29.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.753+0000 7f4d89800640 1 -- 172.21.15.123:0/4015841220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d84068a20 con 0x7f4d84162ee0 2024-04-24T05:46:29.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.754+0000 7f4d7b400640 1 -- 172.21.15.123:0/4015841220 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d7c0043e0 con 0x7f4d84162ee0 2024-04-24T05:46:29.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.755+0000 7f4d7b400640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4d6006c8f0 0x7f4d6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:29.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.755+0000 7f4d7b400640 1 -- 172.21.15.123:0/4015841220 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4d7c08c980 con 0x7f4d84162ee0 2024-04-24T05:46:29.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.755+0000 7f4d82a00640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4d6006c8f0 0x7f4d6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:29.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.756+0000 7f4d82a00640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4d6006c8f0 0x7f4d6006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400b840 tx=0x7f4d74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:29.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.759+0000 7f4d7b400640 1 -- 172.21.15.123:0/4015841220 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d7c05a9d0 con 0x7f4d84162ee0 2024-04-24T05:46:29.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.956+0000 7f4d89800640 1 -- 172.21.15.123:0/4015841220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4d840d0790 con 0x7f4d84162ee0 2024-04-24T05:46:29.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.956+0000 7f4d7b400640 1 -- 172.21.15.123:0/4015841220 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f4d7c05a2d0 con 0x7f4d84162ee0 2024-04-24T05:46:29.958 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:46:29.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.959+0000 7f4d89800640 1 -- 172.21.15.123:0/4015841220 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4d6006c8f0 msgr2=0x7f4d6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:29.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.959+0000 7f4d89800640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4d6006c8f0 0x7f4d6006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400b840 tx=0x7f4d74008040 comp rx=0 tx=0).stop 2024-04-24T05:46:29.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.959+0000 7f4d89800640 1 -- 172.21.15.123:0/4015841220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d84162ee0 msgr2=0x7f4d840c3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:29.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.959+0000 7f4d89800640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d84162ee0 0x7f4d840c3d00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00ccf0 tx=0x7f4d7c00a660 comp rx=0 tx=0).stop 2024-04-24T05:46:29.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.959+0000 7f4d89800640 1 -- 172.21.15.123:0/4015841220 shutdown_connections 2024-04-24T05:46:29.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.959+0000 7f4d89800640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4d6006c8f0 0x7f4d6006edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.959+0000 7f4d89800640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d84162ee0 0x7f4d840c3d00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.959+0000 7f4d89800640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d840c36e0 0x7f4d840c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.959+0000 7f4d89800640 1 --2- 172.21.15.123:0/4015841220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d8406dcb0 0x7f4d840c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:29.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.959+0000 7f4d89800640 1 -- 172.21.15.123:0/4015841220 >> 172.21.15.123:0/4015841220 conn(0x7f4d840b51f0 msgr2=0x7f4d840adc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:29.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.959+0000 7f4d89800640 1 -- 172.21.15.123:0/4015841220 shutdown_connections 2024-04-24T05:46:29.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:29.959+0000 7f4d89800640 1 -- 172.21.15.123:0/4015841220 wait complete. 2024-04-24T05:46:29.980 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:46:30.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.174+0000 7f8c41400640 1 Processor -- start 2024-04-24T05:46:30.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.175+0000 7f8c41400640 1 -- start start 2024-04-24T05:46:30.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.176+0000 7f8c41400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8c3c15b7a0 0x7f8c3c0d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:30.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.176+0000 7f8c41400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c3c0ce920 con 0x7f8c3c06d9b0 2024-04-24T05:46:30.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.176+0000 7f8c41400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c3c0ceba0 con 0x7f8c3c0700d0 2024-04-24T05:46:30.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.176+0000 7f8c41400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8c3c0d0960 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.176+0000 7f8c3a800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8c3c15b7a0 0x7f8c3c0d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:30.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.177+0000 7f8c3a800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8c3c15b7a0 0x7f8c3c0d0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50236/0 (socket says 172.21.15.123:50236) 2024-04-24T05:46:30.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.177+0000 7f8c3a800640 1 -- 172.21.15.123:0/1631157318 learned_addr learned my addr 172.21.15.123:0/1631157318 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:30.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.177+0000 7f8c3a800640 1 -- 172.21.15.123:0/1631157318 >> v1:172.21.15.155:6789/0 conn(0x7f8c3c0700d0 legacy=0x7f8c3c06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:30.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.177+0000 7f8c3a800640 1 -- 172.21.15.123:0/1631157318 >> v1:172.21.15.123:6789/0 conn(0x7f8c3c06d9b0 legacy=0x7f8c3c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:30.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.177+0000 7f8c3a800640 1 -- 172.21.15.123:0/1631157318 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3c0cee80 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.178+0000 7f8c3a800640 1 --2- 172.21.15.123:0/1631157318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c15b7a0 0x7f8c3c0d0420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8c24009870 tx=0x7f8c2402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18aea5f401682f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:30.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.179+0000 7f8c39400640 1 -- 172.21.15.123:0/1631157318 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24037070 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.179+0000 7f8c39400640 1 -- 172.21.15.123:0/1631157318 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c24032440 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.179+0000 7f8c39400640 1 -- 172.21.15.123:0/1631157318 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24032810 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.179+0000 7f8c41400640 1 -- 172.21.15.123:0/1631157318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c15b7a0 msgr2=0x7f8c3c0d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:30.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.179+0000 7f8c41400640 1 --2- 172.21.15.123:0/1631157318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c15b7a0 0x7f8c3c0d0420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8c24009870 tx=0x7f8c2402a210 comp rx=0 tx=0).stop 2024-04-24T05:46:30.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.179+0000 7f8c41400640 1 -- 172.21.15.123:0/1631157318 shutdown_connections 2024-04-24T05:46:30.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.179+0000 7f8c41400640 1 --2- 172.21.15.123:0/1631157318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c15b7a0 0x7f8c3c0d0420 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.179+0000 7f8c41400640 1 -- 172.21.15.123:0/1631157318 >> 172.21.15.123:0/1631157318 conn(0x7f8c3c0724e0 msgr2=0x7f8c3c0728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:30.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.179+0000 7f8c41400640 1 -- 172.21.15.123:0/1631157318 shutdown_connections 2024-04-24T05:46:30.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.180+0000 7f8c41400640 1 -- 172.21.15.123:0/1631157318 wait complete. 2024-04-24T05:46:30.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.181+0000 7f8c41400640 1 Processor -- start 2024-04-24T05:46:30.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.181+0000 7f8c41400640 1 -- start start 2024-04-24T05:46:30.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.182+0000 7f8c41400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c3c06d9b0 0x7f8c3c1f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:30.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.182+0000 7f8c41400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c3c0700d0 0x7f8c3c1f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:30.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.182+0000 7f8c41400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c15b7a0 0x7f8c3c146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:30.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.182+0000 7f8c41400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c0cf440 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.182+0000 7f8c41400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c0cf7b0 con 0x7f8c3c06d9b0 2024-04-24T05:46:30.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.182+0000 7f8c41400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c3c0cfc40 con 0x7f8c3c0700d0 2024-04-24T05:46:30.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.183+0000 7f8c3b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c15b7a0 0x7f8c3c146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:30.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.183+0000 7f8c3b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c15b7a0 0x7f8c3c146e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50238/0 (socket says 172.21.15.123:50238) 2024-04-24T05:46:30.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.183+0000 7f8c3b200640 1 -- 172.21.15.123:0/1135887891 learned_addr learned my addr 172.21.15.123:0/1135887891 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:30.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.183+0000 7f8c3a800640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c3c06d9b0 0x7f8c3c1f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:30.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.183+0000 7f8c3b200640 1 -- 172.21.15.123:0/1135887891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c3c0700d0 msgr2=0x7f8c3c1f51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:30.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.183+0000 7f8c39e00640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c3c0700d0 0x7f8c3c1f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:30.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.183+0000 7f8c3b200640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c3c0700d0 0x7f8c3c1f51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.183+0000 7f8c3b200640 1 -- 172.21.15.123:0/1135887891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c3c06d9b0 msgr2=0x7f8c3c1f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:30.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.183+0000 7f8c3b200640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c3c06d9b0 0x7f8c3c1f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.183+0000 7f8c3b200640 1 -- 172.21.15.123:0/1135887891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3c1f21a0 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.183+0000 7f8c39e00640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c3c0700d0 0x7f8c3c1f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:30.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.183+0000 7f8c3b200640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c15b7a0 0x7f8c3c146e20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400c3c0 tx=0x7f8c3400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:30.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.184+0000 7f8c32a00640 1 -- 172.21.15.123:0/1135887891 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c340052d0 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.184+0000 7f8c32a00640 1 -- 172.21.15.123:0/1135887891 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c34005470 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.184+0000 7f8c32a00640 1 -- 172.21.15.123:0/1135887891 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34007240 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.184+0000 7f8c41400640 1 -- 172.21.15.123:0/1135887891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3c1f5710 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.184+0000 7f8c41400640 1 -- 172.21.15.123:0/1135887891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c3c147c40 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.185+0000 7f8c41400640 1 -- 172.21.15.123:0/1135887891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c3c068a20 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.186+0000 7f8c32a00640 1 -- 172.21.15.123:0/1135887891 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c340073e0 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.187+0000 7f8c32a00640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8c1806c920 0x7f8c1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:30.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.187+0000 7f8c32a00640 1 -- 172.21.15.123:0/1135887891 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8c3408b220 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.188+0000 7f8c3a800640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8c1806c920 0x7f8c1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:30.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.188+0000 7f8c3a800640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8c1806c920 0x7f8c1806ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8c24034000 tx=0x7f8c2402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:30.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.192+0000 7f8c32a00640 1 -- 172.21.15.123:0/1135887891 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c3c0ce920 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.390+0000 7f8c41400640 1 -- 172.21.15.123:0/1135887891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8c3c06bd10 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.391+0000 7f8c32a00640 1 -- 172.21.15.123:0/1135887891 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f8c34059300 con 0x7f8c3c15b7a0 2024-04-24T05:46:30.393 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:30.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.395+0000 7f8c41400640 1 -- 172.21.15.123:0/1135887891 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8c1806c920 msgr2=0x7f8c1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:30.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.395+0000 7f8c41400640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8c1806c920 0x7f8c1806ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8c24034000 tx=0x7f8c2402e000 comp rx=0 tx=0).stop 2024-04-24T05:46:30.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.395+0000 7f8c41400640 1 -- 172.21.15.123:0/1135887891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c15b7a0 msgr2=0x7f8c3c146e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:30.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.395+0000 7f8c41400640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c15b7a0 0x7f8c3c146e20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400c3c0 tx=0x7f8c3400c880 comp rx=0 tx=0).stop 2024-04-24T05:46:30.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.396+0000 7f8c41400640 1 -- 172.21.15.123:0/1135887891 shutdown_connections 2024-04-24T05:46:30.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.396+0000 7f8c41400640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8c1806c920 0x7f8c1806ede0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.396+0000 7f8c41400640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c3c15b7a0 0x7f8c3c146e20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.396+0000 7f8c41400640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c3c0700d0 0x7f8c3c1f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.396+0000 7f8c41400640 1 --2- 172.21.15.123:0/1135887891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c3c06d9b0 0x7f8c3c1f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.396+0000 7f8c41400640 1 -- 172.21.15.123:0/1135887891 >> 172.21.15.123:0/1135887891 conn(0x7f8c3c0724e0 msgr2=0x7f8c3c15db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:30.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.396+0000 7f8c41400640 1 -- 172.21.15.123:0/1135887891 shutdown_connections 2024-04-24T05:46:30.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.396+0000 7f8c41400640 1 -- 172.21.15.123:0/1135887891 wait complete. 2024-04-24T05:46:30.416 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T05:46:30.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.605+0000 7f8b63e00640 1 Processor -- start 2024-04-24T05:46:30.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.605+0000 7f8b63e00640 1 -- start start 2024-04-24T05:46:30.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.606+0000 7f8b63e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8b6406e2f0 0x7f8b640cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:30.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.606+0000 7f8b63e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b640b3000 con 0x7f8b6415b5c0 2024-04-24T05:46:30.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.606+0000 7f8b63e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b6406d0c0 con 0x7f8b640b34b0 2024-04-24T05:46:30.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.606+0000 7f8b63e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8b640ce4b0 con 0x7f8b6406e2f0 2024-04-24T05:46:30.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.607+0000 7f8b62a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8b6415b5c0 0x7f8b6406dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49298/0 (socket says 172.21.15.123:49298) 2024-04-24T05:46:30.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.607+0000 7f8b62a00640 1 -- 172.21.15.123:0/1940356119 learned_addr learned my addr 172.21.15.123:0/1940356119 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:30.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.607+0000 7f8b5a000640 1 --2- 172.21.15.123:0/1940356119 >> v2:172.21.15.123:3300/0 conn(0x7f8b6406e2f0 0x7f8b640cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:30.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.607+0000 7f8b62000640 1 -- 172.21.15.123:0/1940356119 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2386266967 0 0) 0x7f8b640b3000 con 0x7f8b6415b5c0 2024-04-24T05:46:30.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.607+0000 7f8b5a000640 1 -- 172.21.15.123:0/1940356119 >> v1:172.21.15.155:6789/0 conn(0x7f8b640b34b0 legacy=0x7f8b640b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:30.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.607+0000 7f8b5a000640 1 -- 172.21.15.123:0/1940356119 >> v1:172.21.15.123:6789/0 conn(0x7f8b6415b5c0 legacy=0x7f8b6406dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:30.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.607+0000 7f8b5a000640 1 -- 172.21.15.123:0/1940356119 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b640ae230 con 0x7f8b6406e2f0 2024-04-24T05:46:30.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.608+0000 7f8b5a000640 1 --2- 172.21.15.123:0/1940356119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6406e2f0 0x7f8b640cdf70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0097a0 tx=0x7f8b4c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a921d6833a588 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:30.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.608+0000 7f8b62000640 1 -- 172.21.15.123:0/1940356119 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c037070 con 0x7f8b6406e2f0 2024-04-24T05:46:30.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.609+0000 7f8b62000640 1 -- 172.21.15.123:0/1940356119 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b4c032440 con 0x7f8b6406e2f0 2024-04-24T05:46:30.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.609+0000 7f8b62000640 1 -- 172.21.15.123:0/1940356119 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c032810 con 0x7f8b6406e2f0 2024-04-24T05:46:30.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.609+0000 7f8b63e00640 1 -- 172.21.15.123:0/1940356119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6406e2f0 msgr2=0x7f8b640cdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:30.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.609+0000 7f8b63e00640 1 --2- 172.21.15.123:0/1940356119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6406e2f0 0x7f8b640cdf70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0097a0 tx=0x7f8b4c02a1e0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.609+0000 7f8b63e00640 1 -- 172.21.15.123:0/1940356119 shutdown_connections 2024-04-24T05:46:30.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.609+0000 7f8b63e00640 1 --2- 172.21.15.123:0/1940356119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6406e2f0 0x7f8b640cdf70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.609+0000 7f8b63e00640 1 -- 172.21.15.123:0/1940356119 >> 172.21.15.123:0/1940356119 conn(0x7f8b640d0070 msgr2=0x7f8b640d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:30.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.609+0000 7f8b63e00640 1 -- 172.21.15.123:0/1940356119 shutdown_connections 2024-04-24T05:46:30.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.609+0000 7f8b63e00640 1 -- 172.21.15.123:0/1940356119 wait complete. 2024-04-24T05:46:30.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.611+0000 7f8b63e00640 1 Processor -- start 2024-04-24T05:46:30.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.611+0000 7f8b63e00640 1 -- start start 2024-04-24T05:46:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.612+0000 7f8b63e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6406e2f0 0x7f8b641533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.612+0000 7f8b63e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b640b34b0 0x7f8b641553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.612+0000 7f8b63e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b6415b5c0 0x7f8b6409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.612+0000 7f8b63e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6406cd20 con 0x7f8b6406e2f0 2024-04-24T05:46:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.612+0000 7f8b63e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8b640ae510 con 0x7f8b640b34b0 2024-04-24T05:46:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.612+0000 7f8b63e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6406cea0 con 0x7f8b6415b5c0 2024-04-24T05:46:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.612+0000 7f8b62a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6406e2f0 0x7f8b641533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.612+0000 7f8b62a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6406e2f0 0x7f8b641533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50254/0 (socket says 172.21.15.123:50254) 2024-04-24T05:46:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.612+0000 7f8b62a00640 1 -- 172.21.15.123:0/436547502 learned_addr learned my addr 172.21.15.123:0/436547502 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:30.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.612+0000 7f8b62a00640 1 -- 172.21.15.123:0/436547502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b6415b5c0 msgr2=0x7f8b6409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:30.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.612+0000 7f8b62a00640 1 --2- 172.21.15.123:0/436547502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b6415b5c0 0x7f8b6409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.612+0000 7f8b62a00640 1 -- 172.21.15.123:0/436547502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b640b34b0 msgr2=0x7f8b641553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:30.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.612+0000 7f8b62a00640 1 --2- 172.21.15.123:0/436547502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b640b34b0 0x7f8b641553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.612+0000 7f8b62a00640 1 -- 172.21.15.123:0/436547502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6414b2e0 con 0x7f8b6406e2f0 2024-04-24T05:46:30.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.613+0000 7f8b62a00640 1 --2- 172.21.15.123:0/436547502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6406e2f0 0x7f8b641533f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400c460 tx=0x7f8b5400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:30.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.613+0000 7f8b5b400640 1 -- 172.21.15.123:0/436547502 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54018070 con 0x7f8b6406e2f0 2024-04-24T05:46:30.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.613+0000 7f8b5b400640 1 -- 172.21.15.123:0/436547502 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b54007500 con 0x7f8b6406e2f0 2024-04-24T05:46:30.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.613+0000 7f8b5b400640 1 -- 172.21.15.123:0/436547502 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54004a20 con 0x7f8b6406e2f0 2024-04-24T05:46:30.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.613+0000 7f8b63e00640 1 -- 172.21.15.123:0/436547502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6414b5c0 con 0x7f8b6406e2f0 2024-04-24T05:46:30.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.613+0000 7f8b63e00640 1 -- 172.21.15.123:0/436547502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b6415d210 con 0x7f8b6406e2f0 2024-04-24T05:46:30.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.614+0000 7f8b63e00640 1 -- 172.21.15.123:0/436547502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b640b6280 con 0x7f8b6406e2f0 2024-04-24T05:46:30.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.615+0000 7f8b5b400640 1 -- 172.21.15.123:0/436547502 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b54004bc0 con 0x7f8b6406e2f0 2024-04-24T05:46:30.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.616+0000 7f8b5b400640 1 --2- 172.21.15.123:0/436547502 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8b4006c8f0 0x7f8b4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:30.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.616+0000 7f8b5b400640 1 -- 172.21.15.123:0/436547502 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8b5408b210 con 0x7f8b6406e2f0 2024-04-24T05:46:30.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.616+0000 7f8b5a000640 1 --2- 172.21.15.123:0/436547502 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8b4006c8f0 0x7f8b4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:30.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.617+0000 7f8b5a000640 1 --2- 172.21.15.123:0/436547502 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8b4006c8f0 0x7f8b4006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c034000 tx=0x7f8b4c02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:30.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.620+0000 7f8b5b400640 1 -- 172.21.15.123:0/436547502 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b640b6280 con 0x7f8b6406e2f0 2024-04-24T05:46:30.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.871+0000 7f8b63e00640 1 -- 172.21.15.123:0/436547502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b64075b70 con 0x7f8b6406e2f0 2024-04-24T05:46:30.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.872+0000 7f8b5b400640 1 -- 172.21.15.123:0/436547502 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f8b54059260 con 0x7f8b6406e2f0 2024-04-24T05:46:30.873 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:30.873 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2743935658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2743935658},{"type":"v1","addr":"172.21.15.155:6834","nonce":2743935658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1148561953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1148561953},{"type":"v1","addr":"172.21.15.123:6836","nonce":1148561953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2992052647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2992052647},{"type":"v1","addr":"172.21.15.155:6835","nonce":2992052647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T05:45:59.639548+0000","modified":"2024-04-24T05:46:19.813408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/3370584410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3370584410},{"type":"v1","addr":"172.21.15.123:6837","nonce":3370584410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T05:46:30.874 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-24T05:46:30.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.874+0000 7f8b63e00640 1 -- 172.21.15.123:0/436547502 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8b4006c8f0 msgr2=0x7f8b4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:30.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.874+0000 7f8b63e00640 1 --2- 172.21.15.123:0/436547502 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8b4006c8f0 0x7f8b4006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c034000 tx=0x7f8b4c02c000 comp rx=0 tx=0).stop 2024-04-24T05:46:30.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.874+0000 7f8b63e00640 1 -- 172.21.15.123:0/436547502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6406e2f0 msgr2=0x7f8b641533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:30.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.874+0000 7f8b63e00640 1 --2- 172.21.15.123:0/436547502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6406e2f0 0x7f8b641533f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400c460 tx=0x7f8b5400c920 comp rx=0 tx=0).stop 2024-04-24T05:46:30.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.875+0000 7f8b63e00640 1 -- 172.21.15.123:0/436547502 shutdown_connections 2024-04-24T05:46:30.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.875+0000 7f8b63e00640 1 --2- 172.21.15.123:0/436547502 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8b4006c8f0 0x7f8b4006edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.875+0000 7f8b63e00640 1 --2- 172.21.15.123:0/436547502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b6415b5c0 0x7f8b6409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.875+0000 7f8b63e00640 1 --2- 172.21.15.123:0/436547502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b640b34b0 0x7f8b641553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.875+0000 7f8b63e00640 1 --2- 172.21.15.123:0/436547502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6406e2f0 0x7f8b641533f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:30.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.875+0000 7f8b63e00640 1 -- 172.21.15.123:0/436547502 >> 172.21.15.123:0/436547502 conn(0x7f8b640d0070 msgr2=0x7f8b6406ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:30.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.875+0000 7f8b63e00640 1 -- 172.21.15.123:0/436547502 shutdown_connections 2024-04-24T05:46:30.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:30.875+0000 7f8b63e00640 1 -- 172.21.15.123:0/436547502 wait complete. 2024-04-24T05:46:30.897 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:31.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.097+0000 7ff63b800640 1 Processor -- start 2024-04-24T05:46:31.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.097+0000 7ff63b800640 1 -- start start 2024-04-24T05:46:31.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.099+0000 7ff63b800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff63415e970 0x7ff6340d44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.099+0000 7ff63b800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff6340d3e10 0x7ff6340d4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.099+0000 7ff63b800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6340cfdf0 con 0x7ff6340b39b0 2024-04-24T05:46:31.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.099+0000 7ff63b800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff6340d09c0 con 0x7ff63415e970 2024-04-24T05:46:31.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.099+0000 7ff63b800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff6340d1e60 con 0x7ff6340d3e10 2024-04-24T05:46:31.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.099+0000 7ff638e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff63415e970 0x7ff6340d44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.099+0000 7ff638e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff63415e970 0x7ff6340d44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50268/0 (socket says 172.21.15.123:50268) 2024-04-24T05:46:31.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.099+0000 7ff638e00640 1 -- 172.21.15.123:0/1973064075 learned_addr learned my addr 172.21.15.123:0/1973064075 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:31.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.100+0000 7ff633e00640 1 --2- 172.21.15.123:0/1973064075 >> v2:172.21.15.155:3300/0 conn(0x7ff6340d3e10 0x7ff6340d4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.100+0000 7ff638e00640 1 -- 172.21.15.123:0/1973064075 >> v2:172.21.15.155:3300/0 conn(0x7ff6340d3e10 msgr2=0x7ff6340d4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:31.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.100+0000 7ff638e00640 1 --2- 172.21.15.123:0/1973064075 >> v2:172.21.15.155:3300/0 conn(0x7ff6340d3e10 0x7ff6340d4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.100+0000 7ff638e00640 1 -- 172.21.15.123:0/1973064075 >> v1:172.21.15.155:6789/0 conn(0x7ff6340b39b0 legacy=0x7ff6340d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:31.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.100+0000 7ff638e00640 1 -- 172.21.15.123:0/1973064075 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6340d0090 con 0x7ff63415e970 2024-04-24T05:46:31.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.101+0000 7ff638e00640 1 --2- 172.21.15.123:0/1973064075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff63415e970 0x7ff6340d44f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff6340c7f20 tx=0x7ff62402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d03ff822ee8651c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:31.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.101+0000 7ff633400640 1 -- 172.21.15.123:0/1973064075 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624036070 con 0x7ff63415e970 2024-04-24T05:46:31.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.101+0000 7ff633400640 1 -- 172.21.15.123:0/1973064075 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff624031600 con 0x7ff63415e970 2024-04-24T05:46:31.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.101+0000 7ff633400640 1 -- 172.21.15.123:0/1973064075 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6240319d0 con 0x7ff63415e970 2024-04-24T05:46:31.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.102+0000 7ff63b800640 1 -- 172.21.15.123:0/1973064075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff63415e970 msgr2=0x7ff6340d44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.102+0000 7ff63b800640 1 --2- 172.21.15.123:0/1973064075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff63415e970 0x7ff6340d44f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff6340c7f20 tx=0x7ff62402a660 comp rx=0 tx=0).stop 2024-04-24T05:46:31.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.102+0000 7ff63b800640 1 -- 172.21.15.123:0/1973064075 shutdown_connections 2024-04-24T05:46:31.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.102+0000 7ff63b800640 1 --2- 172.21.15.123:0/1973064075 >> v2:172.21.15.155:3300/0 conn(0x7ff6340d3e10 0x7ff6340d4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.102+0000 7ff63b800640 1 --2- 172.21.15.123:0/1973064075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff63415e970 0x7ff6340d44f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.102+0000 7ff63b800640 1 -- 172.21.15.123:0/1973064075 >> 172.21.15.123:0/1973064075 conn(0x7ff6340b9a40 msgr2=0x7ff6340b75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:31.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.102+0000 7ff63b800640 1 -- 172.21.15.123:0/1973064075 shutdown_connections 2024-04-24T05:46:31.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.102+0000 7ff63b800640 1 -- 172.21.15.123:0/1973064075 wait complete. 2024-04-24T05:46:31.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.104+0000 7ff63b800640 1 Processor -- start 2024-04-24T05:46:31.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.104+0000 7ff63b800640 1 -- start start 2024-04-24T05:46:31.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff63b800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6340b39b0 0x7ff6340ce460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff63b800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6340d3e10 0x7ff634153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff63b800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff63415e970 0x7ff6341f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff63b800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff63415ed50 con 0x7ff6340d3e10 2024-04-24T05:46:31.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff63b800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff6340d0650 con 0x7ff63415e970 2024-04-24T05:46:31.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff63b800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff6340b12b0 con 0x7ff6340b39b0 2024-04-24T05:46:31.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff633e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6340d3e10 0x7ff634153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff633e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6340d3e10 0x7ff634153b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50274/0 (socket says 172.21.15.123:50274) 2024-04-24T05:46:31.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff633e00640 1 -- 172.21.15.123:0/597738409 learned_addr learned my addr 172.21.15.123:0/597738409 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:31.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff638e00640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6340b39b0 0x7ff6340ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff639800640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff63415e970 0x7ff6341f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff633e00640 1 -- 172.21.15.123:0/597738409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6340b39b0 msgr2=0x7ff6340ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff633e00640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6340b39b0 0x7ff6340ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff633e00640 1 -- 172.21.15.123:0/597738409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff63415e970 msgr2=0x7ff6341f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff633e00640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff63415e970 0x7ff6341f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff633e00640 1 -- 172.21.15.123:0/597738409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63406cc40 con 0x7ff6340d3e10 2024-04-24T05:46:31.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff638e00640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6340b39b0 0x7ff6340ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:31.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.105+0000 7ff639800640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff63415e970 0x7ff6341f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:31.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.106+0000 7ff633e00640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6340d3e10 0x7ff634153b60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00c7f0 tx=0x7ff61c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:31.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.106+0000 7ff630c00640 1 -- 172.21.15.123:0/597738409 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c015070 con 0x7ff6340d3e10 2024-04-24T05:46:31.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.106+0000 7ff630c00640 1 -- 172.21.15.123:0/597738409 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff61c004220 con 0x7ff6340d3e10 2024-04-24T05:46:31.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.106+0000 7ff630c00640 1 -- 172.21.15.123:0/597738409 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c007620 con 0x7ff6340d3e10 2024-04-24T05:46:31.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.106+0000 7ff63b800640 1 -- 172.21.15.123:0/597738409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6340cb730 con 0x7ff6340d3e10 2024-04-24T05:46:31.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.106+0000 7ff63b800640 1 -- 172.21.15.123:0/597738409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6340cbcf0 con 0x7ff6340d3e10 2024-04-24T05:46:31.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.107+0000 7ff63b800640 1 -- 172.21.15.123:0/597738409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff634068a20 con 0x7ff6340d3e10 2024-04-24T05:46:31.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.108+0000 7ff630c00640 1 -- 172.21.15.123:0/597738409 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff61c0079d0 con 0x7ff6340d3e10 2024-04-24T05:46:31.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.109+0000 7ff630c00640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff61006c8f0 0x7ff61006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.109+0000 7ff630c00640 1 -- 172.21.15.123:0/597738409 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff61c08c330 con 0x7ff6340d3e10 2024-04-24T05:46:31.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.109+0000 7ff638e00640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff61006c8f0 0x7ff61006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.110+0000 7ff638e00640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff61006c8f0 0x7ff61006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff624033000 tx=0x7ff624009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:31.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.114+0000 7ff630c00640 1 -- 172.21.15.123:0/597738409 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff61c05a380 con 0x7ff6340d3e10 2024-04-24T05:46:31.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.316+0000 7ff63b800640 1 -- 172.21.15.123:0/597738409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6341f47a0 con 0x7ff6340d3e10 2024-04-24T05:46:31.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.320+0000 7ff630c00640 1 -- 172.21.15.123:0/597738409 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7ff61c059c80 con 0x7ff6340d3e10 2024-04-24T05:46:31.322 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:31.322 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:04.715857+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:31.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.322+0000 7ff63b800640 1 -- 172.21.15.123:0/597738409 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff61006c8f0 msgr2=0x7ff61006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.322+0000 7ff63b800640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff61006c8f0 0x7ff61006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff624033000 tx=0x7ff624009000 comp rx=0 tx=0).stop 2024-04-24T05:46:31.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.322+0000 7ff63b800640 1 -- 172.21.15.123:0/597738409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6340d3e10 msgr2=0x7ff634153b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.322+0000 7ff63b800640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6340d3e10 0x7ff634153b60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00c7f0 tx=0x7ff61c00ccb0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.322+0000 7ff63b800640 1 -- 172.21.15.123:0/597738409 shutdown_connections 2024-04-24T05:46:31.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.322+0000 7ff63b800640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff61006c8f0 0x7ff61006edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.322+0000 7ff63b800640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff63415e970 0x7ff6341f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.322+0000 7ff63b800640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6340d3e10 0x7ff634153b60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.322+0000 7ff63b800640 1 --2- 172.21.15.123:0/597738409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6340b39b0 0x7ff6340ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.322+0000 7ff63b800640 1 -- 172.21.15.123:0/597738409 >> 172.21.15.123:0/597738409 conn(0x7ff6340b9a40 msgr2=0x7ff63415f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:31.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.323+0000 7ff63b800640 1 -- 172.21.15.123:0/597738409 shutdown_connections 2024-04-24T05:46:31.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.323+0000 7ff63b800640 1 -- 172.21.15.123:0/597738409 wait complete. 2024-04-24T05:46:31.343 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:46:31.344 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:46:31.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.545+0000 7f73a8e00640 1 Processor -- start 2024-04-24T05:46:31.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.545+0000 7f73a8e00640 1 -- start start 2024-04-24T05:46:31.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.546+0000 7f73a8e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73a415c2f0 0x7f73a4141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.546+0000 7f73a8e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f73a40cc5d0 0x7f73a40acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.546+0000 7f73a8e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73a406ba00 con 0x7f73a40ad510 2024-04-24T05:46:31.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.546+0000 7f73a8e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f73a406bd30 con 0x7f73a40cc5d0 2024-04-24T05:46:31.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.546+0000 7f73a8e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f73a4142530 con 0x7f73a415c2f0 2024-04-24T05:46:31.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.546+0000 7f73a3e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f73a40ad510 0x7f73a415ffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49306/0 (socket says 172.21.15.123:49306) 2024-04-24T05:46:31.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.546+0000 7f73a3e00640 1 -- 172.21.15.123:0/3521344845 learned_addr learned my addr 172.21.15.123:0/3521344845 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:31.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.546+0000 7f73a3400640 1 --2- 172.21.15.123:0/3521344845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73a415c2f0 0x7f73a4141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f739aa00640 1 --2- 172.21.15.123:0/3521344845 >> v2:172.21.15.155:3300/0 conn(0x7f73a40cc5d0 0x7f73a40acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 -- 172.21.15.123:0/3521344845 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2139242589 0 0) 0x7f73a406ba00 con 0x7f73a40ad510 2024-04-24T05:46:31.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 -- 172.21.15.123:0/3521344845 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7388003520 con 0x7f73a40ad510 2024-04-24T05:46:31.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 -- 172.21.15.123:0/3521344845 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2416568807 0 0) 0x7f7388003520 con 0x7f73a40ad510 2024-04-24T05:46:31.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 -- 172.21.15.123:0/3521344845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73a415c2f0 msgr2=0x7f73a4141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:31.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 --2- 172.21.15.123:0/3521344845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73a415c2f0 0x7f73a4141ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 -- 172.21.15.123:0/3521344845 >> v2:172.21.15.155:3300/0 conn(0x7f73a40cc5d0 msgr2=0x7f73a40acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:31.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 --2- 172.21.15.123:0/3521344845 >> v2:172.21.15.155:3300/0 conn(0x7f73a40cc5d0 0x7f73a40acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 -- 172.21.15.123:0/3521344845 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a4142810 con 0x7f73a40ad510 2024-04-24T05:46:31.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 -- 172.21.15.123:0/3521344845 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f739c001670 con 0x7f73a40ad510 2024-04-24T05:46:31.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 -- 172.21.15.123:0/3521344845 >> v1:172.21.15.123:6789/0 conn(0x7f73a40ad510 legacy=0x7f73a415ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 --2- 172.21.15.123:0/3521344845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73a415c2f0 0x7f73880039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 --2- 172.21.15.123:0/3521344845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7388003f80 0x7f7388004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 --2- 172.21.15.123:0/3521344845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7388004d40 0x7f73880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 -- 172.21.15.123:0/3521344845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73a406bd30 con 0x7f7388004d40 2024-04-24T05:46:31.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 -- 172.21.15.123:0/3521344845 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4142530 con 0x7f7388003f80 2024-04-24T05:46:31.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a2a00640 1 -- 172.21.15.123:0/3521344845 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f738800bb20 con 0x7f73a415c2f0 2024-04-24T05:46:31.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.547+0000 7f73a3e00640 1 --2- 172.21.15.123:0/3521344845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7388004d40 0x7f73880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.548+0000 7f739aa00640 1 --2- 172.21.15.123:0/3521344845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7388003f80 0x7f7388004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.548+0000 7f73a3400640 1 --2- 172.21.15.123:0/3521344845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73a415c2f0 0x7f73880039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.548+0000 7f73a3e00640 1 -- 172.21.15.123:0/3521344845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73a415c2f0 msgr2=0x7f73880039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:31.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.548+0000 7f73a3e00640 1 --2- 172.21.15.123:0/3521344845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73a415c2f0 0x7f73880039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.548+0000 7f73a3e00640 1 -- 172.21.15.123:0/3521344845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7388003f80 msgr2=0x7f7388004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:31.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.548+0000 7f73a3e00640 1 --2- 172.21.15.123:0/3521344845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7388003f80 0x7f7388004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.548+0000 7f73a3e00640 1 -- 172.21.15.123:0/3521344845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7388003520 con 0x7f7388004d40 2024-04-24T05:46:31.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.548+0000 7f73a3400640 1 --2- 172.21.15.123:0/3521344845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73a415c2f0 0x7f73880039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:31.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.549+0000 7f73a3e00640 1 --2- 172.21.15.123:0/3521344845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7388004d40 0x7f73880051c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f739c00c190 tx=0x7f7388005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcdf8d723e2b29e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:31.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.549+0000 7f73a2a00640 1 -- 172.21.15.123:0/3521344845 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c007c80 con 0x7f7388004d40 2024-04-24T05:46:31.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.549+0000 7f73a2a00640 1 -- 172.21.15.123:0/3521344845 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f739c005ad0 con 0x7f7388004d40 2024-04-24T05:46:31.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.549+0000 7f73a2a00640 1 -- 172.21.15.123:0/3521344845 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c007070 con 0x7f7388004d40 2024-04-24T05:46:31.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.550+0000 7f73a8e00640 1 -- 172.21.15.123:0/3521344845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7388004d40 msgr2=0x7f73880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.550+0000 7f73a8e00640 1 --2- 172.21.15.123:0/3521344845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7388004d40 0x7f73880051c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f739c00c190 tx=0x7f7388005830 comp rx=0 tx=0).stop 2024-04-24T05:46:31.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.550+0000 7f73a3e00640 1 -- 172.21.15.123:0/3521344845 reap_dead start 2024-04-24T05:46:31.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.550+0000 7f73a8e00640 1 -- 172.21.15.123:0/3521344845 shutdown_connections 2024-04-24T05:46:31.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.550+0000 7f73a8e00640 1 -- 172.21.15.123:0/3521344845 >> 172.21.15.123:0/3521344845 conn(0x7f73a40b1cf0 msgr2=0x7f73a40b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:31.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.550+0000 7f73a8e00640 1 -- 172.21.15.123:0/3521344845 shutdown_connections 2024-04-24T05:46:31.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.550+0000 7f73a8e00640 1 -- 172.21.15.123:0/3521344845 wait complete. 2024-04-24T05:46:31.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.551+0000 7f73a8e00640 1 Processor -- start 2024-04-24T05:46:31.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.552+0000 7f73a8e00640 1 -- start start 2024-04-24T05:46:31.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f73a8e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7388004d40 0x7f73a41f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f73a8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73a40b1730 0x7f73a414d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f73a8e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a414a720 0x7f73a414aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f73a8e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73a406d350 con 0x7f73a40b1730 2024-04-24T05:46:31.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f73a8e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f73a406d1d0 con 0x7f73a414a720 2024-04-24T05:46:31.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f73a8e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f73a415c6d0 con 0x7f7388004d40 2024-04-24T05:46:31.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f739aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73a40b1730 0x7f73a414d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f739aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73a40b1730 0x7f73a414d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50288/0 (socket says 172.21.15.123:50288) 2024-04-24T05:46:31.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f739aa00640 1 -- 172.21.15.123:0/4030412537 learned_addr learned my addr 172.21.15.123:0/4030412537 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:31.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f73a3400640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7388004d40 0x7f73a41f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f73a3e00640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a414a720 0x7f73a414aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f739aa00640 1 -- 172.21.15.123:0/4030412537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7388004d40 msgr2=0x7f73a41f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f739aa00640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7388004d40 0x7f73a41f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f739aa00640 1 -- 172.21.15.123:0/4030412537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a414a720 msgr2=0x7f73a414aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f739aa00640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a414a720 0x7f73a414aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f739aa00640 1 -- 172.21.15.123:0/4030412537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a414dc80 con 0x7f73a40b1730 2024-04-24T05:46:31.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.553+0000 7f73a3400640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7388004d40 0x7f73a41f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:31.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.554+0000 7f739aa00640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73a40b1730 0x7f73a414d740 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f738c009ce0 tx=0x7f738c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:31.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.554+0000 7f739be00640 1 -- 172.21.15.123:0/4030412537 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c014610 con 0x7f73a40b1730 2024-04-24T05:46:31.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.554+0000 7f739be00640 1 -- 172.21.15.123:0/4030412537 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f738c019070 con 0x7f73a40b1730 2024-04-24T05:46:31.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.554+0000 7f739be00640 1 -- 172.21.15.123:0/4030412537 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c0149f0 con 0x7f73a40b1730 2024-04-24T05:46:31.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.554+0000 7f73a8e00640 1 -- 172.21.15.123:0/4030412537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a414cc20 con 0x7f73a40b1730 2024-04-24T05:46:31.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.554+0000 7f73a8e00640 1 -- 172.21.15.123:0/4030412537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a4072730 con 0x7f73a40b1730 2024-04-24T05:46:31.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.555+0000 7f73a8e00640 1 -- 172.21.15.123:0/4030412537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a4068a20 con 0x7f73a40b1730 2024-04-24T05:46:31.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.556+0000 7f739be00640 1 -- 172.21.15.123:0/4030412537 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f738c014b90 con 0x7f73a40b1730 2024-04-24T05:46:31.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.557+0000 7f739be00640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f738006c8f0 0x7f738006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.557+0000 7f739be00640 1 -- 172.21.15.123:0/4030412537 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f738c08c830 con 0x7f73a40b1730 2024-04-24T05:46:31.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.557+0000 7f73a3400640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f738006c8f0 0x7f738006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.558+0000 7f73a3400640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f738006c8f0 0x7f738006edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7394009aa0 tx=0x7f7394008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:31.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.562+0000 7f739be00640 1 -- 172.21.15.123:0/4030412537 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f738c05a910 con 0x7f73a40b1730 2024-04-24T05:46:31.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.758+0000 7f73a8e00640 1 -- 172.21.15.123:0/4030412537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73a4142310 con 0x7f73a40b1730 2024-04-24T05:46:31.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.758+0000 7f739be00640 1 -- 172.21.15.123:0/4030412537 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f738c05a210 con 0x7f73a40b1730 2024-04-24T05:46:31.760 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:46:31.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.761+0000 7f73a8e00640 1 -- 172.21.15.123:0/4030412537 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f738006c8f0 msgr2=0x7f738006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.761+0000 7f73a8e00640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f738006c8f0 0x7f738006edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7394009aa0 tx=0x7f7394008040 comp rx=0 tx=0).stop 2024-04-24T05:46:31.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.761+0000 7f73a8e00640 1 -- 172.21.15.123:0/4030412537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73a40b1730 msgr2=0x7f73a414d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.761+0000 7f73a8e00640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73a40b1730 0x7f73a414d740 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f738c009ce0 tx=0x7f738c007700 comp rx=0 tx=0).stop 2024-04-24T05:46:31.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.761+0000 7f73a8e00640 1 -- 172.21.15.123:0/4030412537 shutdown_connections 2024-04-24T05:46:31.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.761+0000 7f73a8e00640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f738006c8f0 0x7f738006edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.761+0000 7f73a8e00640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a414a720 0x7f73a414aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.761+0000 7f73a8e00640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73a40b1730 0x7f73a414d740 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.761+0000 7f73a8e00640 1 --2- 172.21.15.123:0/4030412537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7388004d40 0x7f73a41f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.761+0000 7f73a8e00640 1 -- 172.21.15.123:0/4030412537 >> 172.21.15.123:0/4030412537 conn(0x7f73a40b1cf0 msgr2=0x7f73a406ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:31.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.762+0000 7f73a8e00640 1 -- 172.21.15.123:0/4030412537 shutdown_connections 2024-04-24T05:46:31.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.762+0000 7f73a8e00640 1 -- 172.21.15.123:0/4030412537 wait complete. 2024-04-24T05:46:31.782 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:46:31.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.979+0000 7f99bf000640 1 Processor -- start 2024-04-24T05:46:31.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.980+0000 7f99bf000640 1 -- start start 2024-04-24T05:46:31.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.981+0000 7f99bf000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f99b80ae990 0x7f99b80d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.981+0000 7f99bf000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b806c410 0x7f99b80ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.981+0000 7f99bf000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99b80d0740 con 0x7f99b80ce610 2024-04-24T05:46:31.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.981+0000 7f99bf000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f99b80cea70 con 0x7f99b80ae990 2024-04-24T05:46:31.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.981+0000 7f99bf000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f99b806dcb0 con 0x7f99b806c410 2024-04-24T05:46:31.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.982+0000 7f99b7400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b806c410 0x7f99b80ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.982+0000 7f99b7400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b806c410 0x7f99b80ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:34474/0 (socket says 172.21.15.123:34474) 2024-04-24T05:46:31.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.982+0000 7f99b7400640 1 -- 172.21.15.123:0/3074155039 learned_addr learned my addr 172.21.15.123:0/3074155039 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:31.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.982+0000 7f99b7e00640 1 --2- 172.21.15.123:0/3074155039 >> v2:172.21.15.155:3300/0 conn(0x7f99b80ae990 0x7f99b80d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.982+0000 7f99b7400640 1 -- 172.21.15.123:0/3074155039 >> v2:172.21.15.155:3300/0 conn(0x7f99b80ae990 msgr2=0x7f99b80d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:31.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.982+0000 7f99b7400640 1 --2- 172.21.15.123:0/3074155039 >> v2:172.21.15.155:3300/0 conn(0x7f99b80ae990 0x7f99b80d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.982+0000 7f99b7400640 1 -- 172.21.15.123:0/3074155039 >> v1:172.21.15.155:6789/0 conn(0x7f99b80ce610 legacy=0x7f99b815f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.982+0000 7f99b7400640 1 -- 172.21.15.123:0/3074155039 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b80d0a20 con 0x7f99b806c410 2024-04-24T05:46:31.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.984+0000 7f99b7400640 1 --2- 172.21.15.123:0/3074155039 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b806c410 0x7f99b80ce0d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009870 tx=0x7f99a402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32c94f4531c1f056 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:31.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.984+0000 7f99b6a00640 1 -- 172.21.15.123:0/3074155039 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4037070 con 0x7f99b806c410 2024-04-24T05:46:31.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.984+0000 7f99b6a00640 1 -- 172.21.15.123:0/3074155039 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99a4032440 con 0x7f99b806c410 2024-04-24T05:46:31.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.984+0000 7f99b6a00640 1 -- 172.21.15.123:0/3074155039 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4032810 con 0x7f99b806c410 2024-04-24T05:46:31.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.985+0000 7f99bf000640 1 -- 172.21.15.123:0/3074155039 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b806c410 msgr2=0x7f99b80ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.985+0000 7f99bf000640 1 --2- 172.21.15.123:0/3074155039 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b806c410 0x7f99b80ce0d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009870 tx=0x7f99a402a270 comp rx=0 tx=0).stop 2024-04-24T05:46:31.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.985+0000 7f99bf000640 1 -- 172.21.15.123:0/3074155039 shutdown_connections 2024-04-24T05:46:31.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.985+0000 7f99bf000640 1 --2- 172.21.15.123:0/3074155039 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b806c410 0x7f99b80ce0d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.985+0000 7f99bf000640 1 --2- 172.21.15.123:0/3074155039 >> v2:172.21.15.155:3300/0 conn(0x7f99b80ae990 0x7f99b80d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.985+0000 7f99bf000640 1 -- 172.21.15.123:0/3074155039 >> 172.21.15.123:0/3074155039 conn(0x7f99b80b51f0 msgr2=0x7f99b80b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:31.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.985+0000 7f99bf000640 1 -- 172.21.15.123:0/3074155039 shutdown_connections 2024-04-24T05:46:31.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.985+0000 7f99bf000640 1 -- 172.21.15.123:0/3074155039 wait complete. 2024-04-24T05:46:31.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.987+0000 7f99bf000640 1 Processor -- start 2024-04-24T05:46:31.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.988+0000 7f99bf000640 1 -- start start 2024-04-24T05:46:31.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99bf000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b806c410 0x7f99b80c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99bf000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b80ae990 0x7f99b80c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99bf000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b80ce610 0x7f99b80c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99bf000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f99b80aed70 con 0x7f99b80ae990 2024-04-24T05:46:31.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99bf000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f99b806e270 con 0x7f99b806c410 2024-04-24T05:46:31.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99bf000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f99b806c870 con 0x7f99b80ce610 2024-04-24T05:46:31.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99b7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b80ae990 0x7f99b80c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99b7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b80ae990 0x7f99b80c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50294/0 (socket says 172.21.15.123:50294) 2024-04-24T05:46:31.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99b7400640 1 -- 172.21.15.123:0/3074858745 learned_addr learned my addr 172.21.15.123:0/3074858745 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:31.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99b7e00640 1 --2- 172.21.15.123:0/3074858745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b806c410 0x7f99b80c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99bd000640 1 --2- 172.21.15.123:0/3074858745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b80ce610 0x7f99b80c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99b7400640 1 -- 172.21.15.123:0/3074858745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b80ce610 msgr2=0x7f99b80c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99b7400640 1 --2- 172.21.15.123:0/3074858745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b80ce610 0x7f99b80c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99b7400640 1 -- 172.21.15.123:0/3074858745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b806c410 msgr2=0x7f99b80c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:31.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99b7400640 1 --2- 172.21.15.123:0/3074858745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b806c410 0x7f99b80c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:31.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.989+0000 7f99b7400640 1 -- 172.21.15.123:0/3074858745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b80c6530 con 0x7f99b80ae990 2024-04-24T05:46:31.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.990+0000 7f99b7400640 1 --2- 172.21.15.123:0/3074858745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b80ae990 0x7f99b80c9ce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000c00 tx=0x7f99a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:31.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.990+0000 7f99afe00640 1 -- 172.21.15.123:0/3074858745 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4041020 con 0x7f99b80ae990 2024-04-24T05:46:31.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.990+0000 7f99afe00640 1 -- 172.21.15.123:0/3074858745 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99a40306c0 con 0x7f99b80ae990 2024-04-24T05:46:31.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.990+0000 7f99afe00640 1 -- 172.21.15.123:0/3074858745 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4037040 con 0x7f99b80ae990 2024-04-24T05:46:31.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.990+0000 7f99bf000640 1 -- 172.21.15.123:0/3074858745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b80c5f70 con 0x7f99b80ae990 2024-04-24T05:46:31.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.990+0000 7f99bf000640 1 -- 172.21.15.123:0/3074858745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b80b0e30 con 0x7f99b80ae990 2024-04-24T05:46:31.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.991+0000 7f99bf000640 1 -- 172.21.15.123:0/3074858745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b8068a20 con 0x7f99b80ae990 2024-04-24T05:46:31.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.992+0000 7f99afe00640 1 -- 172.21.15.123:0/3074858745 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99a4044020 con 0x7f99b80ae990 2024-04-24T05:46:31.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.993+0000 7f99afe00640 1 --2- 172.21.15.123:0/3074858745 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f999806c8f0 0x7f999806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:31.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.993+0000 7f99afe00640 1 -- 172.21.15.123:0/3074858745 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f99a40ac6b0 con 0x7f99b80ae990 2024-04-24T05:46:31.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.994+0000 7f99b7e00640 1 --2- 172.21.15.123:0/3074858745 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f999806c8f0 0x7f999806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:31.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.994+0000 7f99b7e00640 1 --2- 172.21.15.123:0/3074858745 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f999806c8f0 0x7f999806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f99a8006fd0 tx=0x7f99a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:31.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:31.998+0000 7f99afe00640 1 -- 172.21.15.123:0/3074858745 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f99a404b030 con 0x7f99b80ae990 2024-04-24T05:46:32.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.198+0000 7f99bf000640 1 -- 172.21.15.123:0/3074858745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f99b806cb80 con 0x7f99b80ae990 2024-04-24T05:46:32.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.199+0000 7f99afe00640 1 -- 172.21.15.123:0/3074858745 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f99a407a790 con 0x7f99b80ae990 2024-04-24T05:46:32.201 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:32.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.202+0000 7f99bf000640 1 -- 172.21.15.123:0/3074858745 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f999806c8f0 msgr2=0x7f999806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:32.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.202+0000 7f99bf000640 1 --2- 172.21.15.123:0/3074858745 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f999806c8f0 0x7f999806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f99a8006fd0 tx=0x7f99a8008040 comp rx=0 tx=0).stop 2024-04-24T05:46:32.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.202+0000 7f99bf000640 1 -- 172.21.15.123:0/3074858745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b80ae990 msgr2=0x7f99b80c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:32.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.202+0000 7f99bf000640 1 --2- 172.21.15.123:0/3074858745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b80ae990 0x7f99b80c9ce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000c00 tx=0x7f99a40046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.202+0000 7f99bf000640 1 -- 172.21.15.123:0/3074858745 shutdown_connections 2024-04-24T05:46:32.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.202+0000 7f99bf000640 1 --2- 172.21.15.123:0/3074858745 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f999806c8f0 0x7f999806edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.202+0000 7f99bf000640 1 --2- 172.21.15.123:0/3074858745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b80ce610 0x7f99b80c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.202+0000 7f99bf000640 1 --2- 172.21.15.123:0/3074858745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b80ae990 0x7f99b80c9ce0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.202+0000 7f99bf000640 1 --2- 172.21.15.123:0/3074858745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b806c410 0x7f99b80c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.202+0000 7f99bf000640 1 -- 172.21.15.123:0/3074858745 >> 172.21.15.123:0/3074858745 conn(0x7f99b80b51f0 msgr2=0x7f99b8151f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:32.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.202+0000 7f99bf000640 1 -- 172.21.15.123:0/3074858745 shutdown_connections 2024-04-24T05:46:32.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.202+0000 7f99bf000640 1 -- 172.21.15.123:0/3074858745 wait complete. 2024-04-24T05:46:32.225 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:46:32.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.421+0000 7f3a15a00640 1 Processor -- start 2024-04-24T05:46:32.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.422+0000 7f3a15a00640 1 -- start start 2024-04-24T05:46:32.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.423+0000 7f3a15a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3a1015c2f0 0x7f3a10141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:32.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.423+0000 7f3a15a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3a100cc5d0 0x7f3a100acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:32.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.423+0000 7f3a15a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a1006ba00 con 0x7f3a100ad510 2024-04-24T05:46:32.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.423+0000 7f3a15a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3a1006bd30 con 0x7f3a1015c2f0 2024-04-24T05:46:32.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.423+0000 7f3a15a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3a100ad940 con 0x7f3a100cc5d0 2024-04-24T05:46:32.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.423+0000 7f3a0fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3a1015c2f0 0x7f3a10141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:32.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.423+0000 7f3a15000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3a100ad510 0x7f3a1015ffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49308/0 (socket says 172.21.15.123:49308) 2024-04-24T05:46:32.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.424+0000 7f3a15000640 1 -- 172.21.15.123:0/1743797548 learned_addr learned my addr 172.21.15.123:0/1743797548 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:32.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.424+0000 7f3a07400640 1 --2- 172.21.15.123:0/1743797548 >> v2:172.21.15.155:3300/0 conn(0x7f3a100cc5d0 0x7f3a100acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:32.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.424+0000 7f3a0fe00640 1 -- 172.21.15.123:0/1743797548 >> v2:172.21.15.155:3300/0 conn(0x7f3a100cc5d0 msgr2=0x7f3a100acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:32.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.424+0000 7f3a0fe00640 1 --2- 172.21.15.123:0/1743797548 >> v2:172.21.15.155:3300/0 conn(0x7f3a100cc5d0 0x7f3a100acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.424+0000 7f3a0fe00640 1 -- 172.21.15.123:0/1743797548 >> v1:172.21.15.123:6789/0 conn(0x7f3a100ad510 legacy=0x7f3a1015ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:32.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.424+0000 7f3a0fe00640 1 -- 172.21.15.123:0/1743797548 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a10142530 con 0x7f3a1015c2f0 2024-04-24T05:46:32.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.424+0000 7f3a0f400640 1 -- 172.21.15.123:0/1743797548 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1844808110 0 0) 0x7f3a1006ba00 con 0x7f3a100ad510 2024-04-24T05:46:32.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.425+0000 7f3a0fe00640 1 --2- 172.21.15.123:0/1743797548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a1015c2f0 0x7f3a10141ce0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3a10073fd0 tx=0x7f3a0002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d261392c905fb4a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:32.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.425+0000 7f3a0f400640 1 -- 172.21.15.123:0/1743797548 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00007740 con 0x7f3a1015c2f0 2024-04-24T05:46:32.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.426+0000 7f3a0f400640 1 -- 172.21.15.123:0/1743797548 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a000078e0 con 0x7f3a1015c2f0 2024-04-24T05:46:32.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.426+0000 7f3a0f400640 1 -- 172.21.15.123:0/1743797548 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00007cb0 con 0x7f3a1015c2f0 2024-04-24T05:46:32.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.426+0000 7f3a15a00640 1 -- 172.21.15.123:0/1743797548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a1015c2f0 msgr2=0x7f3a10141ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:32.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.426+0000 7f3a15a00640 1 --2- 172.21.15.123:0/1743797548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a1015c2f0 0x7f3a10141ce0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3a10073fd0 tx=0x7f3a0002a5f0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.426+0000 7f3a15a00640 1 -- 172.21.15.123:0/1743797548 shutdown_connections 2024-04-24T05:46:32.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.426+0000 7f3a15a00640 1 --2- 172.21.15.123:0/1743797548 >> v2:172.21.15.155:3300/0 conn(0x7f3a100cc5d0 0x7f3a100acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.426+0000 7f3a15a00640 1 --2- 172.21.15.123:0/1743797548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a1015c2f0 0x7f3a10141ce0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.426+0000 7f3a15a00640 1 -- 172.21.15.123:0/1743797548 >> 172.21.15.123:0/1743797548 conn(0x7f3a100b1cf0 msgr2=0x7f3a100b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:32.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.426+0000 7f3a15a00640 1 -- 172.21.15.123:0/1743797548 shutdown_connections 2024-04-24T05:46:32.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.426+0000 7f3a15a00640 1 -- 172.21.15.123:0/1743797548 wait complete. 2024-04-24T05:46:32.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.428+0000 7f3a15a00640 1 Processor -- start 2024-04-24T05:46:32.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.428+0000 7f3a15a00640 1 -- start start 2024-04-24T05:46:32.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.429+0000 7f3a15a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a100ad510 0x7f3a100cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:32.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.429+0000 7f3a15a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a100cc5d0 0x7f3a10153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:32.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.429+0000 7f3a15a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a1015c2f0 0x7f3a10159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:32.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.429+0000 7f3a15a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10142220 con 0x7f3a100ad510 2024-04-24T05:46:32.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.429+0000 7f3a15a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10142af0 con 0x7f3a1015c2f0 2024-04-24T05:46:32.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.429+0000 7f3a15a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3a1006d1d0 con 0x7f3a100cc5d0 2024-04-24T05:46:32.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.429+0000 7f3a0fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a100ad510 0x7f3a100cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:32.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.430+0000 7f3a0fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a100ad510 0x7f3a100cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50308/0 (socket says 172.21.15.123:50308) 2024-04-24T05:46:32.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.430+0000 7f3a0fe00640 1 -- 172.21.15.123:0/3935549300 learned_addr learned my addr 172.21.15.123:0/3935549300 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.430+0000 7f3a15000640 1 --2- 172.21.15.123:0/3935549300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a1015c2f0 0x7f3a10159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.430+0000 7f3a07400640 1 --2- 172.21.15.123:0/3935549300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a100cc5d0 0x7f3a10153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.430+0000 7f3a0fe00640 1 -- 172.21.15.123:0/3935549300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a100cc5d0 msgr2=0x7f3a10153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.430+0000 7f3a0fe00640 1 --2- 172.21.15.123:0/3935549300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a100cc5d0 0x7f3a10153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.430+0000 7f3a0fe00640 1 -- 172.21.15.123:0/3935549300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a1015c2f0 msgr2=0x7f3a10159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.430+0000 7f3a0fe00640 1 --2- 172.21.15.123:0/3935549300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a1015c2f0 0x7f3a10159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.430+0000 7f3a0fe00640 1 -- 172.21.15.123:0/3935549300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a100ce210 con 0x7f3a100ad510 2024-04-24T05:46:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.430+0000 7f3a0fe00640 1 --2- 172.21.15.123:0/3935549300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a100ad510 0x7f3a100cf2b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3a00000f80 tx=0x7f3a00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.431+0000 7f3a0cc00640 1 -- 172.21.15.123:0/3935549300 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00040070 con 0x7f3a100ad510 2024-04-24T05:46:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.431+0000 7f3a0cc00640 1 -- 172.21.15.123:0/3935549300 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a00031c40 con 0x7f3a100ad510 2024-04-24T05:46:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.431+0000 7f3a0cc00640 1 -- 172.21.15.123:0/3935549300 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00009450 con 0x7f3a100ad510 2024-04-24T05:46:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.431+0000 7f3a15a00640 1 -- 172.21.15.123:0/3935549300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a100ce4f0 con 0x7f3a100ad510 2024-04-24T05:46:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.431+0000 7f3a15a00640 1 -- 172.21.15.123:0/3935549300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a100ceab0 con 0x7f3a100ad510 2024-04-24T05:46:32.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.432+0000 7f3a15a00640 1 -- 172.21.15.123:0/3935549300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a1006eb10 con 0x7f3a100ad510 2024-04-24T05:46:32.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.433+0000 7f3a0cc00640 1 -- 172.21.15.123:0/3935549300 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a00039040 con 0x7f3a100ad510 2024-04-24T05:46:32.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.434+0000 7f3a0cc00640 1 --2- 172.21.15.123:0/3935549300 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f39ec06c8f0 0x7f39ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:32.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.435+0000 7f3a0cc00640 1 -- 172.21.15.123:0/3935549300 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3a000ad240 con 0x7f3a100ad510 2024-04-24T05:46:32.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.435+0000 7f3a07400640 1 --2- 172.21.15.123:0/3935549300 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f39ec06c8f0 0x7f39ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:32.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.435+0000 7f3a07400640 1 --2- 172.21.15.123:0/3935549300 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f39ec06c8f0 0x7f39ec06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f39f8006fd0 tx=0x7f39f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:32.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.439+0000 7f3a0cc00640 1 -- 172.21.15.123:0/3935549300 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a1006eb10 con 0x7f3a100ad510 2024-04-24T05:46:32.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.640+0000 7f3a15a00640 1 -- 172.21.15.123:0/3935549300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3a10075ca0 con 0x7f3a100ad510 2024-04-24T05:46:32.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.641+0000 7f3a0cc00640 1 -- 172.21.15.123:0/3935549300 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f3a10075ca0 con 0x7f3a100ad510 2024-04-24T05:46:32.643 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:46:32.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.644+0000 7f3a15a00640 1 -- 172.21.15.123:0/3935549300 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f39ec06c8f0 msgr2=0x7f39ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:32.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.644+0000 7f3a15a00640 1 --2- 172.21.15.123:0/3935549300 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f39ec06c8f0 0x7f39ec06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f39f8006fd0 tx=0x7f39f8008040 comp rx=0 tx=0).stop 2024-04-24T05:46:32.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.644+0000 7f3a15a00640 1 -- 172.21.15.123:0/3935549300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a100ad510 msgr2=0x7f3a100cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:32.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.644+0000 7f3a15a00640 1 --2- 172.21.15.123:0/3935549300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a100ad510 0x7f3a100cf2b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3a00000f80 tx=0x7f3a00004a40 comp rx=0 tx=0).stop 2024-04-24T05:46:32.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.644+0000 7f3a15a00640 1 -- 172.21.15.123:0/3935549300 shutdown_connections 2024-04-24T05:46:32.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.644+0000 7f3a15a00640 1 --2- 172.21.15.123:0/3935549300 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f39ec06c8f0 0x7f39ec06edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.644+0000 7f3a15a00640 1 --2- 172.21.15.123:0/3935549300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a1015c2f0 0x7f3a10159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.644+0000 7f3a15a00640 1 --2- 172.21.15.123:0/3935549300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3a100cc5d0 0x7f3a10153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.644+0000 7f3a15a00640 1 --2- 172.21.15.123:0/3935549300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a100ad510 0x7f3a100cf2b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.644+0000 7f3a15a00640 1 -- 172.21.15.123:0/3935549300 >> 172.21.15.123:0/3935549300 conn(0x7f3a100b1cf0 msgr2=0x7f3a1015b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:32.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.644+0000 7f3a15a00640 1 -- 172.21.15.123:0/3935549300 shutdown_connections 2024-04-24T05:46:32.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.644+0000 7f3a15a00640 1 -- 172.21.15.123:0/3935549300 wait complete. 2024-04-24T05:46:32.664 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:46:32.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.859+0000 7f0d52000640 1 Processor -- start 2024-04-24T05:46:32.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.860+0000 7f0d52000640 1 -- start start 2024-04-24T05:46:32.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.861+0000 7f0d52000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0d4c0d45a0 0x7f0d4c0d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:32.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.861+0000 7f0d52000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d4c0cfac0 0x7f0d4c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:32.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.861+0000 7f0d52000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d4c15ee50 con 0x7f0d4c0b35e0 2024-04-24T05:46:32.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.861+0000 7f0d52000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0d4c0d4ec0 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.861+0000 7f0d52000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0d4c0cff80 con 0x7f0d4c0cfac0 2024-04-24T05:46:32.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.861+0000 7f0d4b400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0d4c0d45a0 0x7f0d4c0d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:32.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.861+0000 7f0d4b400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0d4c0d45a0 0x7f0d4c0d4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50312/0 (socket says 172.21.15.123:50312) 2024-04-24T05:46:32.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.861+0000 7f0d4b400640 1 -- 172.21.15.123:0/3313055654 learned_addr learned my addr 172.21.15.123:0/3313055654 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:32.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.862+0000 7f0d4b400640 1 -- 172.21.15.123:0/3313055654 >> v2:172.21.15.155:3300/0 conn(0x7f0d4c0cfac0 msgr2=0x7f0d4c0b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:32.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.862+0000 7f0d4aa00640 1 --2- 172.21.15.123:0/3313055654 >> v2:172.21.15.155:3300/0 conn(0x7f0d4c0cfac0 0x7f0d4c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:32.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.862+0000 7f0d4b400640 1 --2- 172.21.15.123:0/3313055654 >> v2:172.21.15.155:3300/0 conn(0x7f0d4c0cfac0 0x7f0d4c0b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.862+0000 7f0d4b400640 1 -- 172.21.15.123:0/3313055654 >> v1:172.21.15.155:6789/0 conn(0x7f0d4c0b35e0 legacy=0x7f0d4c0b65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:32.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.862+0000 7f0d4b400640 1 -- 172.21.15.123:0/3313055654 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c0d0260 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.863+0000 7f0d4b400640 1 --2- 172.21.15.123:0/3313055654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d4c0d45a0 0x7f0d4c0d4980 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c09e060 tx=0x7f0d3c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfddf7aaabe97e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:32.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.863+0000 7f0d4a000640 1 -- 172.21.15.123:0/3313055654 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c036070 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.863+0000 7f0d4a000640 1 -- 172.21.15.123:0/3313055654 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3c031600 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.863+0000 7f0d4a000640 1 -- 172.21.15.123:0/3313055654 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c0319d0 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.864+0000 7f0d52000640 1 -- 172.21.15.123:0/3313055654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d4c0d45a0 msgr2=0x7f0d4c0d4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:32.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.864+0000 7f0d52000640 1 --2- 172.21.15.123:0/3313055654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d4c0d45a0 0x7f0d4c0d4980 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c09e060 tx=0x7f0d3c02a6f0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.864+0000 7f0d52000640 1 -- 172.21.15.123:0/3313055654 shutdown_connections 2024-04-24T05:46:32.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.864+0000 7f0d52000640 1 --2- 172.21.15.123:0/3313055654 >> v2:172.21.15.155:3300/0 conn(0x7f0d4c0cfac0 0x7f0d4c0b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.864+0000 7f0d52000640 1 --2- 172.21.15.123:0/3313055654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d4c0d45a0 0x7f0d4c0d4980 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.864+0000 7f0d52000640 1 -- 172.21.15.123:0/3313055654 >> 172.21.15.123:0/3313055654 conn(0x7f0d4c09d970 msgr2=0x7f0d4c0b95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:32.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.864+0000 7f0d52000640 1 -- 172.21.15.123:0/3313055654 shutdown_connections 2024-04-24T05:46:32.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.864+0000 7f0d52000640 1 -- 172.21.15.123:0/3313055654 wait complete. 2024-04-24T05:46:32.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.866+0000 7f0d52000640 1 Processor -- start 2024-04-24T05:46:32.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.866+0000 7f0d52000640 1 -- start start 2024-04-24T05:46:32.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.867+0000 7f0d52000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c0b35e0 0x7f0d4c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:32.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.867+0000 7f0d52000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c0cfac0 0x7f0d4c15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:32.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.867+0000 7f0d52000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d4c0d45a0 0x7f0d4c06db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:32.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.867+0000 7f0d52000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c0d0b00 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.867+0000 7f0d52000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c0d0820 con 0x7f0d4c0b35e0 2024-04-24T05:46:32.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.867+0000 7f0d52000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0d4c0b6ce0 con 0x7f0d4c0cfac0 2024-04-24T05:46:32.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.867+0000 7f0d4be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d4c0d45a0 0x7f0d4c06db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:32.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.867+0000 7f0d4be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d4c0d45a0 0x7f0d4c06db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50318/0 (socket says 172.21.15.123:50318) 2024-04-24T05:46:32.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.867+0000 7f0d4be00640 1 -- 172.21.15.123:0/1195432626 learned_addr learned my addr 172.21.15.123:0/1195432626 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:32.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.868+0000 7f0d4aa00640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c0cfac0 0x7f0d4c15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:32.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.868+0000 7f0d4b400640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c0b35e0 0x7f0d4c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:32.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.868+0000 7f0d4be00640 1 -- 172.21.15.123:0/1195432626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c0cfac0 msgr2=0x7f0d4c15ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:32.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.868+0000 7f0d4be00640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c0cfac0 0x7f0d4c15ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.868+0000 7f0d4be00640 1 -- 172.21.15.123:0/1195432626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c0b35e0 msgr2=0x7f0d4c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:32.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.868+0000 7f0d4be00640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c0b35e0 0x7f0d4c0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:32.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.868+0000 7f0d4be00640 1 -- 172.21.15.123:0/1195432626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c15b0c0 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.868+0000 7f0d4b400640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c0b35e0 0x7f0d4c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:32.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.868+0000 7f0d4be00640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d4c0d45a0 0x7f0d4c06db90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400c440 tx=0x7f0d4400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:32.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.868+0000 7f0d43400640 1 -- 172.21.15.123:0/1195432626 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44011790 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.869+0000 7f0d43400640 1 -- 172.21.15.123:0/1195432626 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d44017070 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.869+0000 7f0d43400640 1 -- 172.21.15.123:0/1195432626 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44011b70 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.869+0000 7f0d52000640 1 -- 172.21.15.123:0/1195432626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c06caf0 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.869+0000 7f0d52000640 1 -- 172.21.15.123:0/1195432626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c06d0b0 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.870+0000 7f0d52000640 1 -- 172.21.15.123:0/1195432626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d4c068a20 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.871+0000 7f0d43400640 1 -- 172.21.15.123:0/1195432626 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d44011d10 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.872+0000 7f0d43400640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0d2c06c8f0 0x7f0d2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:32.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.872+0000 7f0d43400640 1 -- 172.21.15.123:0/1195432626 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0d4408bd10 con 0x7f0d4c0d45a0 2024-04-24T05:46:32.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.872+0000 7f0d4b400640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0d2c06c8f0 0x7f0d2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:32.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.873+0000 7f0d4b400640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0d2c06c8f0 0x7f0d2c06edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c033000 tx=0x7f0d3c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:32.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:32.876+0000 7f0d43400640 1 -- 172.21.15.123:0/1195432626 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d44091020 con 0x7f0d4c0d45a0 2024-04-24T05:46:33.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.074+0000 7f0d52000640 1 -- 172.21.15.123:0/1195432626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0d4c0b5750 con 0x7f0d4c0d45a0 2024-04-24T05:46:33.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.075+0000 7f0d43400640 1 -- 172.21.15.123:0/1195432626 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f0d4c15b0c0 con 0x7f0d4c0d45a0 2024-04-24T05:46:33.077 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:33.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.078+0000 7f0d52000640 1 -- 172.21.15.123:0/1195432626 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0d2c06c8f0 msgr2=0x7f0d2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:33.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.078+0000 7f0d52000640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0d2c06c8f0 0x7f0d2c06edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c033000 tx=0x7f0d3c009000 comp rx=0 tx=0).stop 2024-04-24T05:46:33.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.078+0000 7f0d52000640 1 -- 172.21.15.123:0/1195432626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d4c0d45a0 msgr2=0x7f0d4c06db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:33.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.078+0000 7f0d52000640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d4c0d45a0 0x7f0d4c06db90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400c440 tx=0x7f0d4400c900 comp rx=0 tx=0).stop 2024-04-24T05:46:33.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.078+0000 7f0d52000640 1 -- 172.21.15.123:0/1195432626 shutdown_connections 2024-04-24T05:46:33.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.078+0000 7f0d52000640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0d2c06c8f0 0x7f0d2c06edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.078+0000 7f0d52000640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d4c0d45a0 0x7f0d4c06db90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.078+0000 7f0d52000640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d4c0cfac0 0x7f0d4c15ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.078+0000 7f0d52000640 1 --2- 172.21.15.123:0/1195432626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d4c0b35e0 0x7f0d4c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.078+0000 7f0d52000640 1 -- 172.21.15.123:0/1195432626 >> 172.21.15.123:0/1195432626 conn(0x7f0d4c09d970 msgr2=0x7f0d4c0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:33.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.078+0000 7f0d52000640 1 -- 172.21.15.123:0/1195432626 shutdown_connections 2024-04-24T05:46:33.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.078+0000 7f0d52000640 1 -- 172.21.15.123:0/1195432626 wait complete. 2024-04-24T05:46:33.098 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T05:46:33.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.293+0000 7f0f5a400640 1 Processor -- start 2024-04-24T05:46:33.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.293+0000 7f0f5a400640 1 -- start start 2024-04-24T05:46:33.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.295+0000 7f0f5a400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f540ad860 0x7f0f540af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:33.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.295+0000 7f0f5a400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0f540af780 0x7f0f5406bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:33.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.295+0000 7f0f5a400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f540afbd0 con 0x7f0f5415a2a0 2024-04-24T05:46:33.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.295+0000 7f0f5a400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0f5415a680 con 0x7f0f540af780 2024-04-24T05:46:33.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.295+0000 7f0f5a400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5415da00 con 0x7f0f540ad860 2024-04-24T05:46:33.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.295+0000 7f0f53400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0f5415a2a0 0x7f0f540ad150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49316/0 (socket says 172.21.15.123:49316) 2024-04-24T05:46:33.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.295+0000 7f0f53400640 1 -- 172.21.15.123:0/2667285010 learned_addr learned my addr 172.21.15.123:0/2667285010 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:33.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f4aa00640 1 --2- 172.21.15.123:0/2667285010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f540ad860 0x7f0f540af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:33.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f53e00640 1 --2- 172.21.15.123:0/2667285010 >> v2:172.21.15.155:3300/0 conn(0x7f0f540af780 0x7f0f5406bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:33.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 -- 172.21.15.123:0/2667285010 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2036395944 0 0) 0x7f0f540afbd0 con 0x7f0f5415a2a0 2024-04-24T05:46:33.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 -- 172.21.15.123:0/2667285010 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f38003520 con 0x7f0f5415a2a0 2024-04-24T05:46:33.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 -- 172.21.15.123:0/2667285010 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2883250374 0 0) 0x7f0f38003520 con 0x7f0f5415a2a0 2024-04-24T05:46:33.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 -- 172.21.15.123:0/2667285010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f540ad860 msgr2=0x7f0f540af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:33.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 --2- 172.21.15.123:0/2667285010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f540ad860 0x7f0f540af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 -- 172.21.15.123:0/2667285010 >> v2:172.21.15.155:3300/0 conn(0x7f0f540af780 msgr2=0x7f0f5406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:33.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 --2- 172.21.15.123:0/2667285010 >> v2:172.21.15.155:3300/0 conn(0x7f0f540af780 0x7f0f5406bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 -- 172.21.15.123:0/2667285010 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5415cdf0 con 0x7f0f5415a2a0 2024-04-24T05:46:33.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 -- 172.21.15.123:0/2667285010 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f0f440019f0 con 0x7f0f5415a2a0 2024-04-24T05:46:33.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 -- 172.21.15.123:0/2667285010 >> v1:172.21.15.123:6789/0 conn(0x7f0f5415a2a0 legacy=0x7f0f540ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:33.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 --2- 172.21.15.123:0/2667285010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f540ad860 0x7f0f380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:33.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 --2- 172.21.15.123:0/2667285010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f38003f80 0x7f0f38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:33.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 --2- 172.21.15.123:0/2667285010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f38004d40 0x7f0f380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:33.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 -- 172.21.15.123:0/2667285010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5415a680 con 0x7f0f38003f80 2024-04-24T05:46:33.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 -- 172.21.15.123:0/2667285010 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5415da00 con 0x7f0f38004d40 2024-04-24T05:46:33.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.296+0000 7f0f52a00640 1 -- 172.21.15.123:0/2667285010 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0f3800ba70 con 0x7f0f540ad860 2024-04-24T05:46:33.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.297+0000 7f0f4aa00640 1 --2- 172.21.15.123:0/2667285010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f38003f80 0x7f0f38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:33.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.297+0000 7f0f53e00640 1 --2- 172.21.15.123:0/2667285010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f38004d40 0x7f0f380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:33.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.297+0000 7f0f53400640 1 --2- 172.21.15.123:0/2667285010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f540ad860 0x7f0f380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:33.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.297+0000 7f0f4aa00640 1 -- 172.21.15.123:0/2667285010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f540ad860 msgr2=0x7f0f380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:33.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.297+0000 7f0f4aa00640 1 --2- 172.21.15.123:0/2667285010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f540ad860 0x7f0f380039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.297+0000 7f0f4aa00640 1 -- 172.21.15.123:0/2667285010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f38004d40 msgr2=0x7f0f380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:33.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.297+0000 7f0f4aa00640 1 --2- 172.21.15.123:0/2667285010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f38004d40 0x7f0f380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.297+0000 7f0f4aa00640 1 -- 172.21.15.123:0/2667285010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f38003520 con 0x7f0f38003f80 2024-04-24T05:46:33.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.298+0000 7f0f4aa00640 1 --2- 172.21.15.123:0/2667285010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f38003f80 0x7f0f38004800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009ac0 tx=0x7f0f38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441e5b74b99459a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:33.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.298+0000 7f0f52a00640 1 -- 172.21.15.123:0/2667285010 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c036070 con 0x7f0f38003f80 2024-04-24T05:46:33.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.298+0000 7f0f52a00640 1 -- 172.21.15.123:0/2667285010 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f3c031400 con 0x7f0f38003f80 2024-04-24T05:46:33.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.298+0000 7f0f52a00640 1 -- 172.21.15.123:0/2667285010 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c009690 con 0x7f0f38003f80 2024-04-24T05:46:33.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.299+0000 7f0f5a400640 1 -- 172.21.15.123:0/2667285010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f38003f80 msgr2=0x7f0f38004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:33.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.299+0000 7f0f5a400640 1 --2- 172.21.15.123:0/2667285010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f38003f80 0x7f0f38004800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009ac0 tx=0x7f0f38005830 comp rx=0 tx=0).stop 2024-04-24T05:46:33.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.299+0000 7f0f53e00640 1 -- 172.21.15.123:0/2667285010 reap_dead start 2024-04-24T05:46:33.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.299+0000 7f0f5a400640 1 -- 172.21.15.123:0/2667285010 shutdown_connections 2024-04-24T05:46:33.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.299+0000 7f0f5a400640 1 -- 172.21.15.123:0/2667285010 >> 172.21.15.123:0/2667285010 conn(0x7f0f540b1e70 msgr2=0x7f0f540b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:33.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.299+0000 7f0f5a400640 1 -- 172.21.15.123:0/2667285010 shutdown_connections 2024-04-24T05:46:33.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.299+0000 7f0f5a400640 1 -- 172.21.15.123:0/2667285010 wait complete. 2024-04-24T05:46:33.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.301+0000 7f0f5a400640 1 Processor -- start 2024-04-24T05:46:33.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.301+0000 7f0f5a400640 1 -- start start 2024-04-24T05:46:33.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.302+0000 7f0f5a400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f38003f80 0x7f0f54154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:33.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.302+0000 7f0f5a400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f540ccd70 0x7f0f540ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:33.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.302+0000 7f0f5a400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f540cd1f0 0x7f0f540ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:33.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.302+0000 7f0f5a400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5415d690 con 0x7f0f540cd1f0 2024-04-24T05:46:33.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.302+0000 7f0f5a400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5415d3b0 con 0x7f0f38003f80 2024-04-24T05:46:33.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.302+0000 7f0f5a400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0f540af240 con 0x7f0f540ccd70 2024-04-24T05:46:33.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.302+0000 7f0f53e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f540cd1f0 0x7f0f540ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:33.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.302+0000 7f0f53e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f540cd1f0 0x7f0f540ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50344/0 (socket says 172.21.15.123:50344) 2024-04-24T05:46:33.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.302+0000 7f0f53e00640 1 -- 172.21.15.123:0/2362696863 learned_addr learned my addr 172.21.15.123:0/2362696863 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:33.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.302+0000 7f0f4aa00640 1 --2- 172.21.15.123:0/2362696863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f540ccd70 0x7f0f540ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:33.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.302+0000 7f0f53400640 1 --2- 172.21.15.123:0/2362696863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f38003f80 0x7f0f54154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:33.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.303+0000 7f0f53e00640 1 -- 172.21.15.123:0/2362696863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f540ccd70 msgr2=0x7f0f540ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:33.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.303+0000 7f0f53e00640 1 --2- 172.21.15.123:0/2362696863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f540ccd70 0x7f0f540ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.303+0000 7f0f53e00640 1 -- 172.21.15.123:0/2362696863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f38003f80 msgr2=0x7f0f54154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:33.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.303+0000 7f0f53e00640 1 --2- 172.21.15.123:0/2362696863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f38003f80 0x7f0f54154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.303+0000 7f0f53e00640 1 -- 172.21.15.123:0/2362696863 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f54071db0 con 0x7f0f540cd1f0 2024-04-24T05:46:33.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.303+0000 7f0f53e00640 1 --2- 172.21.15.123:0/2362696863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f540cd1f0 0x7f0f540ceeb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0f5406da30 tx=0x7f0f4c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:33.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.303+0000 7f0f4be00640 1 -- 172.21.15.123:0/2362696863 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c019020 con 0x7f0f540cd1f0 2024-04-24T05:46:33.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.303+0000 7f0f4be00640 1 -- 172.21.15.123:0/2362696863 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f4c004440 con 0x7f0f540cd1f0 2024-04-24T05:46:33.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.303+0000 7f0f5a400640 1 -- 172.21.15.123:0/2362696863 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f541558f0 con 0x7f0f540cd1f0 2024-04-24T05:46:33.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.303+0000 7f0f4be00640 1 -- 172.21.15.123:0/2362696863 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c003830 con 0x7f0f540cd1f0 2024-04-24T05:46:33.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.303+0000 7f0f5a400640 1 -- 172.21.15.123:0/2362696863 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f54072070 con 0x7f0f540cd1f0 2024-04-24T05:46:33.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.304+0000 7f0f5a400640 1 -- 172.21.15.123:0/2362696863 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f54068a20 con 0x7f0f540cd1f0 2024-04-24T05:46:33.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.305+0000 7f0f4be00640 1 -- 172.21.15.123:0/2362696863 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f4c01c020 con 0x7f0f540cd1f0 2024-04-24T05:46:33.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.306+0000 7f0f4be00640 1 --2- 172.21.15.123:0/2362696863 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0f3006c920 0x7f0f3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:33.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.306+0000 7f0f4be00640 1 -- 172.21.15.123:0/2362696863 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0f4c08c020 con 0x7f0f540cd1f0 2024-04-24T05:46:33.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.306+0000 7f0f53400640 1 --2- 172.21.15.123:0/2362696863 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0f3006c920 0x7f0f3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:33.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.307+0000 7f0f53400640 1 --2- 172.21.15.123:0/2362696863 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0f3006c920 0x7f0f3006ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0f440019b0 tx=0x7f0f44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:33.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.311+0000 7f0f4be00640 1 -- 172.21.15.123:0/2362696863 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f4c023030 con 0x7f0f540cd1f0 2024-04-24T05:46:33.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.552+0000 7f0f5a400640 1 -- 172.21.15.123:0/2362696863 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f5406fdf0 con 0x7f0f540cd1f0 2024-04-24T05:46:33.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.553+0000 7f0f4be00640 1 -- 172.21.15.123:0/2362696863 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f0f4c05a110 con 0x7f0f540cd1f0 2024-04-24T05:46:33.555 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:33.555 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2743935658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2743935658},{"type":"v1","addr":"172.21.15.155:6834","nonce":2743935658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1148561953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1148561953},{"type":"v1","addr":"172.21.15.123:6836","nonce":1148561953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2992052647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2992052647},{"type":"v1","addr":"172.21.15.155:6835","nonce":2992052647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T05:45:59.639548+0000","modified":"2024-04-24T05:46:19.813408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/3370584410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3370584410},{"type":"v1","addr":"172.21.15.123:6837","nonce":3370584410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T05:46:33.555 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-24T05:46:33.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.556+0000 7f0f5a400640 1 -- 172.21.15.123:0/2362696863 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0f3006c920 msgr2=0x7f0f3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.556+0000 7f0f5a400640 1 --2- 172.21.15.123:0/2362696863 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0f3006c920 0x7f0f3006ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0f440019b0 tx=0x7f0f44009040 comp rx=0 tx=0).stop 2024-04-24T05:46:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.557+0000 7f0f5a400640 1 -- 172.21.15.123:0/2362696863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f540cd1f0 msgr2=0x7f0f540ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.557+0000 7f0f5a400640 1 --2- 172.21.15.123:0/2362696863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f540cd1f0 0x7f0f540ceeb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0f5406da30 tx=0x7f0f4c011920 comp rx=0 tx=0).stop 2024-04-24T05:46:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.557+0000 7f0f5a400640 1 -- 172.21.15.123:0/2362696863 shutdown_connections 2024-04-24T05:46:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.557+0000 7f0f5a400640 1 --2- 172.21.15.123:0/2362696863 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0f3006c920 0x7f0f3006ede0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.557+0000 7f0f5a400640 1 --2- 172.21.15.123:0/2362696863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f540cd1f0 0x7f0f540ceeb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.557+0000 7f0f5a400640 1 --2- 172.21.15.123:0/2362696863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f540ccd70 0x7f0f540ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.557+0000 7f0f5a400640 1 --2- 172.21.15.123:0/2362696863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f38003f80 0x7f0f54154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.557+0000 7f0f5a400640 1 -- 172.21.15.123:0/2362696863 >> 172.21.15.123:0/2362696863 conn(0x7f0f540b1e70 msgr2=0x7f0f5415f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.557+0000 7f0f5a400640 1 -- 172.21.15.123:0/2362696863 shutdown_connections 2024-04-24T05:46:33.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.557+0000 7f0f5a400640 1 -- 172.21.15.123:0/2362696863 wait complete. 2024-04-24T05:46:33.578 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:33.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.770+0000 7ff5ebc00640 1 Processor -- start 2024-04-24T05:46:33.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.770+0000 7ff5ebc00640 1 -- start start 2024-04-24T05:46:33.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.772+0000 7ff5ebc00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff5e40af950 0x7ff5e4167580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:33.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.772+0000 7ff5ebc00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e415cdf0 con 0x7ff5e415a2a0 2024-04-24T05:46:33.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.772+0000 7ff5ebc00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e415d0d0 con 0x7ff5e40ad860 2024-04-24T05:46:33.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.772+0000 7ff5ebc00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e415a680 con 0x7ff5e40af950 2024-04-24T05:46:33.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.772+0000 7ff5e9c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff5e40af950 0x7ff5e4167580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:33.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.772+0000 7ff5e9c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff5e40af950 0x7ff5e4167580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50348/0 (socket says 172.21.15.123:50348) 2024-04-24T05:46:33.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.772+0000 7ff5e9c00640 1 -- 172.21.15.123:0/1441243210 learned_addr learned my addr 172.21.15.123:0/1441243210 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:33.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.773+0000 7ff5e9c00640 1 -- 172.21.15.123:0/1441243210 >> v1:172.21.15.155:6789/0 conn(0x7ff5e40ad860 legacy=0x7ff5e40af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:33.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.773+0000 7ff5e9c00640 1 -- 172.21.15.123:0/1441243210 >> v1:172.21.15.123:6789/0 conn(0x7ff5e415a2a0 legacy=0x7ff5e40ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:33.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.773+0000 7ff5e9c00640 1 -- 172.21.15.123:0/1441243210 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e415d3b0 con 0x7ff5e40af950 2024-04-24T05:46:33.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.774+0000 7ff5e9c00640 1 --2- 172.21.15.123:0/1441243210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e40af950 0x7ff5e4167580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009e50 tx=0x7ff5dc02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112049cdf91a12c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:33.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.774+0000 7ff5e3e00640 1 -- 172.21.15.123:0/1441243210 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc036070 con 0x7ff5e40af950 2024-04-24T05:46:33.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.774+0000 7ff5e3e00640 1 -- 172.21.15.123:0/1441243210 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5dc031600 con 0x7ff5e40af950 2024-04-24T05:46:33.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.774+0000 7ff5e3e00640 1 -- 172.21.15.123:0/1441243210 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc0319d0 con 0x7ff5e40af950 2024-04-24T05:46:33.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.775+0000 7ff5ebc00640 1 -- 172.21.15.123:0/1441243210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e40af950 msgr2=0x7ff5e4167580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:33.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.775+0000 7ff5ebc00640 1 --2- 172.21.15.123:0/1441243210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e40af950 0x7ff5e4167580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009e50 tx=0x7ff5dc02a6d0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.775+0000 7ff5ebc00640 1 -- 172.21.15.123:0/1441243210 shutdown_connections 2024-04-24T05:46:33.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.775+0000 7ff5ebc00640 1 --2- 172.21.15.123:0/1441243210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e40af950 0x7ff5e4167580 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.775+0000 7ff5ebc00640 1 -- 172.21.15.123:0/1441243210 >> 172.21.15.123:0/1441243210 conn(0x7ff5e40b1e70 msgr2=0x7ff5e40b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:33.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.775+0000 7ff5ebc00640 1 -- 172.21.15.123:0/1441243210 shutdown_connections 2024-04-24T05:46:33.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.775+0000 7ff5ebc00640 1 -- 172.21.15.123:0/1441243210 wait complete. 2024-04-24T05:46:33.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.777+0000 7ff5ebc00640 1 Processor -- start 2024-04-24T05:46:33.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.777+0000 7ff5ebc00640 1 -- start start 2024-04-24T05:46:33.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.778+0000 7ff5ebc00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e40ad860 0x7ff5e414e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:33.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.778+0000 7ff5ebc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e40af950 0x7ff5e4153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:33.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.778+0000 7ff5ebc00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e415a2a0 0x7ff5e415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:33.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.778+0000 7ff5ebc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e406cb10 con 0x7ff5e40af950 2024-04-24T05:46:33.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.778+0000 7ff5ebc00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e415da00 con 0x7ff5e40ad860 2024-04-24T05:46:33.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.778+0000 7ff5ebc00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e406cc90 con 0x7ff5e415a2a0 2024-04-24T05:46:33.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.778+0000 7ff5dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e40af950 0x7ff5e4153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:33.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.778+0000 7ff5dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e40af950 0x7ff5e4153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50358/0 (socket says 172.21.15.123:50358) 2024-04-24T05:46:33.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.778+0000 7ff5dbe00640 1 -- 172.21.15.123:0/2642660625 learned_addr learned my addr 172.21.15.123:0/2642660625 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:33.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.779+0000 7ff5e9200640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e40ad860 0x7ff5e414e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:33.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.779+0000 7ff5e9c00640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e415a2a0 0x7ff5e415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:33.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.779+0000 7ff5dbe00640 1 -- 172.21.15.123:0/2642660625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e415a2a0 msgr2=0x7ff5e415ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:33.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.779+0000 7ff5dbe00640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e415a2a0 0x7ff5e415ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.779+0000 7ff5dbe00640 1 -- 172.21.15.123:0/2642660625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e40ad860 msgr2=0x7ff5e414e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:33.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.779+0000 7ff5dbe00640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e40ad860 0x7ff5e414e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.779+0000 7ff5dbe00640 1 -- 172.21.15.123:0/2642660625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e40cb1f0 con 0x7ff5e40af950 2024-04-24T05:46:33.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.779+0000 7ff5e9c00640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e415a2a0 0x7ff5e415ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:46:33.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.779+0000 7ff5dbe00640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e40af950 0x7ff5e4153a60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00b420 tx=0x7ff5cc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:33.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.780+0000 7ff5e1600640 1 -- 172.21.15.123:0/2642660625 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc011830 con 0x7ff5e40af950 2024-04-24T05:46:33.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.780+0000 7ff5e1600640 1 -- 172.21.15.123:0/2642660625 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5cc016070 con 0x7ff5e40af950 2024-04-24T05:46:33.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.780+0000 7ff5ebc00640 1 -- 172.21.15.123:0/2642660625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e40cb4d0 con 0x7ff5e40af950 2024-04-24T05:46:33.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.780+0000 7ff5e1600640 1 -- 172.21.15.123:0/2642660625 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc011c10 con 0x7ff5e40af950 2024-04-24T05:46:33.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.780+0000 7ff5ebc00640 1 -- 172.21.15.123:0/2642660625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e4154420 con 0x7ff5e40af950 2024-04-24T05:46:33.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.781+0000 7ff5ebc00640 1 -- 172.21.15.123:0/2642660625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e4068a20 con 0x7ff5e40af950 2024-04-24T05:46:33.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.782+0000 7ff5e1600640 1 -- 172.21.15.123:0/2642660625 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5cc01b460 con 0x7ff5e40af950 2024-04-24T05:46:33.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.783+0000 7ff5e1600640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5c006c8f0 0x7ff5c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:33.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.783+0000 7ff5e1600640 1 -- 172.21.15.123:0/2642660625 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff5cc08b9b0 con 0x7ff5e40af950 2024-04-24T05:46:33.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.783+0000 7ff5e9200640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5c006c8f0 0x7ff5c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:33.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.784+0000 7ff5e9200640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5c006c8f0 0x7ff5c006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4001780 tx=0x7ff5d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:33.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.788+0000 7ff5e1600640 1 -- 172.21.15.123:0/2642660625 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5cc020020 con 0x7ff5e40af950 2024-04-24T05:46:33.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.989+0000 7ff5ebc00640 1 -- 172.21.15.123:0/2642660625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5e406fdf0 con 0x7ff5e40af950 2024-04-24T05:46:33.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.992+0000 7ff5e1600640 1 -- 172.21.15.123:0/2642660625 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7ff5cc059a00 con 0x7ff5e40af950 2024-04-24T05:46:33.994 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:33.995 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:04.715857+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:33.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.995+0000 7ff5ebc00640 1 -- 172.21.15.123:0/2642660625 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5c006c8f0 msgr2=0x7ff5c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:33.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.995+0000 7ff5ebc00640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5c006c8f0 0x7ff5c006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4001780 tx=0x7ff5d4009040 comp rx=0 tx=0).stop 2024-04-24T05:46:33.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.995+0000 7ff5ebc00640 1 -- 172.21.15.123:0/2642660625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e40af950 msgr2=0x7ff5e4153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:33.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.995+0000 7ff5ebc00640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e40af950 0x7ff5e4153a60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00b420 tx=0x7ff5cc00b8e0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.995+0000 7ff5ebc00640 1 -- 172.21.15.123:0/2642660625 shutdown_connections 2024-04-24T05:46:33.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.995+0000 7ff5ebc00640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5c006c8f0 0x7ff5c006edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.995+0000 7ff5ebc00640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5e415a2a0 0x7ff5e415ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.995+0000 7ff5ebc00640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e40af950 0x7ff5e4153a60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.995+0000 7ff5ebc00640 1 --2- 172.21.15.123:0/2642660625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5e40ad860 0x7ff5e414e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:33.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.995+0000 7ff5ebc00640 1 -- 172.21.15.123:0/2642660625 >> 172.21.15.123:0/2642660625 conn(0x7ff5e40b1e70 msgr2=0x7ff5e40d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:33.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.995+0000 7ff5ebc00640 1 -- 172.21.15.123:0/2642660625 shutdown_connections 2024-04-24T05:46:33.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:33.995+0000 7ff5ebc00640 1 -- 172.21.15.123:0/2642660625 wait complete. 2024-04-24T05:46:34.017 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T05:46:34.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.213+0000 7f7933e00640 1 Processor -- start 2024-04-24T05:46:34.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.213+0000 7f7933e00640 1 -- start start 2024-04-24T05:46:34.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.214+0000 7f7933e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f792c15a2a0 0x7f792c0ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:34.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.214+0000 7f7933e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f792c0ad690 0x7f792c0af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:34.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.215+0000 7f7933e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f792c0afbf0 con 0x7f792c0af780 2024-04-24T05:46:34.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.215+0000 7f7933e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f792c0adaf0 con 0x7f792c0ad690 2024-04-24T05:46:34.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.215+0000 7f7933e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f792c15a680 con 0x7f792c15a2a0 2024-04-24T05:46:34.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.215+0000 7f7931400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f792c15a2a0 0x7f792c0ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:34.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.215+0000 7f7931400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f792c15a2a0 0x7f792c0ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:34534/0 (socket says 172.21.15.123:34534) 2024-04-24T05:46:34.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.215+0000 7f7931400640 1 -- 172.21.15.123:0/940911811 learned_addr learned my addr 172.21.15.123:0/940911811 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:34.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.215+0000 7f7930a00640 1 --2- 172.21.15.123:0/940911811 >> v2:172.21.15.155:3300/0 conn(0x7f792c0ad690 0x7f792c0af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:34.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.216+0000 7f7930a00640 1 -- 172.21.15.123:0/940911811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f792c15a2a0 msgr2=0x7f792c0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:34.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.216+0000 7f7930a00640 1 --2- 172.21.15.123:0/940911811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f792c15a2a0 0x7f792c0ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:34.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.216+0000 7f7930a00640 1 -- 172.21.15.123:0/940911811 >> v1:172.21.15.155:6789/0 conn(0x7f792c0af780 legacy=0x7f792c141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:34.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.216+0000 7f7930a00640 1 -- 172.21.15.123:0/940911811 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c15cdf0 con 0x7f792c0ad690 2024-04-24T05:46:34.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.218+0000 7f7930a00640 1 --2- 172.21.15.123:0/940911811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f792c0ad690 0x7f792c0af240 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f791c0097a0 tx=0x7f791c02a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eaf7c60cb6150d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:34.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.218+0000 7f792be00640 1 -- 172.21.15.123:0/940911811 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c037070 con 0x7f792c0ad690 2024-04-24T05:46:34.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.218+0000 7f792be00640 1 -- 172.21.15.123:0/940911811 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f791c02ae40 con 0x7f792c0ad690 2024-04-24T05:46:34.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.218+0000 7f792be00640 1 -- 172.21.15.123:0/940911811 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c032660 con 0x7f792c0ad690 2024-04-24T05:46:34.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.219+0000 7f7933e00640 1 -- 172.21.15.123:0/940911811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f792c0ad690 msgr2=0x7f792c0af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:34.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.219+0000 7f7933e00640 1 --2- 172.21.15.123:0/940911811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f792c0ad690 0x7f792c0af240 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f791c0097a0 tx=0x7f791c02a0f0 comp rx=0 tx=0).stop 2024-04-24T05:46:34.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.219+0000 7f7933e00640 1 -- 172.21.15.123:0/940911811 shutdown_connections 2024-04-24T05:46:34.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.219+0000 7f7933e00640 1 --2- 172.21.15.123:0/940911811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f792c0ad690 0x7f792c0af240 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:34.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.219+0000 7f7933e00640 1 --2- 172.21.15.123:0/940911811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f792c15a2a0 0x7f792c0ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:34.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.219+0000 7f7933e00640 1 -- 172.21.15.123:0/940911811 >> 172.21.15.123:0/940911811 conn(0x7f792c0b1e70 msgr2=0x7f792c0b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:34.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.219+0000 7f7933e00640 1 -- 172.21.15.123:0/940911811 shutdown_connections 2024-04-24T05:46:34.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.219+0000 7f7933e00640 1 -- 172.21.15.123:0/940911811 wait complete. 2024-04-24T05:46:34.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.221+0000 7f7933e00640 1 Processor -- start 2024-04-24T05:46:34.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.221+0000 7f7933e00640 1 -- start start 2024-04-24T05:46:34.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.222+0000 7f7933e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f792c0ad690 0x7f792c15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:34.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.222+0000 7f7933e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f792c0af780 0x7f792c15bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:34.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.222+0000 7f7933e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f792c15a2a0 0x7f792c071e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:34.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.222+0000 7f7933e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f792c15d3b0 con 0x7f792c0ad690 2024-04-24T05:46:34.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.222+0000 7f7933e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f792c15da00 con 0x7f792c15a2a0 2024-04-24T05:46:34.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.222+0000 7f7933e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f792c15d690 con 0x7f792c0af780 2024-04-24T05:46:34.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.222+0000 7f7931400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f792c0ad690 0x7f792c15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:34.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.222+0000 7f7931400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f792c0ad690 0x7f792c15ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50364/0 (socket says 172.21.15.123:50364) 2024-04-24T05:46:34.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.222+0000 7f7931400640 1 -- 172.21.15.123:0/1184796774 learned_addr learned my addr 172.21.15.123:0/1184796774 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:34.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.223+0000 7f7931e00640 1 --2- 172.21.15.123:0/1184796774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f792c15a2a0 0x7f792c071e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:34.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.223+0000 7f7930a00640 1 --2- 172.21.15.123:0/1184796774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f792c0af780 0x7f792c15bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:34.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.223+0000 7f7931400640 1 -- 172.21.15.123:0/1184796774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f792c0af780 msgr2=0x7f792c15bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:34.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.223+0000 7f7931400640 1 --2- 172.21.15.123:0/1184796774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f792c0af780 0x7f792c15bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:34.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.223+0000 7f7931400640 1 -- 172.21.15.123:0/1184796774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f792c15a2a0 msgr2=0x7f792c071e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:34.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.223+0000 7f7931400640 1 --2- 172.21.15.123:0/1184796774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f792c15a2a0 0x7f792c071e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:34.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.223+0000 7f7931400640 1 -- 172.21.15.123:0/1184796774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c072b80 con 0x7f792c0ad690 2024-04-24T05:46:34.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.223+0000 7f7931400640 1 --2- 172.21.15.123:0/1184796774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f792c0ad690 0x7f792c15ba20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7920009fd0 tx=0x7f792000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:34.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.224+0000 7f7929600640 1 -- 172.21.15.123:0/1184796774 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920017070 con 0x7f792c0ad690 2024-04-24T05:46:34.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.224+0000 7f7929600640 1 -- 172.21.15.123:0/1184796774 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f792000c8e0 con 0x7f792c0ad690 2024-04-24T05:46:34.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.224+0000 7f7929600640 1 -- 172.21.15.123:0/1184796774 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792000ccb0 con 0x7f792c0ad690 2024-04-24T05:46:34.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.224+0000 7f7933e00640 1 -- 172.21.15.123:0/1184796774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792c072e60 con 0x7f792c0ad690 2024-04-24T05:46:34.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.224+0000 7f7933e00640 1 -- 172.21.15.123:0/1184796774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792c073420 con 0x7f792c0ad690 2024-04-24T05:46:34.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.225+0000 7f7933e00640 1 -- 172.21.15.123:0/1184796774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792c06c650 con 0x7f792c0ad690 2024-04-24T05:46:34.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.226+0000 7f7929600640 1 -- 172.21.15.123:0/1184796774 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f792001a070 con 0x7f792c0ad690 2024-04-24T05:46:34.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.227+0000 7f7929600640 1 --2- 172.21.15.123:0/1184796774 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f791006c8f0 0x7f791006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:34.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.227+0000 7f7929600640 1 -- 172.21.15.123:0/1184796774 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f792008c1f0 con 0x7f792c0ad690 2024-04-24T05:46:34.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.227+0000 7f7930a00640 1 --2- 172.21.15.123:0/1184796774 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f791006c8f0 0x7f791006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:34.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.228+0000 7f7930a00640 1 --2- 172.21.15.123:0/1184796774 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f791006c8f0 0x7f791006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f791c034000 tx=0x7f791c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:34.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.232+0000 7f7929600640 1 -- 172.21.15.123:0/1184796774 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f792c06c650 con 0x7f792c0ad690 2024-04-24T05:46:34.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.476+0000 7f7933e00640 1 -- 172.21.15.123:0/1184796774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f792c15ca70 con 0x7f792c0ad690 2024-04-24T05:46:34.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.477+0000 7f7929600640 1 -- 172.21.15.123:0/1184796774 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f792c15ca70 con 0x7f792c0ad690 2024-04-24T05:46:34.479 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:34.479 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2743935658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2743935658},{"type":"v1","addr":"172.21.15.155:6834","nonce":2743935658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1148561953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1148561953},{"type":"v1","addr":"172.21.15.123:6836","nonce":1148561953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2992052647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2992052647},{"type":"v1","addr":"172.21.15.155:6835","nonce":2992052647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T05:45:59.639548+0000","modified":"2024-04-24T05:46:19.813408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/3370584410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3370584410},{"type":"v1","addr":"172.21.15.123:6837","nonce":3370584410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T05:46:34.480 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-24T05:46:34.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.480+0000 7f7933e00640 1 -- 172.21.15.123:0/1184796774 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f791006c8f0 msgr2=0x7f791006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:34.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.480+0000 7f7933e00640 1 --2- 172.21.15.123:0/1184796774 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f791006c8f0 0x7f791006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f791c034000 tx=0x7f791c02e000 comp rx=0 tx=0).stop 2024-04-24T05:46:34.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.480+0000 7f7933e00640 1 -- 172.21.15.123:0/1184796774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f792c0ad690 msgr2=0x7f792c15ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:34.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.480+0000 7f7933e00640 1 --2- 172.21.15.123:0/1184796774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f792c0ad690 0x7f792c15ba20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7920009fd0 tx=0x7f792000ed40 comp rx=0 tx=0).stop 2024-04-24T05:46:34.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.481+0000 7f7933e00640 1 -- 172.21.15.123:0/1184796774 shutdown_connections 2024-04-24T05:46:34.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.481+0000 7f7933e00640 1 --2- 172.21.15.123:0/1184796774 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f791006c8f0 0x7f791006edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:34.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.481+0000 7f7933e00640 1 --2- 172.21.15.123:0/1184796774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f792c15a2a0 0x7f792c071e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:34.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.481+0000 7f7933e00640 1 --2- 172.21.15.123:0/1184796774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f792c0af780 0x7f792c15bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:34.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.481+0000 7f7933e00640 1 --2- 172.21.15.123:0/1184796774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f792c0ad690 0x7f792c15ba20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:34.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.481+0000 7f7933e00640 1 -- 172.21.15.123:0/1184796774 >> 172.21.15.123:0/1184796774 conn(0x7f792c0b1e70 msgr2=0x7f792c15e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:34.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.481+0000 7f7933e00640 1 -- 172.21.15.123:0/1184796774 shutdown_connections 2024-04-24T05:46:34.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:34.481+0000 7f7933e00640 1 -- 172.21.15.123:0/1184796774 wait complete. 2024-04-24T05:46:34.904 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T05:45:59.639548+0000', 'modified': '2024-04-24T05:46:19.813408+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4273}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4273': {'gid': 4273, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.123:6837/3370584410', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 3370584410}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 3370584410}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4273, 'qdb_cluster': [4273]} 2024-04-24T05:46:34.904 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T05:46:34.904 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T05:46:35.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.114+0000 7f6fd5200640 1 Processor -- start 2024-04-24T05:46:35.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.114+0000 7f6fd5200640 1 -- start start 2024-04-24T05:46:35.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.116+0000 7f6fd5200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6fd015ab10 0x7f6fd015e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.116+0000 7f6fd5200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6fd0072640 0x7f6fd015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.116+0000 7f6fd5200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd00ad7c0 con 0x7f6fd015f1c0 2024-04-24T05:46:35.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.116+0000 7f6fd5200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6fd0073450 con 0x7f6fd015ab10 2024-04-24T05:46:35.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.116+0000 7f6fd5200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6fd0072bb0 con 0x7f6fd0072640 2024-04-24T05:46:35.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.116+0000 7f6fcde00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6fd015ab10 0x7f6fd015e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.116+0000 7f6fce800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6fd015f1c0 0x7f6fd006c290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49334/0 (socket says 172.21.15.123:49334) 2024-04-24T05:46:35.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.116+0000 7f6fce800640 1 -- 172.21.15.123:0/2887696503 learned_addr learned my addr 172.21.15.123:0/2887696503 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:35.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.117+0000 7f6fc5400640 1 --2- 172.21.15.123:0/2887696503 >> v2:172.21.15.155:3300/0 conn(0x7f6fd0072640 0x7f6fd015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.117+0000 7f6fcde00640 1 -- 172.21.15.123:0/2887696503 >> v2:172.21.15.155:3300/0 conn(0x7f6fd0072640 msgr2=0x7f6fd015ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:35.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.117+0000 7f6fcde00640 1 --2- 172.21.15.123:0/2887696503 >> v2:172.21.15.155:3300/0 conn(0x7f6fd0072640 0x7f6fd015ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.117+0000 7f6fcde00640 1 -- 172.21.15.123:0/2887696503 >> v1:172.21.15.123:6789/0 conn(0x7f6fd015f1c0 legacy=0x7f6fd006c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:35.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.117+0000 7f6fcde00640 1 -- 172.21.15.123:0/2887696503 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd0072e90 con 0x7f6fd015ab10 2024-04-24T05:46:35.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.118+0000 7f6fcde00640 1 --2- 172.21.15.123:0/2887696503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd015ab10 0x7f6fd015e740 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6fd009e4a0 tx=0x7f6fc002b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8bba1a56a2bc70 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:35.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.119+0000 7f6fcd400640 1 -- 172.21.15.123:0/2887696503 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc000b040 con 0x7f6fd015ab10 2024-04-24T05:46:35.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.119+0000 7f6fcd400640 1 -- 172.21.15.123:0/2887696503 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc0030710 con 0x7f6fd015ab10 2024-04-24T05:46:35.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.119+0000 7f6fcd400640 1 -- 172.21.15.123:0/2887696503 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0030ae0 con 0x7f6fd015ab10 2024-04-24T05:46:35.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.119+0000 7f6fd5200640 1 -- 172.21.15.123:0/2887696503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd015ab10 msgr2=0x7f6fd015e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:35.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.119+0000 7f6fd5200640 1 --2- 172.21.15.123:0/2887696503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd015ab10 0x7f6fd015e740 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6fd009e4a0 tx=0x7f6fc002b560 comp rx=0 tx=0).stop 2024-04-24T05:46:35.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.119+0000 7f6fd5200640 1 -- 172.21.15.123:0/2887696503 shutdown_connections 2024-04-24T05:46:35.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.119+0000 7f6fd5200640 1 --2- 172.21.15.123:0/2887696503 >> v2:172.21.15.155:3300/0 conn(0x7f6fd0072640 0x7f6fd015ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.119+0000 7f6fd5200640 1 --2- 172.21.15.123:0/2887696503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd015ab10 0x7f6fd015e740 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.119+0000 7f6fd5200640 1 -- 172.21.15.123:0/2887696503 >> 172.21.15.123:0/2887696503 conn(0x7f6fd00b1e70 msgr2=0x7f6fd00b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:35.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.119+0000 7f6fd5200640 1 -- 172.21.15.123:0/2887696503 shutdown_connections 2024-04-24T05:46:35.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.119+0000 7f6fd5200640 1 -- 172.21.15.123:0/2887696503 wait complete. 2024-04-24T05:46:35.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.121+0000 7f6fd5200640 1 Processor -- start 2024-04-24T05:46:35.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.121+0000 7f6fd5200640 1 -- start start 2024-04-24T05:46:35.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.122+0000 7f6fd5200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6fd0072640 0x7f6fd015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.122+0000 7f6fd5200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6fd015ab10 0x7f6fd01f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.124 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.122+0000 7f6fd5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd015f1c0 0x7f6fd01ed950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.122+0000 7f6fd5200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd015aef0 con 0x7f6fd015f1c0 2024-04-24T05:46:35.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.122+0000 7f6fd5200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd00ada40 con 0x7f6fd015ab10 2024-04-24T05:46:35.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.122+0000 7f6fd5200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd015b070 con 0x7f6fd0072640 2024-04-24T05:46:35.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.122+0000 7f6fce800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd015f1c0 0x7f6fd01ed950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.122+0000 7f6fce800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd015f1c0 0x7f6fd01ed950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50380/0 (socket says 172.21.15.123:50380) 2024-04-24T05:46:35.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.123+0000 7f6fce800640 1 -- 172.21.15.123:0/36837508 learned_addr learned my addr 172.21.15.123:0/36837508 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:35.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.123+0000 7f6fc5400640 1 --2- 172.21.15.123:0/36837508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6fd015ab10 0x7f6fd01f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.123+0000 7f6fcde00640 1 --2- 172.21.15.123:0/36837508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6fd0072640 0x7f6fd015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.123+0000 7f6fce800640 1 -- 172.21.15.123:0/36837508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6fd0072640 msgr2=0x7f6fd015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:35.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.123+0000 7f6fce800640 1 --2- 172.21.15.123:0/36837508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6fd0072640 0x7f6fd015ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.123+0000 7f6fce800640 1 -- 172.21.15.123:0/36837508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6fd015ab10 msgr2=0x7f6fd01f0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:35.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.123+0000 7f6fce800640 1 --2- 172.21.15.123:0/36837508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6fd015ab10 0x7f6fd01f0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.123+0000 7f6fce800640 1 -- 172.21.15.123:0/36837508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd01f14b0 con 0x7f6fd015f1c0 2024-04-24T05:46:35.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.123+0000 7f6fce800640 1 --2- 172.21.15.123:0/36837508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd015f1c0 0x7f6fd01ed950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800caa0 tx=0x7f6fc800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:35.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.123+0000 7f6fc6a00640 1 -- 172.21.15.123:0/36837508 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8017070 con 0x7f6fd015f1c0 2024-04-24T05:46:35.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.124+0000 7f6fc6a00640 1 -- 172.21.15.123:0/36837508 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc800ab40 con 0x7f6fd015f1c0 2024-04-24T05:46:35.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.124+0000 7f6fd5200640 1 -- 172.21.15.123:0/36837508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd015b9d0 con 0x7f6fd015f1c0 2024-04-24T05:46:35.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.124+0000 7f6fc6a00640 1 -- 172.21.15.123:0/36837508 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8012400 con 0x7f6fd015f1c0 2024-04-24T05:46:35.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.124+0000 7f6fd5200640 1 -- 172.21.15.123:0/36837508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd015bf90 con 0x7f6fd015f1c0 2024-04-24T05:46:35.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.126+0000 7f6fc6a00640 1 -- 172.21.15.123:0/36837508 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fc801a070 con 0x7f6fd015f1c0 2024-04-24T05:46:35.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.127+0000 7f6fc6a00640 1 --2- 172.21.15.123:0/36837508 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6fac06c920 0x7f6fac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.127+0000 7f6fc6a00640 1 -- 172.21.15.123:0/36837508 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6fc808ba70 con 0x7f6fd015f1c0 2024-04-24T05:46:35.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.127+0000 7f6fcde00640 1 --2- 172.21.15.123:0/36837508 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6fac06c920 0x7f6fac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.128+0000 7f6fd801f740 1 Processor -- start 2024-04-24T05:46:35.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.128+0000 7f6fd801f740 1 -- start start 2024-04-24T05:46:35.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.128+0000 7f6fcde00640 1 --2- 172.21.15.123:0/36837508 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6fac06c920 0x7f6fac06ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0035000 tx=0x7f6fc000c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:35.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.129+0000 7f6fd801f740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b30dbc0 0x56109b2c0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.129+0000 7f6fd801f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2f0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.129+0000 7f6fd801f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2f05a0 0x56109b2c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.129+0000 7f6fd801f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56109b226b10 con 0x56109b2f0ce0 2024-04-24T05:46:35.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.129+0000 7f6fd801f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56109b1cd2b0 con 0x56109b2f05a0 2024-04-24T05:46:35.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.129+0000 7f6fd801f740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56109b222ad0 con 0x56109b30dbc0 2024-04-24T05:46:35.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.129+0000 7f6fc5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.129+0000 7f6fc5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2f0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50394/0 (socket says 172.21.15.123:50394) 2024-04-24T05:46:35.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.129+0000 7f6fc5400640 1 -- 172.21.15.123:0/50658004 learned_addr learned my addr 172.21.15.123:0/50658004 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:35.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.130+0000 7f6fcde00640 1 --2- 172.21.15.123:0/50658004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b30dbc0 0x56109b2c0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.130+0000 7f6fc5400640 1 -- 172.21.15.123:0/50658004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b30dbc0 msgr2=0x56109b2c0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:35.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.130+0000 7f6fc5400640 1 --2- 172.21.15.123:0/50658004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b30dbc0 0x56109b2c0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.130+0000 7f6fce800640 1 --2- 172.21.15.123:0/50658004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2f05a0 0x56109b2c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.130+0000 7f6fc5400640 1 -- 172.21.15.123:0/50658004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2f05a0 msgr2=0x56109b2c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:35.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.130+0000 7f6fc5400640 1 --2- 172.21.15.123:0/50658004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2f05a0 0x56109b2c53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.130+0000 7f6fc5400640 1 -- 172.21.15.123:0/50658004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56109af392d0 con 0x56109b2f0ce0 2024-04-24T05:46:35.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.130+0000 7f6fce800640 1 --2- 172.21.15.123:0/50658004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2f05a0 0x56109b2c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:35.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.130+0000 7f6fc5400640 1 --2- 172.21.15.123:0/50658004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2f0060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800b8a0 tx=0x7f6fb800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82a0f22c1ea969 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:35.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.130+0000 7f6fbea00640 1 -- 172.21.15.123:0/50658004 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8018030 con 0x56109b2f0ce0 2024-04-24T05:46:35.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.130+0000 7f6fbea00640 1 -- 172.21.15.123:0/50658004 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb8004020 con 0x56109b2f0ce0 2024-04-24T05:46:35.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.130+0000 7f6fbea00640 1 -- 172.21.15.123:0/50658004 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8004410 con 0x56109b2f0ce0 2024-04-24T05:46:35.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.131+0000 7f6fd801f740 1 -- 172.21.15.123:0/50658004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 msgr2=0x56109b2f0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:35.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.131+0000 7f6fd801f740 1 --2- 172.21.15.123:0/50658004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2f0060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800b8a0 tx=0x7f6fb800bd60 comp rx=0 tx=0).stop 2024-04-24T05:46:35.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.131+0000 7f6fd801f740 1 -- 172.21.15.123:0/50658004 shutdown_connections 2024-04-24T05:46:35.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.131+0000 7f6fd801f740 1 --2- 172.21.15.123:0/50658004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2f05a0 0x56109b2c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.131+0000 7f6fd801f740 1 --2- 172.21.15.123:0/50658004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2f0060 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.131+0000 7f6fd801f740 1 --2- 172.21.15.123:0/50658004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b30dbc0 0x56109b2c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.131+0000 7f6fd801f740 1 -- 172.21.15.123:0/50658004 >> 172.21.15.123:0/50658004 conn(0x56109b2c27c0 msgr2=0x56109b30b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:35.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.131+0000 7f6fd801f740 1 -- 172.21.15.123:0/50658004 shutdown_connections 2024-04-24T05:46:35.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.131+0000 7f6fd801f740 1 -- 172.21.15.123:0/50658004 wait complete. 2024-04-24T05:46:35.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.132+0000 7f6fd801f740 1 -- start start 2024-04-24T05:46:35.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.132+0000 7f6fd801f740 1 Processor -- start 2024-04-24T05:46:35.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.133+0000 7f6fd801f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2da8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.133+0000 7f6fd801f740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b30dbc0 0x56109b308c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.133+0000 7f6fd801f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2ec6e0 0x56109b2ecb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.133+0000 7f6fd801f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56109b2c4970 con 0x56109b2f0ce0 2024-04-24T05:46:35.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.133+0000 7f6fd801f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56109b1e0590 con 0x56109b2ec6e0 2024-04-24T05:46:35.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.133+0000 7f6fd801f740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56109b2c3d60 con 0x56109b30dbc0 2024-04-24T05:46:35.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.133+0000 7f6fd801f740 10 client.0 _ll_register_callbacks cb 0x56109b1c33a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T05:46:35.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.133+0000 7f6fcde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2da8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.133+0000 7f6fcde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2da8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50406/0 (socket says 172.21.15.123:50406) 2024-04-24T05:46:35.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.133+0000 7f6fcde00640 1 -- 172.21.15.123:0/1360923491 learned_addr learned my addr 172.21.15.123:0/1360923491 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:35.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.134+0000 7f6fcde00640 1 -- 172.21.15.123:0/1360923491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b30dbc0 msgr2=0x56109b308c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:35.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.134+0000 7f6fcde00640 1 --2- 172.21.15.123:0/1360923491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b30dbc0 0x56109b308c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.134+0000 7f6fcde00640 1 -- 172.21.15.123:0/1360923491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2ec6e0 msgr2=0x56109b2ecb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.134+0000 7f6fcde00640 1 --2- 172.21.15.123:0/1360923491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2ec6e0 0x56109b2ecb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.134+0000 7f6fcde00640 1 -- 172.21.15.123:0/1360923491 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56109b0036b0 con 0x56109b2f0ce0 2024-04-24T05:46:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.134+0000 7f6fcde00640 1 --2- 172.21.15.123:0/1360923491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2da8a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000aa00 tx=0x7f6fc002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf6b2c05ccc02c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.134+0000 7f6fb3400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-24T05:46:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.134+0000 7f6fb3400640 1 -- 172.21.15.123:0/1360923491 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0040070 con 0x56109b2f0ce0 2024-04-24T05:46:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.134+0000 7f6fb3400640 1 -- 172.21.15.123:0/1360923491 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6fc003c960 con 0x56109b2f0ce0 2024-04-24T05:46:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.134+0000 7f6fd801f740 1 -- 172.21.15.123:0/1360923491 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fb8004020 con 0x56109b2f0ce0 2024-04-24T05:46:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.135+0000 7f6fb3400640 1 -- 172.21.15.123:0/1360923491 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f6fc0054050 con 0x56109b2f0ce0 2024-04-24T05:46:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.135+0000 7f6fd801f740 10 client.4818 fetch_fsmap learned FSMap version 9 2024-04-24T05:46:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.135+0000 7f6fd801f740 1 -- 172.21.15.123:0/1360923491 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x56109b309de0 con 0x56109b2f0ce0 2024-04-24T05:46:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.135+0000 7f6fb3400640 1 -- 172.21.15.123:0/1360923491 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb8004020 con 0x56109b2f0ce0 2024-04-24T05:46:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.135+0000 7f6fb3400640 1 -- 172.21.15.123:0/1360923491 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f6fc0057020 con 0x56109b2f0ce0 2024-04-24T05:46:35.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.135+0000 7f6fd801f740 10 client.4818 fetch_fsmap finished waiting for FSMap version 9 2024-04-24T05:46:35.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.135+0000 7f6fd801f740 10 client.4818 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T05:46:35.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.135+0000 7f6fd801f740 20 client.4818 populate_metadata read hostname 'smithi123' 2024-04-24T05:46:35.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.135+0000 7f6fd801f740 1 --2- 172.21.15.123:0/1360923491 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x56109b2c56b0 0x56109b329b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.135+0000 7f6fd801f740 4 client.4818 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T05:46:35.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.135+0000 7f6fd801f740 1 -- 172.21.15.123:0/1360923491 --> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56109b234530 con 0x56109b2c56b0 2024-04-24T05:46:35.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.135+0000 7f6fc5400640 1 --2- 172.21.15.123:0/1360923491 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x56109b2c56b0 0x56109b329b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.136+0000 7f6fb3400640 10 client.4818 ms_handle_connect on v2:172.21.15.123:6834/3370584410 2024-04-24T05:46:35.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.136+0000 7f6fc5400640 1 --2- 172.21.15.123:0/1360923491 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x56109b2c56b0 0x56109b329b40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8be9588cd9c61ad1 server_cookie=a8aa889f8537d6ce in_seq=0 out_seq=0 2024-04-24T05:46:35.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fb3400640 1 -- 172.21.15.123:0/1360923491 <== mds.0 v2:172.21.15.123:6834/3370584410 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56109b234530 con 0x56109b2c56b0 2024-04-24T05:46:35.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fb3400640 10 client.4818 handle_command_reply: tid=0 2024-04-24T05:46:35.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fd801f740 1 client.4818 shutdown 2024-04-24T05:46:35.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fd801f740 1 -- 172.21.15.123:0/1360923491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 msgr2=0x56109b2da8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:35.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fd801f740 1 --2- 172.21.15.123:0/1360923491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2da8a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000aa00 tx=0x7f6fc002d000 comp rx=0 tx=0).stop 2024-04-24T05:46:35.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fd801f740 1 -- 172.21.15.123:0/1360923491 shutdown_connections 2024-04-24T05:46:35.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fd801f740 1 --2- 172.21.15.123:0/1360923491 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x56109b2c56b0 0x56109b329b40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fd801f740 1 --2- 172.21.15.123:0/1360923491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2ec6e0 0x56109b2ecb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fd801f740 1 --2- 172.21.15.123:0/1360923491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b30dbc0 0x56109b308c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fb3400640 0 client.4818 ms_handle_reset on v2:172.21.15.123:6834/3370584410 2024-04-24T05:46:35.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fd801f740 1 --2- 172.21.15.123:0/1360923491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2da8a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fd801f740 1 -- 172.21.15.123:0/1360923491 >> 172.21.15.123:0/1360923491 conn(0x56109b2f05a0 msgr2=0x56109b2bb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:35.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fd801f740 1 -- 172.21.15.123:0/1360923491 shutdown_connections 2024-04-24T05:46:35.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fd801f740 1 -- 172.21.15.123:0/1360923491 wait complete. 2024-04-24T05:46:35.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.137+0000 7f6fd801f740 20 client.4818 trim_cache size 0 max 16384 2024-04-24T05:46:35.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.157+0000 7f6fd801f740 1 Processor -- start 2024-04-24T05:46:35.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.157+0000 7f6fd801f740 1 -- start start 2024-04-24T05:46:35.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.158+0000 7f6fd801f740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b2ec6e0 0x56109b308bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.158+0000 7f6fd801f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2c1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.158+0000 7f6fd801f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b30dbc0 0x56109b2c4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.158+0000 7f6fd801f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01fcda0 con 0x56109b2f0ce0 2024-04-24T05:46:35.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.158+0000 7f6fd801f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd01f4470 con 0x56109b30dbc0 2024-04-24T05:46:35.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.158+0000 7f6fd801f740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd01fc780 con 0x56109b2ec6e0 2024-04-24T05:46:35.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.158+0000 7f6fc5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2c1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.158+0000 7f6fc5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2c1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50412/0 (socket says 172.21.15.123:50412) 2024-04-24T05:46:35.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.158+0000 7f6fc5400640 1 -- 172.21.15.123:0/786435295 learned_addr learned my addr 172.21.15.123:0/786435295 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:35.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.159+0000 7f6fcde00640 1 --2- 172.21.15.123:0/786435295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b2ec6e0 0x56109b308bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.159+0000 7f6fce800640 1 --2- 172.21.15.123:0/786435295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b30dbc0 0x56109b2c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.159+0000 7f6fc5400640 1 -- 172.21.15.123:0/786435295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b2ec6e0 msgr2=0x56109b308bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:35.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.159+0000 7f6fc5400640 1 --2- 172.21.15.123:0/786435295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b2ec6e0 0x56109b308bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.159+0000 7f6fc5400640 1 -- 172.21.15.123:0/786435295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b30dbc0 msgr2=0x56109b2c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.159+0000 7f6fc5400640 1 --2- 172.21.15.123:0/786435295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b30dbc0 0x56109b2c4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.159+0000 7f6fc5400640 1 -- 172.21.15.123:0/786435295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56109b2c4c10 con 0x56109b2f0ce0 2024-04-24T05:46:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.159+0000 7f6fc5400640 1 --2- 172.21.15.123:0/786435295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2c1290 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8016000 tx=0x7f6fb8009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56a1d739dc25b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.159+0000 7f6fbe000640 1 -- 172.21.15.123:0/786435295 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8027330 con 0x56109b2f0ce0 2024-04-24T05:46:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.160+0000 7f6fbe000640 1 -- 172.21.15.123:0/786435295 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb8027510 con 0x56109b2f0ce0 2024-04-24T05:46:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.160+0000 7f6fbe000640 1 -- 172.21.15.123:0/786435295 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8027900 con 0x56109b2f0ce0 2024-04-24T05:46:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.160+0000 7f6fd801f740 1 -- 172.21.15.123:0/786435295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 msgr2=0x56109b2c1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.160+0000 7f6fd801f740 1 --2- 172.21.15.123:0/786435295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2c1290 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8016000 tx=0x7f6fb8009000 comp rx=0 tx=0).stop 2024-04-24T05:46:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.160+0000 7f6fd801f740 1 -- 172.21.15.123:0/786435295 shutdown_connections 2024-04-24T05:46:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.160+0000 7f6fd801f740 1 --2- 172.21.15.123:0/786435295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b30dbc0 0x56109b2c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.160+0000 7f6fd801f740 1 --2- 172.21.15.123:0/786435295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b2c1290 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.160+0000 7f6fd801f740 1 --2- 172.21.15.123:0/786435295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b2ec6e0 0x56109b308bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.160+0000 7f6fd801f740 1 -- 172.21.15.123:0/786435295 >> 172.21.15.123:0/786435295 conn(0x56109b2f05a0 msgr2=0x56109b2c6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:35.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.160+0000 7f6fd801f740 1 -- 172.21.15.123:0/786435295 shutdown_connections 2024-04-24T05:46:35.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.160+0000 7f6fd801f740 1 -- 172.21.15.123:0/786435295 wait complete. 2024-04-24T05:46:35.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.161+0000 7f6fd801f740 1 -- start start 2024-04-24T05:46:35.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.161+0000 7f6fd801f740 1 Processor -- start 2024-04-24T05:46:35.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.162+0000 7f6fd801f740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2ec6e0 0x56109b2bb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.162+0000 7f6fd801f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b3848b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.162+0000 7f6fd801f740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b30dbc0 0x56109b31d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.162+0000 7f6fd801f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56109b30a960 con 0x56109b2f0ce0 2024-04-24T05:46:35.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.162+0000 7f6fd801f740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56109b30a180 con 0x56109b2ec6e0 2024-04-24T05:46:35.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.162+0000 7f6fd801f740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56109b2f22e0 con 0x56109b30dbc0 2024-04-24T05:46:35.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.162+0000 7f6fd801f740 10 client.0 _ll_register_callbacks cb 0x7f6fc0030b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T05:46:35.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.162+0000 7f6fc5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b3848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.163+0000 7f6fc5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b3848b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50426/0 (socket says 172.21.15.123:50426) 2024-04-24T05:46:35.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.163+0000 7f6fc5400640 1 -- 172.21.15.123:0/4145478977 learned_addr learned my addr 172.21.15.123:0/4145478977 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:35.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.163+0000 7f6fce800640 1 --2- 172.21.15.123:0/4145478977 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b30dbc0 0x56109b31d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.163+0000 7f6fcde00640 1 --2- 172.21.15.123:0/4145478977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2ec6e0 0x56109b2bb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.163+0000 7f6fc5400640 1 -- 172.21.15.123:0/4145478977 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b30dbc0 msgr2=0x56109b31d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:35.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.163+0000 7f6fc5400640 1 --2- 172.21.15.123:0/4145478977 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b30dbc0 0x56109b31d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.163+0000 7f6fc5400640 1 -- 172.21.15.123:0/4145478977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2ec6e0 msgr2=0x56109b2bb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:35.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.163+0000 7f6fc5400640 1 --2- 172.21.15.123:0/4145478977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2ec6e0 0x56109b2bb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.163+0000 7f6fc5400640 1 -- 172.21.15.123:0/4145478977 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56109b31e530 con 0x56109b2f0ce0 2024-04-24T05:46:35.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.163+0000 7f6fcde00640 1 --2- 172.21.15.123:0/4145478977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2ec6e0 0x56109b2bb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:35.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.163+0000 7f6fc5400640 1 --2- 172.21.15.123:0/4145478977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b3848b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800b8a0 tx=0x7f6fb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b711900de7cd9705 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:35.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.163+0000 7f6fb3400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-24T05:46:35.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.164+0000 7f6fb3400640 1 -- 172.21.15.123:0/4145478977 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8018040 con 0x56109b2f0ce0 2024-04-24T05:46:35.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.164+0000 7f6fb3400640 1 -- 172.21.15.123:0/4145478977 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6fb80278c0 con 0x56109b2f0ce0 2024-04-24T05:46:35.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.164+0000 7f6fd801f740 1 -- 172.21.15.123:0/4145478977 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fb8027510 con 0x56109b2f0ce0 2024-04-24T05:46:35.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.164+0000 7f6fb3400640 1 -- 172.21.15.123:0/4145478977 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f6fb803c050 con 0x56109b2f0ce0 2024-04-24T05:46:35.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.164+0000 7f6fd801f740 10 client.4824 fetch_fsmap learned FSMap version 9 2024-04-24T05:46:35.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.164+0000 7f6fd801f740 1 -- 172.21.15.123:0/4145478977 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x56109b385290 con 0x56109b2f0ce0 2024-04-24T05:46:35.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.164+0000 7f6fb3400640 1 -- 172.21.15.123:0/4145478977 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb8027510 con 0x56109b2f0ce0 2024-04-24T05:46:35.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.164+0000 7f6fb3400640 1 -- 172.21.15.123:0/4145478977 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f6fb803f020 con 0x56109b2f0ce0 2024-04-24T05:46:35.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.164+0000 7f6fd801f740 10 client.4824 fetch_fsmap finished waiting for FSMap version 9 2024-04-24T05:46:35.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.164+0000 7f6fd801f740 10 client.4824 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T05:46:35.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.164+0000 7f6fd801f740 20 client.4824 populate_metadata read hostname 'smithi123' 2024-04-24T05:46:35.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.164+0000 7f6fd801f740 1 --2- 172.21.15.123:0/4145478977 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x56109b31dd30 0x56109b2ee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.164+0000 7f6fd801f740 4 client.4824 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T05:46:35.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.164+0000 7f6fd801f740 1 -- 172.21.15.123:0/4145478977 --> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] -- command(tid 0: {"prefix": "status"}) -- 0x56109b22e3a0 con 0x56109b31dd30 2024-04-24T05:46:35.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.165+0000 7f6fcde00640 1 --2- 172.21.15.123:0/4145478977 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x56109b31dd30 0x56109b2ee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.165+0000 7f6fcde00640 1 --2- 172.21.15.123:0/4145478977 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x56109b31dd30 0x56109b2ee4c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25bda29ef30908f9 server_cookie=c8ae1a9f19f66d4c in_seq=0 out_seq=0 2024-04-24T05:46:35.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.165+0000 7f6fb3400640 10 client.4824 ms_handle_connect on v2:172.21.15.123:6834/3370584410 2024-04-24T05:46:35.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.165+0000 7f6fb3400640 1 -- 172.21.15.123:0/4145478977 <== mds.0 v2:172.21.15.123:6834/3370584410 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x56109b22e3a0 con 0x56109b31dd30 2024-04-24T05:46:35.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.165+0000 7f6fb3400640 10 client.4824 handle_command_reply: tid=0 2024-04-24T05:46:35.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.165+0000 7f6fd801f740 1 client.4824 shutdown 2024-04-24T05:46:35.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.166+0000 7f6fd801f740 1 -- 172.21.15.123:0/4145478977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 msgr2=0x56109b3848b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:35.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.166+0000 7f6fd801f740 1 --2- 172.21.15.123:0/4145478977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b3848b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800b8a0 tx=0x7f6fb8004730 comp rx=0 tx=0).stop 2024-04-24T05:46:35.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.166+0000 7f6fd801f740 1 -- 172.21.15.123:0/4145478977 shutdown_connections 2024-04-24T05:46:35.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.166+0000 7f6fd801f740 1 --2- 172.21.15.123:0/4145478977 >> [v2:172.21.15.123:6834/3370584410,v1:172.21.15.123:6837/3370584410] conn(0x56109b31dd30 0x56109b2ee4c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.166+0000 7f6fd801f740 1 --2- 172.21.15.123:0/4145478977 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56109b30dbc0 0x56109b31d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.166+0000 7f6fd801f740 1 --2- 172.21.15.123:0/4145478977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109b2f0ce0 0x56109b3848b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.166+0000 7f6fd801f740 1 --2- 172.21.15.123:0/4145478977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56109b2ec6e0 0x56109b2bb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.166+0000 7f6fb3400640 0 client.4824 ms_handle_reset on v2:172.21.15.123:6834/3370584410 2024-04-24T05:46:35.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.166+0000 7f6fd801f740 1 -- 172.21.15.123:0/4145478977 >> 172.21.15.123:0/4145478977 conn(0x56109b2f05a0 msgr2=0x56109b2c0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:35.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.166+0000 7f6fd801f740 1 -- 172.21.15.123:0/4145478977 shutdown_connections 2024-04-24T05:46:35.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.166+0000 7f6fd801f740 1 -- 172.21.15.123:0/4145478977 wait complete. 2024-04-24T05:46:35.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.166+0000 7f6fd801f740 20 client.4824 trim_cache size 0 max 16384 2024-04-24T05:46:35.172 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-24T05:46:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "6efb8524-c0d1-4076-b7c9-7485084c7ada", 2024-04-24T05:46:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2024-04-24T05:46:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4273, 2024-04-24T05:46:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-24T05:46:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-24T05:46:35.173 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-24T05:46:35.173 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 34.966002201999999, 2024-04-24T05:46:35.173 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 9, 2024-04-24T05:46:35.173 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 25, 2024-04-24T05:46:35.173 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 25, 2024-04-24T05:46:35.173 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 40.471394963999998 2024-04-24T05:46:35.173 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-24T05:46:35.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.167+0000 7f6fd5200640 1 -- 172.21.15.123:0/36837508 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6fac06c920 msgr2=0x7f6fac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:35.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.167+0000 7f6fd5200640 1 --2- 172.21.15.123:0/36837508 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6fac06c920 0x7f6fac06ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0035000 tx=0x7f6fc000c040 comp rx=0 tx=0).stop 2024-04-24T05:46:35.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.167+0000 7f6fd5200640 1 -- 172.21.15.123:0/36837508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd015f1c0 msgr2=0x7f6fd01ed950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:35.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.167+0000 7f6fd5200640 1 --2- 172.21.15.123:0/36837508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd015f1c0 0x7f6fd01ed950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800caa0 tx=0x7f6fc800cf60 comp rx=0 tx=0).stop 2024-04-24T05:46:35.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.167+0000 7f6fd5200640 1 -- 172.21.15.123:0/36837508 shutdown_connections 2024-04-24T05:46:35.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.167+0000 7f6fd5200640 1 --2- 172.21.15.123:0/36837508 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6fac06c920 0x7f6fac06ede0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.167+0000 7f6fd5200640 1 --2- 172.21.15.123:0/36837508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd015f1c0 0x7f6fd01ed950 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.167+0000 7f6fd5200640 1 --2- 172.21.15.123:0/36837508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6fd015ab10 0x7f6fd01f0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.167+0000 7f6fd5200640 1 --2- 172.21.15.123:0/36837508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6fd0072640 0x7f6fd015ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.167+0000 7f6fd5200640 1 -- 172.21.15.123:0/36837508 >> 172.21.15.123:0/36837508 conn(0x7f6fd00b1e70 msgr2=0x7f6fd00af850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:35.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.167+0000 7f6fd5200640 1 -- 172.21.15.123:0/36837508 shutdown_connections 2024-04-24T05:46:35.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.167+0000 7f6fd5200640 1 -- 172.21.15.123:0/36837508 wait complete. 2024-04-24T05:46:35.185 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T05:46:35.195 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T05:46:35.195 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T05:46:35.196 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T05:46:35.196 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-24T05:46:35.208 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-24T05:46:35.208 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-24T05:46:35.209 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-24T05:46:35.209 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T05:46:35.209 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T05:46:35.209 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-24T05:46:35.209 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T05:46:35.209 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-24T05:46:35.209 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-24T05:46:35.209 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T05:46:35.209 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T05:46:35.209 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-24T05:46:35.209 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T05:46:35.210 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-24T05:46:35.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:35.210 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-24T05:46:35.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:35.270 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link delete ceph-brx 2024-04-24T05:46:35.363 INFO:teuthology.orchestra.run.smithi155.stderr:Cannot find device "ceph-brx" 2024-04-24T05:46:35.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T05:46:35.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:35.368 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-24T05:46:35.424 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-24T05:46:35.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:35.426 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T05:46:35.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:35.519 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-04-24T05:46:35.637 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-24T05:46:35.638 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T05:46:35.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.816+0000 7f8344600640 1 Processor -- start 2024-04-24T05:46:35.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.816+0000 7f8344600640 1 -- start start 2024-04-24T05:46:35.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.818+0000 7f8344600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f833c15e970 0x7f833c0d44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.818+0000 7f8344600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f833c0b39b0 0x7f833c0d4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.818+0000 7f8344600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f833c0cfdf0 con 0x7f833c0d3e10 2024-04-24T05:46:35.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.818+0000 7f8344600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f833c0d09c0 con 0x7f833c15e970 2024-04-24T05:46:35.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.818+0000 7f8344600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f833c0d1e60 con 0x7f833c0b39b0 2024-04-24T05:46:35.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.819+0000 7f8342600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f833c0b39b0 0x7f833c0d4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.819+0000 7f8342600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f833c0b39b0 0x7f833c0d4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:34580/0 (socket says 172.21.15.123:34580) 2024-04-24T05:46:35.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.819+0000 7f8342600640 1 -- 172.21.15.123:0/1060376376 learned_addr learned my addr 172.21.15.123:0/1060376376 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:35.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.819+0000 7f8341c00640 1 --2- 172.21.15.123:0/1060376376 >> v2:172.21.15.155:3300/0 conn(0x7f833c15e970 0x7f833c0d44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.819+0000 7f8342600640 1 -- 172.21.15.123:0/1060376376 >> v2:172.21.15.155:3300/0 conn(0x7f833c15e970 msgr2=0x7f833c0d44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:35.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.819+0000 7f8342600640 1 --2- 172.21.15.123:0/1060376376 >> v2:172.21.15.155:3300/0 conn(0x7f833c15e970 0x7f833c0d44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.820+0000 7f8342600640 1 -- 172.21.15.123:0/1060376376 >> v1:172.21.15.155:6789/0 conn(0x7f833c0d3e10 legacy=0x7f833c0d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:35.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.820+0000 7f8342600640 1 -- 172.21.15.123:0/1060376376 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833c0d0090 con 0x7f833c0b39b0 2024-04-24T05:46:35.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.821+0000 7f8342600640 1 --2- 172.21.15.123:0/1060376376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f833c0b39b0 0x7f833c0d4a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8334009870 tx=0x7f833402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d741d9ecec5084 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:35.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.822+0000 7f833be00640 1 -- 172.21.15.123:0/1060376376 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334037070 con 0x7f833c0b39b0 2024-04-24T05:46:35.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.822+0000 7f833be00640 1 -- 172.21.15.123:0/1060376376 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8334032440 con 0x7f833c0b39b0 2024-04-24T05:46:35.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.822+0000 7f833be00640 1 -- 172.21.15.123:0/1060376376 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334032810 con 0x7f833c0b39b0 2024-04-24T05:46:35.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.822+0000 7f8344600640 1 -- 172.21.15.123:0/1060376376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f833c0b39b0 msgr2=0x7f833c0d4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:35.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.822+0000 7f8344600640 1 --2- 172.21.15.123:0/1060376376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f833c0b39b0 0x7f833c0d4a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8334009870 tx=0x7f833402a1c0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.823+0000 7f8344600640 1 -- 172.21.15.123:0/1060376376 shutdown_connections 2024-04-24T05:46:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.823+0000 7f8344600640 1 --2- 172.21.15.123:0/1060376376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f833c0b39b0 0x7f833c0d4a30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.823+0000 7f8344600640 1 --2- 172.21.15.123:0/1060376376 >> v2:172.21.15.155:3300/0 conn(0x7f833c15e970 0x7f833c0d44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.823+0000 7f8344600640 1 -- 172.21.15.123:0/1060376376 >> 172.21.15.123:0/1060376376 conn(0x7f833c0b9a40 msgr2=0x7f833c0b75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.823+0000 7f8344600640 1 -- 172.21.15.123:0/1060376376 shutdown_connections 2024-04-24T05:46:35.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.823+0000 7f8344600640 1 -- 172.21.15.123:0/1060376376 wait complete. 2024-04-24T05:46:35.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.824+0000 7f8344600640 1 Processor -- start 2024-04-24T05:46:35.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.824+0000 7f8344600640 1 -- start start 2024-04-24T05:46:35.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.825+0000 7f8344600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f833c0b39b0 0x7f833c0cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.825+0000 7f8344600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f833c0d3e10 0x7f833c157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.825+0000 7f8344600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c15e970 0x7f833c0b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.825+0000 7f8344600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f833c15ed50 con 0x7f833c15e970 2024-04-24T05:46:35.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.825+0000 7f8344600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f833c0d0650 con 0x7f833c0d3e10 2024-04-24T05:46:35.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.825+0000 7f8344600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f833c0b12b0 con 0x7f833c0b39b0 2024-04-24T05:46:35.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.826+0000 7f8342600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c15e970 0x7f833c0b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.826+0000 7f8342600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c15e970 0x7f833c0b66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50442/0 (socket says 172.21.15.123:50442) 2024-04-24T05:46:35.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.826+0000 7f8342600640 1 -- 172.21.15.123:0/2147282414 learned_addr learned my addr 172.21.15.123:0/2147282414 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:35.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.826+0000 7f8341c00640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f833c0b39b0 0x7f833c0cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.826+0000 7f8341200640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f833c0d3e10 0x7f833c157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.826+0000 7f8342600640 1 -- 172.21.15.123:0/2147282414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f833c0b39b0 msgr2=0x7f833c0cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:35.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.826+0000 7f8342600640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f833c0b39b0 0x7f833c0cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.826+0000 7f8342600640 1 -- 172.21.15.123:0/2147282414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f833c0d3e10 msgr2=0x7f833c157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:35.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.826+0000 7f8342600640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f833c0d3e10 0x7f833c157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:35.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.826+0000 7f8342600640 1 -- 172.21.15.123:0/2147282414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833c0b5640 con 0x7f833c15e970 2024-04-24T05:46:35.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.826+0000 7f8341200640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f833c0d3e10 0x7f833c157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:46:35.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.827+0000 7f8342600640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c15e970 0x7f833c0b66e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f833402a200 tx=0x7f83340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:35.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.827+0000 7f8339600640 1 -- 172.21.15.123:0/2147282414 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334041020 con 0x7f833c15e970 2024-04-24T05:46:35.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.827+0000 7f8339600640 1 -- 172.21.15.123:0/2147282414 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833403d640 con 0x7f833c15e970 2024-04-24T05:46:35.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.827+0000 7f8339600640 1 -- 172.21.15.123:0/2147282414 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334037040 con 0x7f833c15e970 2024-04-24T05:46:35.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.827+0000 7f8344600640 1 -- 172.21.15.123:0/2147282414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833c0b5920 con 0x7f833c15e970 2024-04-24T05:46:35.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.827+0000 7f8344600640 1 -- 172.21.15.123:0/2147282414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f833c0b5ee0 con 0x7f833c15e970 2024-04-24T05:46:35.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.828+0000 7f8344600640 1 -- 172.21.15.123:0/2147282414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f833c068a20 con 0x7f833c15e970 2024-04-24T05:46:35.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.829+0000 7f8339600640 1 -- 172.21.15.123:0/2147282414 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8334044020 con 0x7f833c15e970 2024-04-24T05:46:35.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.830+0000 7f8339600640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f831c06c920 0x7f831c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:35.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.830+0000 7f8339600640 1 -- 172.21.15.123:0/2147282414 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f83340ac4e0 con 0x7f833c15e970 2024-04-24T05:46:35.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.831+0000 7f8341c00640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f831c06c920 0x7f831c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:35.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.831+0000 7f8341c00640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f831c06c920 0x7f831c06ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f833c1f7860 tx=0x7f833000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:35.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:35.836+0000 7f8339600640 1 -- 172.21.15.123:0/2147282414 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f833404b030 con 0x7f833c15e970 2024-04-24T05:46:36.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36.081+0000 7f8344600640 1 -- 172.21.15.123:0/2147282414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f833c0b18b0 con 0x7f833c15e970 2024-04-24T05:46:36.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36.082+0000 7f8339600640 1 -- 172.21.15.123:0/2147282414 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f833407a5e0 con 0x7f833c15e970 2024-04-24T05:46:36.083 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T05:46:36.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36.084+0000 7f8344600640 1 -- 172.21.15.123:0/2147282414 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f831c06c920 msgr2=0x7f831c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:36.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36.084+0000 7f8344600640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f831c06c920 0x7f831c06ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f833c1f7860 tx=0x7f833000b040 comp rx=0 tx=0).stop 2024-04-24T05:46:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36.085+0000 7f8344600640 1 -- 172.21.15.123:0/2147282414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c15e970 msgr2=0x7f833c0b66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36.085+0000 7f8344600640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c15e970 0x7f833c0b66e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f833402a200 tx=0x7f83340046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36.085+0000 7f8344600640 1 -- 172.21.15.123:0/2147282414 shutdown_connections 2024-04-24T05:46:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36.085+0000 7f8344600640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f831c06c920 0x7f831c06ede0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36.085+0000 7f8344600640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c15e970 0x7f833c0b66e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36.085+0000 7f8344600640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f833c0d3e10 0x7f833c157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36.085+0000 7f8344600640 1 --2- 172.21.15.123:0/2147282414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f833c0b39b0 0x7f833c0cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36.085+0000 7f8344600640 1 -- 172.21.15.123:0/2147282414 >> 172.21.15.123:0/2147282414 conn(0x7f833c0b9a40 msgr2=0x7f833c0d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36.085+0000 7f8344600640 1 -- 172.21.15.123:0/2147282414 shutdown_connections 2024-04-24T05:46:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36.085+0000 7f8344600640 1 -- 172.21.15.123:0/2147282414 wait complete. 2024-04-24T05:46:36.106 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T05:46:36.106 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T05:46:36.106 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-04-24T05:46:36.106 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T05:46:36.106 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T05:46:36.106 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T05:46:36.106 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T05:46:36.106 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T05:46:36.106 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T05:46:36.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:36.107 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-04-24T05:46:36.155 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T05:46:36.155 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T05:46:36.155 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T05:46:36.155 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:36.155 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-24T05:46:36.155 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:36.155 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:36.155 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:36.155 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-04-24T05:46:36.156 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:36.156 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:36.156 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-04-24T05:46:36.156 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:36.156 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:36.156 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T05:46:36.156 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:36.156 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T05:46:36.156 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42591sec preferred_lft 42591sec 2024-04-24T05:46:36.156 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-04-24T05:46:36.156 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:36.157 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T05:46:36.157 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:36.157 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:46:36.158 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2024-04-24T05:46:36.158 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2024-04-24T05:46:36.158 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2024-04-24T05:46:36.158 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T05:46:36.158 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:46:36.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:36.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:36.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:36.392 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T05:46:36.458 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-04-24T05:46:36.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:36.463 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-04-24T05:46:36.511 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-04-24T05:46:36.511 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T05:46:36.511 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T05:46:36.511 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T05:46:36.512 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T05:46:36.512 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:36.512 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:46:36.512 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T05:46:36.513 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T05:46:36.513 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T05:46:36.513 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:46:36.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:36.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:36.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:36.719 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-24T05:46:36.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:36.768 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-04-24T05:46:36.820 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:36.820 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:46:36.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:46:36.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T05:46:36.820 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:46:36.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:36.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:37.002 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T05:46:37.002 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:37.002 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:46:37.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T05:46:37.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T05:46:37.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T05:46:37.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T05:46:37.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T05:46:37.003 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:46:37.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:37.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:37.267 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:37.267 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:46:37.267 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-04-24T05:46:37.267 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T05:46:37.267 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:46:37.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:37.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:37.463 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T05:46:37.463 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T05:46:37.463 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T05:46:37.507 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T05:46:37.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T05:46:37.508 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T05:46:37.601 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-24T05:46:37.685 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T05:46:37.741 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-24T05:46:37.741 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-24T05:46:37.741 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-24T05:46:37.741 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-24T05:46:37.741 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-24T05:46:37.741 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-24T05:46:37.741 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-24T05:46:37.741 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-24T05:46:37.741 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-24T05:46:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-24T05:46:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-24T05:46:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-24T05:46:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T05:46:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-24T05:46:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-24T05:46:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-24T05:46:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-24T05:46:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-24T05:46:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-24T05:46:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T05:46:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T05:46:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T05:46:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T05:46:37.743 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-24T05:46:37.743 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-24T05:46:37.743 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T05:46:37.743 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T05:46:37.743 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T05:46:37.743 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T05:46:37.743 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-24T05:46:37.743 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:46:37.743 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:46:37.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:37.744 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-24T05:46:37.793 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T05:46:37.793 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T05:46:37.799 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-24T05:46:37.867 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T05:46:37.906 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-24T05:46:37.906 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-24T05:46:37.906 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-24T05:46:37.906 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-24T05:46:37.906 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-24T05:46:37.906 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-24T05:46:37.906 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-24T05:46:37.906 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-24T05:46:37.906 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T05:46:37.907 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-24T05:46:37.908 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-24T05:46:37.908 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T05:46:37.908 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T05:46:37.908 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T05:46:37.908 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T05:46:37.908 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-24T05:46:37.908 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:46:37.908 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:46:37.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:37.909 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-24T05:46:37.963 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:2024-04-24T05:46:37.962+0000 7f0066eea480 -1 init, newargv = 0x558fd39085c0 newargc=15 2024-04-24T05:46:37.964 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:ceph-fuse[43835]: starting ceph client 2024-04-24T05:46:37.973 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:ceph-fuse[43835]: starting fuse 2024-04-24T05:46:37.992 INFO:teuthology.orchestra.run.smithi123.stdout:41 2024-04-24T05:46:37.992 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-24T05:46:37.993 DEBUG:teuthology.orchestra.run.smithi123:> sudo stdin-killer -- python3 -c ' 2024-04-24T05:46:37.993 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2024-04-24T05:46:37.993 DEBUG:teuthology.orchestra.run.smithi123:> import re 2024-04-24T05:46:37.993 DEBUG:teuthology.orchestra.run.smithi123:> import os 2024-04-24T05:46:37.993 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2024-04-24T05:46:37.993 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:46:37.993 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2024-04-24T05:46:37.993 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T05:46:37.993 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2024-04-24T05:46:37.993 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T05:46:37.993 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:46:37.993 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2024-04-24T05:46:37.993 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2024-04-24T05:46:37.993 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2024-04-24T05:46:37.994 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2024-04-24T05:46:37.994 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:46:37.994 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2024-04-24T05:46:37.994 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T05:46:37.994 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T05:46:37.994 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T05:46:37.994 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2024-04-24T05:46:37.994 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2024-04-24T05:46:37.994 DEBUG:teuthology.orchestra.run.smithi123:> return f 2024-04-24T05:46:37.994 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T05:46:37.994 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:46:37.994 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.0")) 2024-04-24T05:46:37.994 DEBUG:teuthology.orchestra.run.smithi123:> ' 2024-04-24T05:46:38.183 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.0.43835.asok 2024-04-24T05:46:38.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:38.200 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43835.asok 2024-04-24T05:46:38.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:38.200 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43835.asok status 2024-04-24T05:46:38.429 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-24T05:46:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2024-04-24T05:46:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T05:46:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T05:46:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "0", 2024-04-24T05:46:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2024-04-24T05:46:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T05:46:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "43835", 2024-04-24T05:46:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2024-04-24T05:46:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:46:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2024-04-24T05:46:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2024-04-24T05:46:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4833, 2024-04-24T05:46:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2024-04-24T05:46:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2024-04-24T05:46:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2024-04-24T05:46:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 4833 2024-04-24T05:46:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:46:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-24T05:46:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-24T05:46:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-04-24T05:46:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 56639471 2024-04-24T05:46:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:46:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:46:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-24T05:46:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-24T05:46:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-04-24T05:46:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 56639471 2024-04-24T05:46:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:46:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.4833 192.168.0.1:0/56639471", 2024-04-24T05:46:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.1:0/56639471", 2024-04-24T05:46:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2024-04-24T05:46:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 9, 2024-04-24T05:46:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 25, 2024-04-24T05:46:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2024-04-24T05:46:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2024-04-24T05:46:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2024-04-24T05:46:38.431 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-24T05:46:38.443 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T05:46:38.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.644+0000 7fa0ae800640 1 Processor -- start 2024-04-24T05:46:38.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.644+0000 7fa0ae800640 1 -- start start 2024-04-24T05:46:38.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.646+0000 7fa0ae800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa0a8073550 0x7fa0a815ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:38.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.646+0000 7fa0ae800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a815f660 con 0x7fa0a806ed20 2024-04-24T05:46:38.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.646+0000 7fa0ae800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a815f940 con 0x7fa0a806d2a0 2024-04-24T05:46:38.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.646+0000 7fa0ae800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa0a815fc20 con 0x7fa0a8073550 2024-04-24T05:46:38.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.646+0000 7fa0a7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa0a806ed20 0x7fa0a806f190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49346/0 (socket says 172.21.15.123:49346) 2024-04-24T05:46:38.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.646+0000 7fa0a7e00640 1 -- 172.21.15.123:0/2395759222 learned_addr learned my addr 172.21.15.123:0/2395759222 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:38.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa09fe00640 1 --2- 172.21.15.123:0/2395759222 >> v2:172.21.15.155:3300/0 conn(0x7fa0a8073550 0x7fa0a815ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:38.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 -- 172.21.15.123:0/2395759222 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3534675284 0 0) 0x7fa0a815f660 con 0x7fa0a806ed20 2024-04-24T05:46:38.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 -- 172.21.15.123:0/2395759222 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa08c003520 con 0x7fa0a806ed20 2024-04-24T05:46:38.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 -- 172.21.15.123:0/2395759222 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2151878185 0 0) 0x7fa08c003520 con 0x7fa0a806ed20 2024-04-24T05:46:38.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 -- 172.21.15.123:0/2395759222 >> v2:172.21.15.155:3300/0 conn(0x7fa0a8073550 msgr2=0x7fa0a815ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:38.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 --2- 172.21.15.123:0/2395759222 >> v2:172.21.15.155:3300/0 conn(0x7fa0a8073550 0x7fa0a815ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:38.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 -- 172.21.15.123:0/2395759222 >> v1:172.21.15.155:6789/0 conn(0x7fa0a806d2a0 legacy=0x7fa0a806d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:38.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 -- 172.21.15.123:0/2395759222 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a806e520 con 0x7fa0a806ed20 2024-04-24T05:46:38.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 -- 172.21.15.123:0/2395759222 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7fa0a00019f0 con 0x7fa0a806ed20 2024-04-24T05:46:38.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 -- 172.21.15.123:0/2395759222 >> v1:172.21.15.123:6789/0 conn(0x7fa0a806ed20 legacy=0x7fa0a806f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 --2- 172.21.15.123:0/2395759222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa08c0039e0 0x7fa08c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 --2- 172.21.15.123:0/2395759222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa08c004800 0x7fa08c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 --2- 172.21.15.123:0/2395759222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c0051c0 0x7fa08c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 -- 172.21.15.123:0/2395759222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a815fc20 con 0x7fa08c0051c0 2024-04-24T05:46:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 -- 172.21.15.123:0/2395759222 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c00c070 con 0x7fa08c0039e0 2024-04-24T05:46:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.647+0000 7fa0a6a00640 1 -- 172.21.15.123:0/2395759222 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c00c270 con 0x7fa08c004800 2024-04-24T05:46:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.648+0000 7fa0a7e00640 1 --2- 172.21.15.123:0/2395759222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c0051c0 0x7fa08c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.648+0000 7fa09fe00640 1 --2- 172.21.15.123:0/2395759222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa08c004800 0x7fa08c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.648+0000 7fa0a7400640 1 --2- 172.21.15.123:0/2395759222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa08c0039e0 0x7fa08c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.648+0000 7fa0a7e00640 1 -- 172.21.15.123:0/2395759222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa08c004800 msgr2=0x7fa08c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.648+0000 7fa0a7e00640 1 --2- 172.21.15.123:0/2395759222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa08c004800 0x7fa08c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.648+0000 7fa0a7e00640 1 -- 172.21.15.123:0/2395759222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa08c0039e0 msgr2=0x7fa08c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.648+0000 7fa0a7e00640 1 --2- 172.21.15.123:0/2395759222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa08c0039e0 0x7fa08c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:38.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.648+0000 7fa0a7e00640 1 -- 172.21.15.123:0/2395759222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c003520 con 0x7fa08c0051c0 2024-04-24T05:46:38.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.649+0000 7fa0a7e00640 1 -- 172.21.15.123:0/2395759222 reap_dead start 2024-04-24T05:46:38.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.649+0000 7fa0a7e00640 1 --2- 172.21.15.123:0/2395759222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c0051c0 0x7fa08c00b9c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000a850 tx=0x7fa08c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9289edda7097f284 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:38.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.649+0000 7fa0a6a00640 1 -- 172.21.15.123:0/2395759222 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000a4a0 con 0x7fa08c0051c0 2024-04-24T05:46:38.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.650+0000 7fa0a6a00640 1 -- 172.21.15.123:0/2395759222 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a002f070 con 0x7fa08c0051c0 2024-04-24T05:46:38.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.650+0000 7fa0a6a00640 1 -- 172.21.15.123:0/2395759222 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a002bce0 con 0x7fa08c0051c0 2024-04-24T05:46:38.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.650+0000 7fa0ae800640 1 -- 172.21.15.123:0/2395759222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c0051c0 msgr2=0x7fa08c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:38.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.650+0000 7fa0ae800640 1 --2- 172.21.15.123:0/2395759222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c0051c0 0x7fa08c00b9c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000a850 tx=0x7fa08c005830 comp rx=0 tx=0).stop 2024-04-24T05:46:38.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.650+0000 7fa0ae800640 1 -- 172.21.15.123:0/2395759222 shutdown_connections 2024-04-24T05:46:38.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.650+0000 7fa0ae800640 1 --2- 172.21.15.123:0/2395759222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c0051c0 0x7fa08c00b9c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:38.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.650+0000 7fa0ae800640 1 -- 172.21.15.123:0/2395759222 >> 172.21.15.123:0/2395759222 conn(0x7fa0a8072550 msgr2=0x7fa0a8072960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:38.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.650+0000 7fa0ae800640 1 -- 172.21.15.123:0/2395759222 shutdown_connections 2024-04-24T05:46:38.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.650+0000 7fa0ae800640 1 -- 172.21.15.123:0/2395759222 wait complete. 2024-04-24T05:46:38.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.652+0000 7fa0ae800640 1 Processor -- start 2024-04-24T05:46:38.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.652+0000 7fa0ae800640 1 -- start start 2024-04-24T05:46:38.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.653+0000 7fa0ae800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c0051c0 0x7fa0a81502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:38.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.653+0000 7fa0ae800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0a814ee00 0x7fa0a814d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:38.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.653+0000 7fa0ae800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0a8150b20 0x7fa0a8150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:38.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.653+0000 7fa0ae800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a81591b0 con 0x7fa08c0051c0 2024-04-24T05:46:38.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.653+0000 7fa0ae800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a8159030 con 0x7fa0a8150b20 2024-04-24T05:46:38.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.653+0000 7fa0ae800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a806d680 con 0x7fa0a814ee00 2024-04-24T05:46:38.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.653+0000 7fa0a7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c0051c0 0x7fa0a81502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:38.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.653+0000 7fa0a7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c0051c0 0x7fa0a81502e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50466/0 (socket says 172.21.15.123:50466) 2024-04-24T05:46:38.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.653+0000 7fa0a7400640 1 -- 172.21.15.123:0/3380603485 learned_addr learned my addr 172.21.15.123:0/3380603485 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:38.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.654+0000 7fa09fe00640 1 --2- 172.21.15.123:0/3380603485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0a814ee00 0x7fa0a814d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:38.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.654+0000 7fa0a7e00640 1 --2- 172.21.15.123:0/3380603485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0a8150b20 0x7fa0a8150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:38.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.654+0000 7fa0a7400640 1 -- 172.21.15.123:0/3380603485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0a814ee00 msgr2=0x7fa0a814d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:38.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.654+0000 7fa0a7400640 1 --2- 172.21.15.123:0/3380603485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0a814ee00 0x7fa0a814d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:38.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.654+0000 7fa0a7400640 1 -- 172.21.15.123:0/3380603485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0a8150b20 msgr2=0x7fa0a8150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:38.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.654+0000 7fa0a7400640 1 --2- 172.21.15.123:0/3380603485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0a8150b20 0x7fa0a8150fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:38.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.654+0000 7fa0a7400640 1 -- 172.21.15.123:0/3380603485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a80ad8f0 con 0x7fa08c0051c0 2024-04-24T05:46:38.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.654+0000 7fa0a7400640 1 --2- 172.21.15.123:0/3380603485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c0051c0 0x7fa0a81502e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa09800e4b0 tx=0x7fa09800e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:38.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.654+0000 7fa09f400640 1 -- 172.21.15.123:0/3380603485 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0980052e0 con 0x7fa08c0051c0 2024-04-24T05:46:38.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.654+0000 7fa09f400640 1 -- 172.21.15.123:0/3380603485 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa098018070 con 0x7fa08c0051c0 2024-04-24T05:46:38.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.654+0000 7fa09f400640 1 -- 172.21.15.123:0/3380603485 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098003830 con 0x7fa08c0051c0 2024-04-24T05:46:38.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.655+0000 7fa0ae800640 1 -- 172.21.15.123:0/3380603485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a814f330 con 0x7fa08c0051c0 2024-04-24T05:46:38.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.655+0000 7fa0ae800640 1 -- 172.21.15.123:0/3380603485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a814f8f0 con 0x7fa08c0051c0 2024-04-24T05:46:38.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.656+0000 7fa0ae800640 1 -- 172.21.15.123:0/3380603485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a804fe80 con 0x7fa08c0051c0 2024-04-24T05:46:38.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.657+0000 7fa09f400640 1 -- 172.21.15.123:0/3380603485 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0980039d0 con 0x7fa08c0051c0 2024-04-24T05:46:38.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.658+0000 7fa09f400640 1 --2- 172.21.15.123:0/3380603485 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa08406c8f0 0x7fa08406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:38.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.658+0000 7fa09f400640 1 -- 172.21.15.123:0/3380603485 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa09808bae0 con 0x7fa08c0051c0 2024-04-24T05:46:38.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.658+0000 7fa09fe00640 1 --2- 172.21.15.123:0/3380603485 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa08406c8f0 0x7fa08406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:38.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.659+0000 7fa09fe00640 1 --2- 172.21.15.123:0/3380603485 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa08406c8f0 0x7fa08406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8154c80 tx=0x7fa090008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:38.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.662+0000 7fa09f400640 1 -- 172.21.15.123:0/3380603485 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa0a804fe80 con 0x7fa08c0051c0 2024-04-24T05:46:38.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.903+0000 7fa0ae800640 1 -- 172.21.15.123:0/3380603485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa0a8068a20 con 0x7fa08c0051c0 2024-04-24T05:46:38.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.904+0000 7fa09f400640 1 -- 172.21.15.123:0/3380603485 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fa098059bc0 con 0x7fa08c0051c0 2024-04-24T05:46:38.906 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T05:46:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.907+0000 7fa0ae800640 1 -- 172.21.15.123:0/3380603485 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa08406c8f0 msgr2=0x7fa08406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.907+0000 7fa0ae800640 1 --2- 172.21.15.123:0/3380603485 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa08406c8f0 0x7fa08406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8154c80 tx=0x7fa090008040 comp rx=0 tx=0).stop 2024-04-24T05:46:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.907+0000 7fa0ae800640 1 -- 172.21.15.123:0/3380603485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c0051c0 msgr2=0x7fa0a81502e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.907+0000 7fa0ae800640 1 --2- 172.21.15.123:0/3380603485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c0051c0 0x7fa0a81502e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa09800e4b0 tx=0x7fa09800e970 comp rx=0 tx=0).stop 2024-04-24T05:46:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.907+0000 7fa0ae800640 1 -- 172.21.15.123:0/3380603485 shutdown_connections 2024-04-24T05:46:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.907+0000 7fa0ae800640 1 --2- 172.21.15.123:0/3380603485 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa08406c8f0 0x7fa08406edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.907+0000 7fa0ae800640 1 --2- 172.21.15.123:0/3380603485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0a8150b20 0x7fa0a8150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.907+0000 7fa0ae800640 1 --2- 172.21.15.123:0/3380603485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa0a814ee00 0x7fa0a814d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.907+0000 7fa0ae800640 1 --2- 172.21.15.123:0/3380603485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa08c0051c0 0x7fa0a81502e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.907+0000 7fa0ae800640 1 -- 172.21.15.123:0/3380603485 >> 172.21.15.123:0/3380603485 conn(0x7fa0a8072550 msgr2=0x7fa0a8157340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.907+0000 7fa0ae800640 1 -- 172.21.15.123:0/3380603485 shutdown_connections 2024-04-24T05:46:38.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:38.907+0000 7fa0ae800640 1 -- 172.21.15.123:0/3380603485 wait complete. 2024-04-24T05:46:38.927 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T05:46:38.927 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T05:46:38.927 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-04-24T05:46:38.927 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-24T05:46:38.927 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-24T05:46:38.927 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T05:46:38.927 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T05:46:38.927 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T05:46:38.927 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-24T05:46:38.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:38.927 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-04-24T05:46:38.979 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T05:46:38.979 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T05:46:38.979 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T05:46:38.980 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:38.980 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-24T05:46:38.980 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:38.980 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:38.980 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:38.980 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-04-24T05:46:38.980 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:38.980 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:38.980 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-04-24T05:46:38.980 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:38.980 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:38.980 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T05:46:38.980 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:38.980 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T05:46:38.981 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42588sec preferred_lft 42588sec 2024-04-24T05:46:38.981 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-04-24T05:46:38.981 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:38.981 INFO:teuthology.orchestra.run.smithi123.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T05:46:38.981 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 4a:4a:d7:fd:35:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:38.981 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T05:46:38.981 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:38.981 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::9041:44ff:fe5a:59c3/64 scope link 2024-04-24T05:46:38.981 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:38.981 INFO:teuthology.orchestra.run.smithi123.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T05:46:38.981 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 4a:4a:d7:fd:35:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:46:38.981 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::484a:d7ff:fefd:35cb/64 scope link 2024-04-24T05:46:38.981 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:38.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:38.982 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T05:46:39.056 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-04-24T05:46:39.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:39.060 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-04-24T05:46:39.110 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-04-24T05:46:39.110 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T05:46:39.111 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T05:46:39.111 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T05:46:39.111 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T05:46:39.111 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:39.112 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:46:39.112 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T05:46:39.112 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T05:46:39.112 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T05:46:39.112 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:46:39.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:39.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:39.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:39.321 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-24T05:46:39.367 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T05:46:39.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:39.368 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-04-24T05:46:39.464 INFO:teuthology.orchestra.run.smithi123.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-24T05:46:39.465 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:39.465 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:46:39.465 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T05:46:39.465 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-24T05:46:39.465 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:46:39.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:39.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:39.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:39.651 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-24T05:46:39.729 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T05:46:39.729 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T05:46:39.729 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T05:46:39.729 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:39.729 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-24T05:46:39.729 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:39.729 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T05:46:39.729 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 06:96:df:33:af:b6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T05:46:39.729 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T05:46:39.729 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:39.730 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::496:dfff:fe33:afb6/64 scope link 2024-04-24T05:46:39.730 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:39.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:39.733 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-24T05:46:39.816 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T05:46:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T05:46:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T05:46:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-24T05:46:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:39.816 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T05:46:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 06:96:df:33:af:b6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T05:46:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T05:46:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::496:dfff:fe33:afb6/64 scope link 2024-04-24T05:46:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:39.820 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-24T05:46:39.820 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:39.820 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:46:39.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-24T05:46:39.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-24T05:46:39.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-24T05:46:39.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-24T05:46:39.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-24T05:46:39.821 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:46:39.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:40.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:40.094 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:40.094 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:46:40.094 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.1 up 2024-04-24T05:46:40.094 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.1 master ceph-brx 2024-04-24T05:46:40.094 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:46:40.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:40.292 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:40.301 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-24T05:46:40.301 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T05:46:40.301 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-24T05:46:40.347 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-24T05:46:40.348 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T05:46:40.348 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-24T05:46:40.444 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-24T05:46:40.523 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T05:46:40.567 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-24T05:46:40.568 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-24T05:46:40.568 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-24T05:46:40.568 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-24T05:46:40.568 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-24T05:46:40.568 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-24T05:46:40.568 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-24T05:46:40.568 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-24T05:46:40.568 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-24T05:46:40.568 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-24T05:46:40.568 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-24T05:46:40.568 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-24T05:46:40.568 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T05:46:40.569 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T05:46:40.570 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T05:46:40.570 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-24T05:46:40.570 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:46:40.570 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:46:40.570 INFO:teuthology.orchestra.run.smithi123.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-24T05:46:40.570 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T05:46:40.570 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T05:46:40.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:40.571 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-24T05:46:40.659 INFO:teuthology.orchestra.run.smithi123.stdout:41 2024-04-24T05:46:40.659 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-04-24T05:46:40.659 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-24T05:46:40.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-24T05:46:40.772 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T05:46:40.811 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-24T05:46:40.811 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-24T05:46:40.811 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-24T05:46:40.812 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-24T05:46:40.812 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-24T05:46:40.812 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-24T05:46:40.812 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-24T05:46:40.812 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-24T05:46:40.812 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-24T05:46:40.812 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-24T05:46:40.812 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-24T05:46:40.812 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-24T05:46:40.812 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T05:46:40.812 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-24T05:46:40.812 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-24T05:46:40.812 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-24T05:46:40.812 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:46:40.813 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:46:40.814 INFO:teuthology.orchestra.run.smithi123.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-24T05:46:40.814 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T05:46:40.814 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T05:46:40.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:40.815 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-24T05:46:40.869 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi123.stderr:2024-04-24T05:46:40.868+0000 7f4bedaec480 -1 init, newargv = 0x557085af3010 newargc=15 2024-04-24T05:46:40.869 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi123.stderr:ceph-fuse[44334]: starting ceph client 2024-04-24T05:46:40.879 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi123.stderr:ceph-fuse[44334]: starting fuse 2024-04-24T05:46:40.897 INFO:teuthology.orchestra.run.smithi123.stdout:41 2024-04-24T05:46:40.897 INFO:teuthology.orchestra.run.smithi123.stdout:43 2024-04-24T05:46:40.897 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 43] 2024-04-24T05:46:40.898 DEBUG:teuthology.orchestra.run.smithi123:> sudo stdin-killer -- python3 -c ' 2024-04-24T05:46:40.898 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2024-04-24T05:46:40.898 DEBUG:teuthology.orchestra.run.smithi123:> import re 2024-04-24T05:46:40.898 DEBUG:teuthology.orchestra.run.smithi123:> import os 2024-04-24T05:46:40.898 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2024-04-24T05:46:40.898 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:46:40.898 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2024-04-24T05:46:40.898 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-24T05:46:40.898 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2024-04-24T05:46:40.898 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-24T05:46:40.898 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:46:40.898 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2024-04-24T05:46:40.898 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2024-04-24T05:46:40.898 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2024-04-24T05:46:40.899 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2024-04-24T05:46:40.899 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:46:40.899 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2024-04-24T05:46:40.899 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T05:46:40.899 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T05:46:40.899 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T05:46:40.899 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2024-04-24T05:46:40.899 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2024-04-24T05:46:40.899 DEBUG:teuthology.orchestra.run.smithi123:> return f 2024-04-24T05:46:40.899 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T05:46:40.899 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:46:40.899 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.2")) 2024-04-24T05:46:40.899 DEBUG:teuthology.orchestra.run.smithi123:> ' 2024-04-24T05:46:41.075 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.2.44334.asok 2024-04-24T05:46:41.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:41.092 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.44334.asok 2024-04-24T05:46:41.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:41.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.44334.asok status 2024-04-24T05:46:41.301 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-24T05:46:41.301 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2024-04-24T05:46:41.302 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T05:46:41.302 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T05:46:41.302 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "2", 2024-04-24T05:46:41.302 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2024-04-24T05:46:41.302 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-24T05:46:41.302 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "44334", 2024-04-24T05:46:41.302 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2024-04-24T05:46:41.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:46:41.302 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2024-04-24T05:46:41.302 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2024-04-24T05:46:41.303 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4845, 2024-04-24T05:46:41.303 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2024-04-24T05:46:41.303 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2024-04-24T05:46:41.303 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2024-04-24T05:46:41.303 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 4845 2024-04-24T05:46:41.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:46:41.303 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-24T05:46:41.303 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-24T05:46:41.303 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.2:0", 2024-04-24T05:46:41.303 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3846685046 2024-04-24T05:46:41.303 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:46:41.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:46:41.304 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-24T05:46:41.304 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-24T05:46:41.304 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.2:0", 2024-04-24T05:46:41.304 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3846685046 2024-04-24T05:46:41.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:46:41.304 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.4845 192.168.0.2:0/3846685046", 2024-04-24T05:46:41.304 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.2:0/3846685046", 2024-04-24T05:46:41.304 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2024-04-24T05:46:41.304 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 9, 2024-04-24T05:46:41.304 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 26, 2024-04-24T05:46:41.304 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2024-04-24T05:46:41.304 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2024-04-24T05:46:41.304 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2024-04-24T05:46:41.305 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-24T05:46:41.315 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T05:46:41.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.501+0000 7f4e2b200640 1 Processor -- start 2024-04-24T05:46:41.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.501+0000 7f4e2b200640 1 -- start start 2024-04-24T05:46:41.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.502+0000 7f4e2b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4e2c15e9f0 0x7f4e2c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:41.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.502+0000 7f4e2b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e2c0d13e0 0x7f4e2c0d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:41.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.502+0000 7f4e2b200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e2c162370 con 0x7f4e2c15e610 2024-04-24T05:46:41.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.502+0000 7f4e2b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4e2c162630 con 0x7f4e2c15e9f0 2024-04-24T05:46:41.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.502+0000 7f4e2b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2c162910 con 0x7f4e2c0d13e0 2024-04-24T05:46:41.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.503+0000 7f4e2a800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e2c0d13e0 0x7f4e2c0d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:41.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.503+0000 7f4e29e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f4e2c15e610 0x7f4e2c0d45a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.123:42118/0 (socket says 172.21.15.123:42118) 2024-04-24T05:46:41.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.503+0000 7f4e29e00640 1 -- 172.21.15.123:0/3139648599 learned_addr learned my addr 172.21.15.123:0/3139648599 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:41.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.503+0000 7f4e29400640 1 --2- 172.21.15.123:0/3139648599 >> v2:172.21.15.155:3300/0 conn(0x7f4e2c15e9f0 0x7f4e2c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:41.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.504+0000 7f4e29400640 1 -- 172.21.15.123:0/3139648599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e2c0d13e0 msgr2=0x7f4e2c0d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:41.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.504+0000 7f4e29400640 1 --2- 172.21.15.123:0/3139648599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e2c0d13e0 0x7f4e2c0d1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:41.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.504+0000 7f4e29400640 1 -- 172.21.15.123:0/3139648599 >> v1:172.21.15.155:6789/0 conn(0x7f4e2c15e610 legacy=0x7f4e2c0d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:41.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.504+0000 7f4e28a00640 1 -- 172.21.15.123:0/3139648599 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1829409754 0 0) 0x7f4e2c162370 con 0x7f4e2c15e610 2024-04-24T05:46:41.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.504+0000 7f4e29400640 1 -- 172.21.15.123:0/3139648599 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c162bf0 con 0x7f4e2c15e9f0 2024-04-24T05:46:41.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.504+0000 7f4e2a800640 1 --2- 172.21.15.123:0/3139648599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e2c0d13e0 0x7f4e2c0d1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:46:41.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.505+0000 7f4e29400640 1 --2- 172.21.15.123:0/3139648599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c15e9f0 0x7f4e2c0cfb20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4e180097a0 tx=0x7f4e1802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9798cf750493814 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:41.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.506+0000 7f4e28a00640 1 -- 172.21.15.123:0/3139648599 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e18037070 con 0x7f4e2c15e9f0 2024-04-24T05:46:41.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.506+0000 7f4e28a00640 1 -- 172.21.15.123:0/3139648599 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e18032440 con 0x7f4e2c15e9f0 2024-04-24T05:46:41.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.506+0000 7f4e28a00640 1 -- 172.21.15.123:0/3139648599 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e18032810 con 0x7f4e2c15e9f0 2024-04-24T05:46:41.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.506+0000 7f4e2b200640 1 -- 172.21.15.123:0/3139648599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c15e9f0 msgr2=0x7f4e2c0cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:41.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.506+0000 7f4e2b200640 1 --2- 172.21.15.123:0/3139648599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c15e9f0 0x7f4e2c0cfb20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4e180097a0 tx=0x7f4e1802a180 comp rx=0 tx=0).stop 2024-04-24T05:46:41.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.506+0000 7f4e2b200640 1 -- 172.21.15.123:0/3139648599 shutdown_connections 2024-04-24T05:46:41.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.506+0000 7f4e2b200640 1 --2- 172.21.15.123:0/3139648599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e2c0d13e0 0x7f4e2c0d1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:41.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.506+0000 7f4e2b200640 1 --2- 172.21.15.123:0/3139648599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c15e9f0 0x7f4e2c0cfb20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:41.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.506+0000 7f4e2b200640 1 -- 172.21.15.123:0/3139648599 >> 172.21.15.123:0/3139648599 conn(0x7f4e2c0b9cd0 msgr2=0x7f4e2c09d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:41.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.506+0000 7f4e2b200640 1 -- 172.21.15.123:0/3139648599 shutdown_connections 2024-04-24T05:46:41.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.506+0000 7f4e2b200640 1 -- 172.21.15.123:0/3139648599 wait complete. 2024-04-24T05:46:41.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.508+0000 7f4e2b200640 1 Processor -- start 2024-04-24T05:46:41.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.508+0000 7f4e2b200640 1 -- start start 2024-04-24T05:46:41.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.509+0000 7f4e2b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c0d13e0 0x7f4e2c0cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:41.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.509+0000 7f4e2b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e2c15e610 0x7f4e2c159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:41.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.509+0000 7f4e2b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e2c15e9f0 0x7f4e2c1e99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:41.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.509+0000 7f4e2b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c0d4cb0 con 0x7f4e2c15e610 2024-04-24T05:46:41.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.509+0000 7f4e2b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1631b0 con 0x7f4e2c0d13e0 2024-04-24T05:46:41.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.509+0000 7f4e2b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2c0d4e30 con 0x7f4e2c15e9f0 2024-04-24T05:46:41.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.509+0000 7f4e29400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e2c15e610 0x7f4e2c159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:41.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.509+0000 7f4e29400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e2c15e610 0x7f4e2c159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48440/0 (socket says 172.21.15.123:48440) 2024-04-24T05:46:41.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.509+0000 7f4e29400640 1 -- 172.21.15.123:0/2061076875 learned_addr learned my addr 172.21.15.123:0/2061076875 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:41.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.510+0000 7f4e29e00640 1 --2- 172.21.15.123:0/2061076875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c0d13e0 0x7f4e2c0cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:41.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.510+0000 7f4e2a800640 1 --2- 172.21.15.123:0/2061076875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e2c15e9f0 0x7f4e2c1e99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:41.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.510+0000 7f4e29400640 1 -- 172.21.15.123:0/2061076875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e2c15e9f0 msgr2=0x7f4e2c1e99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:41.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.510+0000 7f4e29400640 1 --2- 172.21.15.123:0/2061076875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e2c15e9f0 0x7f4e2c1e99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:41.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.510+0000 7f4e29400640 1 -- 172.21.15.123:0/2061076875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c0d13e0 msgr2=0x7f4e2c0cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:41.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.510+0000 7f4e29400640 1 --2- 172.21.15.123:0/2061076875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c0d13e0 0x7f4e2c0cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:41.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.510+0000 7f4e29400640 1 -- 172.21.15.123:0/2061076875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c15efd0 con 0x7f4e2c15e610 2024-04-24T05:46:41.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.510+0000 7f4e29400640 1 --2- 172.21.15.123:0/2061076875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e2c15e610 0x7f4e2c159050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4e180098c0 tx=0x7f4e180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:41.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.510+0000 7f4e22000640 1 -- 172.21.15.123:0/2061076875 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e18037040 con 0x7f4e2c15e610 2024-04-24T05:46:41.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.511+0000 7f4e22000640 1 -- 172.21.15.123:0/2061076875 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e18030500 con 0x7f4e2c15e610 2024-04-24T05:46:41.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.511+0000 7f4e22000640 1 -- 172.21.15.123:0/2061076875 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e180308d0 con 0x7f4e2c15e610 2024-04-24T05:46:41.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.511+0000 7f4e2b200640 1 -- 172.21.15.123:0/2061076875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2c15f2b0 con 0x7f4e2c15e610 2024-04-24T05:46:41.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.511+0000 7f4e2b200640 1 -- 172.21.15.123:0/2061076875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2c1e9f00 con 0x7f4e2c15e610 2024-04-24T05:46:41.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.512+0000 7f4e2b200640 1 -- 172.21.15.123:0/2061076875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e2c075b70 con 0x7f4e2c15e610 2024-04-24T05:46:41.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.513+0000 7f4e22000640 1 -- 172.21.15.123:0/2061076875 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e18044020 con 0x7f4e2c15e610 2024-04-24T05:46:41.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.514+0000 7f4e22000640 1 --2- 172.21.15.123:0/2061076875 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:41.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.514+0000 7f4e22000640 1 -- 172.21.15.123:0/2061076875 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6743+0+0 (secure 0 0 0) 0x7f4e180ac2c0 con 0x7f4e2c15e610 2024-04-24T05:46:41.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.514+0000 7f4e29e00640 1 --2- 172.21.15.123:0/2061076875 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:41.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.515+0000 7f4e29e00640 1 --2- 172.21.15.123:0/2061076875 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c000f60 tx=0x7f4e1c007b50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:41.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.519+0000 7f4e22000640 1 -- 172.21.15.123:0/2061076875 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e1807a240 con 0x7f4e2c15e610 2024-04-24T05:46:41.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.771+0000 7f4e2b200640 1 -- 172.21.15.123:0/2061076875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4e2c0b2180 con 0x7f4e2c15e610 2024-04-24T05:46:41.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.771+0000 7f4e22000640 1 -- 172.21.15.123:0/2061076875 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f4e18079b40 con 0x7f4e2c15e610 2024-04-24T05:46:41.773 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T05:46:41.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.774+0000 7f4e2b200640 1 -- 172.21.15.123:0/2061076875 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4e0c06c8f0 msgr2=0x7f4e0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:41.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.774+0000 7f4e2b200640 1 --2- 172.21.15.123:0/2061076875 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c000f60 tx=0x7f4e1c007b50 comp rx=0 tx=0).stop 2024-04-24T05:46:41.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.774+0000 7f4e2b200640 1 -- 172.21.15.123:0/2061076875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e2c15e610 msgr2=0x7f4e2c159050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:41.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.774+0000 7f4e2b200640 1 --2- 172.21.15.123:0/2061076875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e2c15e610 0x7f4e2c159050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4e180098c0 tx=0x7f4e180046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:41.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.774+0000 7f4e2b200640 1 -- 172.21.15.123:0/2061076875 shutdown_connections 2024-04-24T05:46:41.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.774+0000 7f4e2b200640 1 --2- 172.21.15.123:0/2061076875 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:41.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.774+0000 7f4e2b200640 1 --2- 172.21.15.123:0/2061076875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e2c15e9f0 0x7f4e2c1e99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:41.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.774+0000 7f4e2b200640 1 --2- 172.21.15.123:0/2061076875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e2c15e610 0x7f4e2c159050 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:41.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.774+0000 7f4e2b200640 1 --2- 172.21.15.123:0/2061076875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c0d13e0 0x7f4e2c0cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:41.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.774+0000 7f4e2b200640 1 -- 172.21.15.123:0/2061076875 >> 172.21.15.123:0/2061076875 conn(0x7f4e2c0b9cd0 msgr2=0x7f4e2c0d1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:41.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.774+0000 7f4e2b200640 1 -- 172.21.15.123:0/2061076875 shutdown_connections 2024-04-24T05:46:41.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:41.774+0000 7f4e2b200640 1 -- 172.21.15.123:0/2061076875 wait complete. 2024-04-24T05:46:41.793 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T05:46:41.794 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T05:46:41.794 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-04-24T05:46:41.794 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T05:46:41.794 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T05:46:41.794 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T05:46:41.794 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T05:46:41.794 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T05:46:41.794 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T05:46:41.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:41.794 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-04-24T05:46:41.850 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T05:46:41.900 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T05:46:41.900 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T05:46:41.900 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:41.900 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-24T05:46:41.900 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:41.900 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:41.901 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:41.901 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-04-24T05:46:41.901 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:41.901 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:41.901 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-04-24T05:46:41.901 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:41.901 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:41.901 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T05:46:41.901 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:41.901 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T05:46:41.901 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42606sec preferred_lft 42606sec 2024-04-24T05:46:41.901 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-04-24T05:46:41.901 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:41.903 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T05:46:41.903 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:41.903 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-24T05:46:41.903 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add name ceph-brx type bridge 2024-04-24T05:46:41.903 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip addr flush dev ceph-brx 2024-04-24T05:46:41.903 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set ceph-brx up 2024-04-24T05:46:41.903 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T05:46:41.903 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-24T05:46:42.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:42.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:42.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:42.384 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T05:46:42.444 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-24T05:46:42.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:42.448 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-04-24T05:46:42.503 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-04-24T05:46:42.503 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T05:46:42.503 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T05:46:42.503 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T05:46:42.503 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T05:46:42.503 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:42.504 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-24T05:46:42.504 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T05:46:42.504 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T05:46:42.504 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T05:46:42.504 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-24T05:46:42.594 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:43.217 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:43.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:43.227 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-24T05:46:43.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:43.281 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list-id 2024-04-24T05:46:43.332 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:43.332 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-24T05:46:43.332 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T05:46:43.332 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-24T05:46:43.332 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-24T05:46:43.435 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:43.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:43.516 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-24T05:46:43.517 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:43.517 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-24T05:46:43.517 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-24T05:46:43.517 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T05:46:43.517 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-24T05:46:43.517 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-24T05:46:43.517 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-24T05:46:43.517 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-24T05:46:43.614 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:43.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:43.793 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:43.793 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-24T05:46:43.793 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2024-04-24T05:46:43.793 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T05:46:43.794 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-24T05:46:43.887 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:43.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:43.991 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-24T05:46:43.991 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T05:46:43.991 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T05:46:44.036 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T05:46:44.036 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T05:46:44.036 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T05:46:44.131 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2024-04-24T05:46:44.203 DEBUG:teuthology.orchestra.run.smithi155:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T05:46:44.252 INFO:teuthology.orchestra.run.smithi155.stdout:/proc 2024-04-24T05:46:44.252 INFO:teuthology.orchestra.run.smithi155.stdout:/sys 2024-04-24T05:46:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:/dev 2024-04-24T05:46:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/security 2024-04-24T05:46:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/shm 2024-04-24T05:46:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/pts 2024-04-24T05:46:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:/run 2024-04-24T05:46:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/cgroup 2024-04-24T05:46:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/pstore 2024-04-24T05:46:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/bpf 2024-04-24T05:46:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:/ 2024-04-24T05:46:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/selinux 2024-04-24T05:46:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T05:46:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/hugepages 2024-04-24T05:46:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/mqueue 2024-04-24T05:46:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/debug 2024-04-24T05:46:44.254 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/tracing 2024-04-24T05:46:44.254 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/fuse/connections 2024-04-24T05:46:44.254 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/config 2024-04-24T05:46:44.254 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T05:46:44.254 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T05:46:44.254 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T05:46:44.254 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T05:46:44.254 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/1000 2024-04-24T05:46:44.254 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph 2024-04-24T05:46:44.254 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-4 2024-04-24T05:46:44.254 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-5 2024-04-24T05:46:44.254 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-6 2024-04-24T05:46:44.254 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-7 2024-04-24T05:46:44.255 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns 2024-04-24T05:46:44.255 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T05:46:44.255 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T05:46:44.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:44.256 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2024-04-24T05:46:44.301 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T05:46:44.301 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-24T05:46:44.349 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2024-04-24T05:46:44.419 DEBUG:teuthology.orchestra.run.smithi155:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T05:46:44.465 INFO:teuthology.orchestra.run.smithi155.stdout:/proc 2024-04-24T05:46:44.466 INFO:teuthology.orchestra.run.smithi155.stdout:/sys 2024-04-24T05:46:44.466 INFO:teuthology.orchestra.run.smithi155.stdout:/dev 2024-04-24T05:46:44.466 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/security 2024-04-24T05:46:44.466 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/shm 2024-04-24T05:46:44.466 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/pts 2024-04-24T05:46:44.466 INFO:teuthology.orchestra.run.smithi155.stdout:/run 2024-04-24T05:46:44.466 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/cgroup 2024-04-24T05:46:44.466 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/pstore 2024-04-24T05:46:44.466 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/bpf 2024-04-24T05:46:44.466 INFO:teuthology.orchestra.run.smithi155.stdout:/ 2024-04-24T05:46:44.466 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/selinux 2024-04-24T05:46:44.466 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/hugepages 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/mqueue 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/debug 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/tracing 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/fuse/connections 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/config 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/1000 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-4 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-5 2024-04-24T05:46:44.467 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-6 2024-04-24T05:46:44.468 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-7 2024-04-24T05:46:44.468 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns 2024-04-24T05:46:44.468 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T05:46:44.468 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T05:46:44.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:44.469 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2024-04-24T05:46:44.543 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2024-04-24T05:46:44.542+0000 7f0d5d96a480 -1 init, newargv = 0x55f09fedaf90 newargc=15 2024-04-24T05:46:44.543 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[39211]: starting ceph client 2024-04-24T05:46:44.556 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[39211]: starting fuse 2024-04-24T05:46:45.608 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2024-04-24T05:46:45.701 DEBUG:teuthology.orchestra.run.smithi155:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T05:46:45.756 INFO:teuthology.orchestra.run.smithi155.stdout:/proc 2024-04-24T05:46:45.757 INFO:teuthology.orchestra.run.smithi155.stdout:/sys 2024-04-24T05:46:45.757 INFO:teuthology.orchestra.run.smithi155.stdout:/dev 2024-04-24T05:46:45.757 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/security 2024-04-24T05:46:45.757 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/shm 2024-04-24T05:46:45.757 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/pts 2024-04-24T05:46:45.757 INFO:teuthology.orchestra.run.smithi155.stdout:/run 2024-04-24T05:46:45.757 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/cgroup 2024-04-24T05:46:45.757 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/pstore 2024-04-24T05:46:45.757 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/bpf 2024-04-24T05:46:45.757 INFO:teuthology.orchestra.run.smithi155.stdout:/ 2024-04-24T05:46:45.757 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/selinux 2024-04-24T05:46:45.757 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T05:46:45.757 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/hugepages 2024-04-24T05:46:45.758 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/mqueue 2024-04-24T05:46:45.758 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/debug 2024-04-24T05:46:45.758 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/tracing 2024-04-24T05:46:45.758 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/fuse/connections 2024-04-24T05:46:45.758 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/config 2024-04-24T05:46:45.758 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T05:46:45.758 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T05:46:45.758 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T05:46:45.758 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T05:46:45.758 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/1000 2024-04-24T05:46:45.758 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph 2024-04-24T05:46:45.758 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-4 2024-04-24T05:46:45.758 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-5 2024-04-24T05:46:45.759 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-6 2024-04-24T05:46:45.759 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-7 2024-04-24T05:46:45.759 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns 2024-04-24T05:46:45.759 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T05:46:45.759 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T05:46:45.759 INFO:teuthology.orchestra.run.smithi155.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-24T05:46:45.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:45.760 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2024-04-24T05:46:45.811 INFO:teuthology.orchestra.run.smithi155.stdout:42 2024-04-24T05:46:45.812 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-24T05:46:45.812 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2024-04-24T05:46:45.812 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2024-04-24T05:46:45.812 DEBUG:teuthology.orchestra.run.smithi155:> import re 2024-04-24T05:46:45.812 DEBUG:teuthology.orchestra.run.smithi155:> import os 2024-04-24T05:46:45.812 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2024-04-24T05:46:45.812 DEBUG:teuthology.orchestra.run.smithi155:> 2024-04-24T05:46:45.812 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2024-04-24T05:46:45.812 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-24T05:46:45.812 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> return f 2024-04-24T05:46:45.813 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T05:46:45.814 DEBUG:teuthology.orchestra.run.smithi155:> 2024-04-24T05:46:45.814 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2024-04-24T05:46:45.814 DEBUG:teuthology.orchestra.run.smithi155:> ' 2024-04-24T05:46:45.988 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.39211.asok 2024-04-24T05:46:45.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:46.005 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39211.asok 2024-04-24T05:46:46.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:46.005 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39211.asok status 2024-04-24T05:46:46.234 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-04-24T05:46:46.234 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2024-04-24T05:46:46.234 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T05:46:46.235 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T05:46:46.235 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2024-04-24T05:46:46.235 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2024-04-24T05:46:46.235 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-24T05:46:46.235 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "39211", 2024-04-24T05:46:46.235 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2024-04-24T05:46:46.235 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-24T05:46:46.235 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2024-04-24T05:46:46.235 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2024-04-24T05:46:46.235 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 4645, 2024-04-24T05:46:46.235 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2024-04-24T05:46:46.235 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2024-04-24T05:46:46.235 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2024-04-24T05:46:46.236 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 4645 2024-04-24T05:46:46.236 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-24T05:46:46.236 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2024-04-24T05:46:46.236 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-24T05:46:46.236 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2024-04-24T05:46:46.236 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 1837941462 2024-04-24T05:46:46.236 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-24T05:46:46.236 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-24T05:46:46.236 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2024-04-24T05:46:46.236 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-24T05:46:46.236 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2024-04-24T05:46:46.236 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 1837941462 2024-04-24T05:46:46.236 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-24T05:46:46.237 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.4645 192.168.0.1:0/1837941462", 2024-04-24T05:46:46.237 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/1837941462", 2024-04-24T05:46:46.237 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2024-04-24T05:46:46.237 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 9, 2024-04-24T05:46:46.237 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 27, 2024-04-24T05:46:46.237 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2024-04-24T05:46:46.237 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2024-04-24T05:46:46.237 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2024-04-24T05:46:46.237 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-04-24T05:46:46.249 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T05:46:46.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.452+0000 7fc390800640 1 Processor -- start 2024-04-24T05:46:46.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.453+0000 7fc390800640 1 -- start start 2024-04-24T05:46:46.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.454+0000 7fc390800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc38815a2a0 0x7fc3880ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:46.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.454+0000 7fc390800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc3880ad690 0x7fc3880af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:46.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.454+0000 7fc390800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3880af780 0x7fc388141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:46.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.454+0000 7fc390800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc38815a680 con 0x7fc3880ad690 2024-04-24T05:46:46.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.454+0000 7fc390800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc38815da00 con 0x7fc38815a2a0 2024-04-24T05:46:46.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.454+0000 7fc390800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3880afbf0 con 0x7fc3880af780 2024-04-24T05:46:46.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.455+0000 7fc38d400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc3880ad690 0x7fc3880af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:46.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.455+0000 7fc38d400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc3880ad690 0x7fc3880af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48456/0 (socket says 172.21.15.123:48456) 2024-04-24T05:46:46.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.455+0000 7fc38d400640 1 -- 172.21.15.123:0/2406921621 learned_addr learned my addr 172.21.15.123:0/2406921621 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:46.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.455+0000 7fc38de00640 1 --2- 172.21.15.123:0/2406921621 >> v2:172.21.15.155:3300/0 conn(0x7fc38815a2a0 0x7fc3880ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:46.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.455+0000 7fc38e800640 1 --2- 172.21.15.123:0/2406921621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3880af780 0x7fc388141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:46.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.455+0000 7fc38d400640 1 -- 172.21.15.123:0/2406921621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3880af780 msgr2=0x7fc388141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:46.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.455+0000 7fc38d400640 1 --2- 172.21.15.123:0/2406921621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3880af780 0x7fc388141e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:46.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.455+0000 7fc38d400640 1 -- 172.21.15.123:0/2406921621 >> v2:172.21.15.155:3300/0 conn(0x7fc38815a2a0 msgr2=0x7fc3880ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:46.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.455+0000 7fc38d400640 1 --2- 172.21.15.123:0/2406921621 >> v2:172.21.15.155:3300/0 conn(0x7fc38815a2a0 0x7fc3880ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:46.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.455+0000 7fc38d400640 1 -- 172.21.15.123:0/2406921621 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38815cdf0 con 0x7fc3880ad690 2024-04-24T05:46:46.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.456+0000 7fc38d400640 1 --2- 172.21.15.123:0/2406921621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3880ad690 0x7fc3880af240 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc3880b62d0 tx=0x7fc37802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b86c811ed6062d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:46.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.456+0000 7fc38ca00640 1 -- 172.21.15.123:0/2406921621 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378037070 con 0x7fc3880ad690 2024-04-24T05:46:46.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.457+0000 7fc38ca00640 1 -- 172.21.15.123:0/2406921621 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc378032440 con 0x7fc3880ad690 2024-04-24T05:46:46.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.457+0000 7fc38ca00640 1 -- 172.21.15.123:0/2406921621 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378032810 con 0x7fc3880ad690 2024-04-24T05:46:46.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.457+0000 7fc390800640 1 -- 172.21.15.123:0/2406921621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3880ad690 msgr2=0x7fc3880af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:46.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.457+0000 7fc390800640 1 --2- 172.21.15.123:0/2406921621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3880ad690 0x7fc3880af240 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc3880b62d0 tx=0x7fc37802a140 comp rx=0 tx=0).stop 2024-04-24T05:46:46.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.457+0000 7fc390800640 1 -- 172.21.15.123:0/2406921621 shutdown_connections 2024-04-24T05:46:46.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.457+0000 7fc390800640 1 --2- 172.21.15.123:0/2406921621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3880af780 0x7fc388141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:46.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.457+0000 7fc390800640 1 --2- 172.21.15.123:0/2406921621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3880ad690 0x7fc3880af240 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:46.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.457+0000 7fc390800640 1 --2- 172.21.15.123:0/2406921621 >> v2:172.21.15.155:3300/0 conn(0x7fc38815a2a0 0x7fc3880ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:46.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.457+0000 7fc390800640 1 -- 172.21.15.123:0/2406921621 >> 172.21.15.123:0/2406921621 conn(0x7fc3880b1e70 msgr2=0x7fc3880b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:46.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.457+0000 7fc390800640 1 -- 172.21.15.123:0/2406921621 shutdown_connections 2024-04-24T05:46:46.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.457+0000 7fc390800640 1 -- 172.21.15.123:0/2406921621 wait complete. 2024-04-24T05:46:46.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.459+0000 7fc390800640 1 Processor -- start 2024-04-24T05:46:46.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.459+0000 7fc390800640 1 -- start start 2024-04-24T05:46:46.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.460+0000 7fc390800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3880ad690 0x7fc3881517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:46.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.460+0000 7fc390800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3880af780 0x7fc3881f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:46.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.460+0000 7fc390800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38815a2a0 0x7fc3880ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:46.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.460+0000 7fc390800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc38815d690 con 0x7fc38815a2a0 2024-04-24T05:46:46.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.460+0000 7fc390800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc38815d3b0 con 0x7fc3880ad690 2024-04-24T05:46:46.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.460+0000 7fc390800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3881423c0 con 0x7fc3880af780 2024-04-24T05:46:46.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.461+0000 7fc38e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38815a2a0 0x7fc3880ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:46.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.461+0000 7fc38e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38815a2a0 0x7fc3880ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48466/0 (socket says 172.21.15.123:48466) 2024-04-24T05:46:46.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.461+0000 7fc38e800640 1 -- 172.21.15.123:0/1432544872 learned_addr learned my addr 172.21.15.123:0/1432544872 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.461+0000 7fc38de00640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3880ad690 0x7fc3881517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.461+0000 7fc38d400640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3880af780 0x7fc3881f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.461+0000 7fc38e800640 1 -- 172.21.15.123:0/1432544872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3880af780 msgr2=0x7fc3881f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.461+0000 7fc38e800640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3880af780 0x7fc3881f4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.461+0000 7fc38e800640 1 -- 172.21.15.123:0/1432544872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3880ad690 msgr2=0x7fc3881517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.461+0000 7fc38e800640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3880ad690 0x7fc3881517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.461+0000 7fc38e800640 1 -- 172.21.15.123:0/1432544872 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc388154390 con 0x7fc38815a2a0 2024-04-24T05:46:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.461+0000 7fc38d400640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3880af780 0x7fc3881f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.461+0000 7fc38de00640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3880ad690 0x7fc3881517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.461+0000 7fc38e800640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38815a2a0 0x7fc3880ca5c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc38400cd20 tx=0x7fc38400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.462+0000 7fc382000640 1 -- 172.21.15.123:0/1432544872 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384003b90 con 0x7fc38815a2a0 2024-04-24T05:46:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.462+0000 7fc382000640 1 -- 172.21.15.123:0/1432544872 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc384003d30 con 0x7fc38815a2a0 2024-04-24T05:46:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.462+0000 7fc382000640 1 -- 172.21.15.123:0/1432544872 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384004240 con 0x7fc38815a2a0 2024-04-24T05:46:46.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.462+0000 7fc390800640 1 -- 172.21.15.123:0/1432544872 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3881535d0 con 0x7fc38815a2a0 2024-04-24T05:46:46.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.462+0000 7fc390800640 1 -- 172.21.15.123:0/1432544872 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc388153b90 con 0x7fc38815a2a0 2024-04-24T05:46:46.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.463+0000 7fc390800640 1 -- 172.21.15.123:0/1432544872 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc388068a20 con 0x7fc38815a2a0 2024-04-24T05:46:46.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.464+0000 7fc382000640 1 -- 172.21.15.123:0/1432544872 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3840043e0 con 0x7fc38815a2a0 2024-04-24T05:46:46.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.465+0000 7fc382000640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc36c06c8f0 0x7fc36c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:46.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.466+0000 7fc382000640 1 -- 172.21.15.123:0/1432544872 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fc38408ca10 con 0x7fc38815a2a0 2024-04-24T05:46:46.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.466+0000 7fc38de00640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc36c06c8f0 0x7fc36c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:46.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.466+0000 7fc38de00640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc36c06c8f0 0x7fc36c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc37c006fd0 tx=0x7fc37c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:46.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.470+0000 7fc382000640 1 -- 172.21.15.123:0/1432544872 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc38405a990 con 0x7fc38815a2a0 2024-04-24T05:46:46.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.711+0000 7fc390800640 1 -- 172.21.15.123:0/1432544872 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc38806fdf0 con 0x7fc38815a2a0 2024-04-24T05:46:46.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.711+0000 7fc382000640 1 -- 172.21.15.123:0/1432544872 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fc38405a290 con 0x7fc38815a2a0 2024-04-24T05:46:46.713 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T05:46:46.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.714+0000 7fc390800640 1 -- 172.21.15.123:0/1432544872 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc36c06c8f0 msgr2=0x7fc36c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:46.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.714+0000 7fc390800640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc36c06c8f0 0x7fc36c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc37c006fd0 tx=0x7fc37c008040 comp rx=0 tx=0).stop 2024-04-24T05:46:46.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.714+0000 7fc390800640 1 -- 172.21.15.123:0/1432544872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38815a2a0 msgr2=0x7fc3880ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:46.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.714+0000 7fc390800640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38815a2a0 0x7fc3880ca5c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc38400cd20 tx=0x7fc38400a660 comp rx=0 tx=0).stop 2024-04-24T05:46:46.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.714+0000 7fc390800640 1 -- 172.21.15.123:0/1432544872 shutdown_connections 2024-04-24T05:46:46.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.714+0000 7fc390800640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc36c06c8f0 0x7fc36c06edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:46.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.714+0000 7fc390800640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38815a2a0 0x7fc3880ca5c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:46.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.714+0000 7fc390800640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3880af780 0x7fc3881f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:46.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.714+0000 7fc390800640 1 --2- 172.21.15.123:0/1432544872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3880ad690 0x7fc3881517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:46.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.714+0000 7fc390800640 1 -- 172.21.15.123:0/1432544872 >> 172.21.15.123:0/1432544872 conn(0x7fc3880b1e70 msgr2=0x7fc3880b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:46.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.714+0000 7fc390800640 1 -- 172.21.15.123:0/1432544872 shutdown_connections 2024-04-24T05:46:46.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:46.714+0000 7fc390800640 1 -- 172.21.15.123:0/1432544872 wait complete. 2024-04-24T05:46:46.735 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T05:46:46.735 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T05:46:46.735 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-04-24T05:46:46.735 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-24T05:46:46.735 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-24T05:46:46.735 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T05:46:46.735 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T05:46:46.736 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T05:46:46.736 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-24T05:46:46.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:46.736 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-04-24T05:46:46.794 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T05:46:46.795 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:46.796 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T05:46:46.796 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42601sec preferred_lft 42601sec 2024-04-24T05:46:46.796 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-04-24T05:46:46.796 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:46.796 INFO:teuthology.orchestra.run.smithi155.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T05:46:46.796 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 72:a4:d0:8d:4d:44 brd ff:ff:ff:ff:ff:ff 2024-04-24T05:46:46.796 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T05:46:46.796 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:46.796 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::b0ee:6bff:fede:f563/64 scope link 2024-04-24T05:46:46.796 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:46.796 INFO:teuthology.orchestra.run.smithi155.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T05:46:46.796 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 72:a4:d0:8d:4d:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T05:46:46.796 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::70a4:d0ff:fe8d:4d44/64 scope link 2024-04-24T05:46:46.797 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:46.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:46.798 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T05:46:46.882 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-04-24T05:46:46.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:46.886 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-04-24T05:46:46.938 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-04-24T05:46:46.938 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T05:46:46.938 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T05:46:46.938 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T05:46:46.938 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T05:46:46.939 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:46.939 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-24T05:46:46.939 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T05:46:46.939 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T05:46:46.939 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T05:46:46.939 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-24T05:46:47.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:47.139 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:47.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:47.149 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-04-24T05:46:47.198 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-24T05:46:47.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:47.199 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list-id 2024-04-24T05:46:47.293 INFO:teuthology.orchestra.run.smithi155.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-24T05:46:47.295 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:47.295 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-24T05:46:47.295 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T05:46:47.295 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-24T05:46:47.295 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-24T05:46:47.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:47.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:47.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:47.485 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-24T05:46:47.558 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T05:46:47.558 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T05:46:47.558 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T05:46:47.558 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:47.558 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-24T05:46:47.558 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:47.558 INFO:teuthology.orchestra.run.smithi155.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T05:46:47.558 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether b2:85:09:f9:01:ed brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T05:46:47.558 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T05:46:47.559 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:47.559 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::b085:9ff:fef9:1ed/64 scope link 2024-04-24T05:46:47.559 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:47.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:47.562 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-24T05:46:47.642 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T05:46:47.643 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T05:46:47.643 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T05:46:47.643 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:47.643 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-04-24T05:46:47.643 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:47.643 INFO:teuthology.orchestra.run.smithi155.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T05:46:47.643 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether b2:85:09:f9:01:ed brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-24T05:46:47.643 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-24T05:46:47.643 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:47.643 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::b085:9ff:fef9:1ed/64 scope link 2024-04-24T05:46:47.643 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:46:47.646 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-24T05:46:47.647 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:47.647 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-24T05:46:47.647 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-24T05:46:47.647 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-24T05:46:47.647 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-24T05:46:47.647 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-24T05:46:47.647 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-24T05:46:47.647 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-24T05:46:47.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:47.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:47.917 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:46:47.917 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-04-24T05:46:47.917 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.1 up 2024-04-24T05:46:47.917 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set dev brx.1 master ceph-brx 2024-04-24T05:46:47.917 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-04-24T05:46:48.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:46:48.107 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:48.117 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-24T05:46:48.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T05:46:48.117 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-24T05:46:48.163 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-24T05:46:48.164 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T05:46:48.164 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-24T05:46:48.260 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2024-04-24T05:46:48.340 DEBUG:teuthology.orchestra.run.smithi155:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T05:46:48.392 INFO:teuthology.orchestra.run.smithi155.stdout:/proc 2024-04-24T05:46:48.392 INFO:teuthology.orchestra.run.smithi155.stdout:/sys 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/dev 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/security 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/shm 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/pts 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/run 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/cgroup 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/pstore 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/bpf 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/ 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/selinux 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/hugepages 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/mqueue 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/debug 2024-04-24T05:46:48.393 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/tracing 2024-04-24T05:46:48.394 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/fuse/connections 2024-04-24T05:46:48.394 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/config 2024-04-24T05:46:48.394 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T05:46:48.394 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T05:46:48.394 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T05:46:48.394 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T05:46:48.394 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/1000 2024-04-24T05:46:48.394 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph 2024-04-24T05:46:48.394 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-4 2024-04-24T05:46:48.394 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-5 2024-04-24T05:46:48.394 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-6 2024-04-24T05:46:48.394 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-7 2024-04-24T05:46:48.394 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns 2024-04-24T05:46:48.394 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T05:46:48.395 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T05:46:48.395 INFO:teuthology.orchestra.run.smithi155.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-24T05:46:48.395 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T05:46:48.395 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T05:46:48.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:48.396 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2024-04-24T05:46:48.447 INFO:teuthology.orchestra.run.smithi155.stdout:42 2024-04-24T05:46:48.448 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-04-24T05:46:48.448 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-24T05:46:48.453 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2024-04-24T05:46:48.521 DEBUG:teuthology.orchestra.run.smithi155:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T05:46:48.561 INFO:teuthology.orchestra.run.smithi155.stdout:/proc 2024-04-24T05:46:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:/sys 2024-04-24T05:46:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:/dev 2024-04-24T05:46:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/security 2024-04-24T05:46:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/shm 2024-04-24T05:46:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/pts 2024-04-24T05:46:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:/run 2024-04-24T05:46:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/cgroup 2024-04-24T05:46:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/pstore 2024-04-24T05:46:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/bpf 2024-04-24T05:46:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:/ 2024-04-24T05:46:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/selinux 2024-04-24T05:46:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T05:46:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/hugepages 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/mqueue 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/debug 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/tracing 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/fuse/connections 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/config 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/1000 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-4 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-5 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-6 2024-04-24T05:46:48.563 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-7 2024-04-24T05:46:48.564 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns 2024-04-24T05:46:48.564 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T05:46:48.564 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T05:46:48.564 INFO:teuthology.orchestra.run.smithi155.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-24T05:46:48.564 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T05:46:48.564 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-24T05:46:48.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:48.565 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2024-04-24T05:46:48.621 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi155.stderr:2024-04-24T05:46:48.620+0000 7fca5d587480 -1 init, newargv = 0x557d1acc5000 newargc=15 2024-04-24T05:46:48.621 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi155.stderr:ceph-fuse[39719]: starting ceph client 2024-04-24T05:46:48.633 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi155.stderr:ceph-fuse[39719]: starting fuse 2024-04-24T05:46:48.651 INFO:teuthology.orchestra.run.smithi155.stdout:42 2024-04-24T05:46:48.651 INFO:teuthology.orchestra.run.smithi155.stdout:43 2024-04-24T05:46:48.651 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-04-24T05:46:48.651 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2024-04-24T05:46:48.651 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2024-04-24T05:46:48.651 DEBUG:teuthology.orchestra.run.smithi155:> import re 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> import os 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T05:46:48.652 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T05:46:48.653 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T05:46:48.653 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2024-04-24T05:46:48.653 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2024-04-24T05:46:48.653 DEBUG:teuthology.orchestra.run.smithi155:> return f 2024-04-24T05:46:48.653 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T05:46:48.653 DEBUG:teuthology.orchestra.run.smithi155:> 2024-04-24T05:46:48.653 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.3")) 2024-04-24T05:46:48.653 DEBUG:teuthology.orchestra.run.smithi155:> ' 2024-04-24T05:46:48.844 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.3.39719.asok 2024-04-24T05:46:48.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-04-24T05:46:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:46:48.861 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.39719.asok 2024-04-24T05:46:48.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:48.861 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.39719.asok status 2024-04-24T05:46:49.089 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-04-24T05:46:49.089 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2024-04-24T05:46:49.089 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T05:46:49.089 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T05:46:49.089 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "3", 2024-04-24T05:46:49.089 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2024-04-24T05:46:49.089 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-24T05:46:49.089 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "39719", 2024-04-24T05:46:49.090 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2024-04-24T05:46:49.090 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-24T05:46:49.090 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2024-04-24T05:46:49.090 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2024-04-24T05:46:49.090 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 4654, 2024-04-24T05:46:49.090 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2024-04-24T05:46:49.090 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2024-04-24T05:46:49.090 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2024-04-24T05:46:49.090 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 4654 2024-04-24T05:46:49.090 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-24T05:46:49.090 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2024-04-24T05:46:49.090 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-24T05:46:49.090 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.2:0", 2024-04-24T05:46:49.090 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 3593291138 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.2:0", 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 3593291138 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.4654 192.168.0.2:0/3593291138", 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.2:0/3593291138", 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 9, 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 27, 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2024-04-24T05:46:49.091 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2024-04-24T05:46:49.092 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-04-24T05:46:49.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:49.104 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2024-04-24T05:46:49.104 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T05:46:49.162 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2024-04-24T05:46:49.163 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T05:46:49.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:49.163 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T05:46:49.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:49.249 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2024-04-24T05:46:49.249 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-24T05:46:49.299 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2024-04-24T05:46:49.300 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-24T05:46:49.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:49.300 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-24T05:46:49.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:49.378 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2024-04-24T05:46:49.378 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-24T05:46:49.435 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2024-04-24T05:46:49.436 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-24T05:46:49.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:49.436 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T05:46:49.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:49.521 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2024-04-24T05:46:49.521 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-24T05:46:49.573 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2024-04-24T05:46:49.573 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-24T05:46:49.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:46:49.573 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-24T05:46:49.657 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-24T05:46:49.675 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:49.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.879+0000 7f5f72600640 1 Processor -- start 2024-04-24T05:46:49.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.879+0000 7f5f72600640 1 -- start start 2024-04-24T05:46:49.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.881+0000 7f5f72600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5f6c15c5f0 0x7f5f6c160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:49.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.881+0000 7f5f72600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f6c15d700 con 0x7f5f6c0cc930 2024-04-24T05:46:49.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.881+0000 7f5f72600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f6c160160 con 0x7f5f6c15d0e0 2024-04-24T05:46:49.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.881+0000 7f5f72600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5f6c15d9e0 con 0x7f5f6c15c5f0 2024-04-24T05:46:49.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.881+0000 7f5f71200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5f6c15c5f0 0x7f5f6c160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:49.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.881+0000 7f5f6be00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f5f6c0cc930 0x7f5f6c15c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43992/0 (socket says 172.21.15.123:43992) 2024-04-24T05:46:49.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.881+0000 7f5f71200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5f6c15c5f0 0x7f5f6c160c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56426/0 (socket says 172.21.15.123:56426) 2024-04-24T05:46:49.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.881+0000 7f5f6be00640 1 -- 172.21.15.123:0/1056017412 learned_addr learned my addr 172.21.15.123:0/1056017412 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:49.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.882+0000 7f5f71200640 1 -- 172.21.15.123:0/1056017412 >> v1:172.21.15.155:6789/0 conn(0x7f5f6c15d0e0 legacy=0x7f5f6c0cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:49.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.882+0000 7f5f71200640 1 -- 172.21.15.123:0/1056017412 >> v1:172.21.15.123:6789/0 conn(0x7f5f6c0cc930 legacy=0x7f5f6c15c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:49.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.882+0000 7f5f71200640 1 -- 172.21.15.123:0/1056017412 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c160440 con 0x7f5f6c15c5f0 2024-04-24T05:46:49.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.882+0000 7f5f6b400640 1 -- 172.21.15.123:0/1056017412 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3436054929 0 0) 0x7f5f6c15d700 con 0x7f5f6c0cc930 2024-04-24T05:46:49.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.883+0000 7f5f71200640 1 --2- 172.21.15.123:0/1056017412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c15c5f0 0x7f5f6c160c40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c15db60 tx=0x7f5f6002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbca12272fb74a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:49.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.883+0000 7f5f6b400640 1 -- 172.21.15.123:0/1056017412 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60037070 con 0x7f5f6c15c5f0 2024-04-24T05:46:49.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.883+0000 7f5f6b400640 1 -- 172.21.15.123:0/1056017412 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f60032440 con 0x7f5f6c15c5f0 2024-04-24T05:46:49.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.883+0000 7f5f6b400640 1 -- 172.21.15.123:0/1056017412 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60032810 con 0x7f5f6c15c5f0 2024-04-24T05:46:49.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.884+0000 7f5f72600640 1 -- 172.21.15.123:0/1056017412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c15c5f0 msgr2=0x7f5f6c160c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:49.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.884+0000 7f5f72600640 1 --2- 172.21.15.123:0/1056017412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c15c5f0 0x7f5f6c160c40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c15db60 tx=0x7f5f6002a160 comp rx=0 tx=0).stop 2024-04-24T05:46:49.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.884+0000 7f5f72600640 1 -- 172.21.15.123:0/1056017412 shutdown_connections 2024-04-24T05:46:49.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.884+0000 7f5f72600640 1 --2- 172.21.15.123:0/1056017412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c15c5f0 0x7f5f6c160c40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:49.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.884+0000 7f5f72600640 1 -- 172.21.15.123:0/1056017412 >> 172.21.15.123:0/1056017412 conn(0x7f5f6c0b3e50 msgr2=0x7f5f6c0b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:49.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.884+0000 7f5f72600640 1 -- 172.21.15.123:0/1056017412 shutdown_connections 2024-04-24T05:46:49.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.884+0000 7f5f72600640 1 -- 172.21.15.123:0/1056017412 wait complete. 2024-04-24T05:46:49.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.886+0000 7f5f72600640 1 Processor -- start 2024-04-24T05:46:49.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.886+0000 7f5f72600640 1 -- start start 2024-04-24T05:46:49.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.887+0000 7f5f72600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f6c0cc930 0x7f5f6c0cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:49.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.887+0000 7f5f72600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f6c15c5f0 0x7f5f6c0ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:49.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.887+0000 7f5f72600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c15d0e0 0x7f5f6c1f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:49.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.887+0000 7f5f72600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c0ccf60 con 0x7f5f6c15d0e0 2024-04-24T05:46:49.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.887+0000 7f5f72600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c0ccde0 con 0x7f5f6c0cc930 2024-04-24T05:46:49.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.887+0000 7f5f72600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6c0cd0e0 con 0x7f5f6c15c5f0 2024-04-24T05:46:49.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.887+0000 7f5f71c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c15d0e0 0x7f5f6c1f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:49.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.887+0000 7f5f71c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c15d0e0 0x7f5f6c1f7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56428/0 (socket says 172.21.15.123:56428) 2024-04-24T05:46:49.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.887+0000 7f5f71c00640 1 -- 172.21.15.123:0/1081784829 learned_addr learned my addr 172.21.15.123:0/1081784829 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:49.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.888+0000 7f5f6be00640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f6c15c5f0 0x7f5f6c0ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:49.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.888+0000 7f5f71200640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f6c0cc930 0x7f5f6c0cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:49.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.888+0000 7f5f71c00640 1 -- 172.21.15.123:0/1081784829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f6c15c5f0 msgr2=0x7f5f6c0ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:49.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.888+0000 7f5f71c00640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f6c15c5f0 0x7f5f6c0ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:49.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.888+0000 7f5f71c00640 1 -- 172.21.15.123:0/1081784829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f6c0cc930 msgr2=0x7f5f6c0cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:49.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.888+0000 7f5f71c00640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f6c0cc930 0x7f5f6c0cc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:49.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.888+0000 7f5f71c00640 1 -- 172.21.15.123:0/1081784829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c1e7660 con 0x7f5f6c15d0e0 2024-04-24T05:46:49.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.888+0000 7f5f6be00640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f6c15c5f0 0x7f5f6c0ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:49.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.888+0000 7f5f71c00640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c15d0e0 0x7f5f6c1f7970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400c780 tx=0x7f5f6400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:49.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.888+0000 7f5f68c00640 1 -- 172.21.15.123:0/1081784829 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64017070 con 0x7f5f6c15d0e0 2024-04-24T05:46:49.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.888+0000 7f5f68c00640 1 -- 172.21.15.123:0/1081784829 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f64003da0 con 0x7f5f6c15d0e0 2024-04-24T05:46:49.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.888+0000 7f5f68c00640 1 -- 172.21.15.123:0/1081784829 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f640052b0 con 0x7f5f6c15d0e0 2024-04-24T05:46:49.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.888+0000 7f5f72600640 1 -- 172.21.15.123:0/1081784829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6c1e7940 con 0x7f5f6c15d0e0 2024-04-24T05:46:49.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.889+0000 7f5f72600640 1 -- 172.21.15.123:0/1081784829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f6c1e7f00 con 0x7f5f6c15d0e0 2024-04-24T05:46:49.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.890+0000 7f5f72600640 1 -- 172.21.15.123:0/1081784829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f6c075b70 con 0x7f5f6c15d0e0 2024-04-24T05:46:49.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.891+0000 7f5f68c00640 1 -- 172.21.15.123:0/1081784829 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f64005450 con 0x7f5f6c15d0e0 2024-04-24T05:46:49.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.892+0000 7f5f68c00640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5f4c06c8f0 0x7f5f4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:49.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.892+0000 7f5f68c00640 1 -- 172.21.15.123:0/1081784829 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f5f6408b500 con 0x7f5f6c15d0e0 2024-04-24T05:46:49.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.892+0000 7f5f71200640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5f4c06c8f0 0x7f5f4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:49.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.893+0000 7f5f71200640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5f4c06c8f0 0x7f5f4c06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5f60034000 tx=0x7f5f6002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:49.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:49.896+0000 7f5f68c00640 1 -- 172.21.15.123:0/1081784829 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f64059510 con 0x7f5f6c15d0e0 2024-04-24T05:46:50.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.096+0000 7f5f72600640 1 -- 172.21.15.123:0/1081784829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f6c0b3740 con 0x7f5f6c15d0e0 2024-04-24T05:46:50.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.098+0000 7f5f68c00640 1 -- 172.21.15.123:0/1081784829 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7f5f64058e10 con 0x7f5f6c15d0e0 2024-04-24T05:46:50.101 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:50.101 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:50.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.101+0000 7f5f72600640 1 -- 172.21.15.123:0/1081784829 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5f4c06c8f0 msgr2=0x7f5f4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.101+0000 7f5f72600640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5f4c06c8f0 0x7f5f4c06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5f60034000 tx=0x7f5f6002e000 comp rx=0 tx=0).stop 2024-04-24T05:46:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.101+0000 7f5f72600640 1 -- 172.21.15.123:0/1081784829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c15d0e0 msgr2=0x7f5f6c1f7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.101+0000 7f5f72600640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c15d0e0 0x7f5f6c1f7970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400c780 tx=0x7f5f6400cc40 comp rx=0 tx=0).stop 2024-04-24T05:46:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.101+0000 7f5f72600640 1 -- 172.21.15.123:0/1081784829 shutdown_connections 2024-04-24T05:46:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.101+0000 7f5f72600640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5f4c06c8f0 0x7f5f4c06edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.101+0000 7f5f72600640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f6c15d0e0 0x7f5f6c1f7970 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.101+0000 7f5f72600640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f6c15c5f0 0x7f5f6c0ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.101+0000 7f5f72600640 1 --2- 172.21.15.123:0/1081784829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f6c0cc930 0x7f5f6c0cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.101+0000 7f5f72600640 1 -- 172.21.15.123:0/1081784829 >> 172.21.15.123:0/1081784829 conn(0x7f5f6c0b3e50 msgr2=0x7f5f6c15db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.101+0000 7f5f72600640 1 -- 172.21.15.123:0/1081784829 shutdown_connections 2024-04-24T05:46:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.101+0000 7f5f72600640 1 -- 172.21.15.123:0/1081784829 wait complete. 2024-04-24T05:46:50.121 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:46:50.121 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:46:50.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.317+0000 7ffb69c00640 1 Processor -- start 2024-04-24T05:46:50.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.317+0000 7ffb69c00640 1 -- start start 2024-04-24T05:46:50.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.319+0000 7ffb69c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ffb6415c2f0 0x7ffb64141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:50.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.319+0000 7ffb69c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffb640cc5d0 0x7ffb640acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:50.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.319+0000 7ffb69c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb6406ba00 con 0x7ffb640ad510 2024-04-24T05:46:50.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.319+0000 7ffb69c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ffb6406bd30 con 0x7ffb640cc5d0 2024-04-24T05:46:50.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.319+0000 7ffb69c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ffb640ad940 con 0x7ffb6415c2f0 2024-04-24T05:46:50.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.319+0000 7ffb63400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffb640cc5d0 0x7ffb640acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:50.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.319+0000 7ffb63400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffb640cc5d0 0x7ffb640acfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56430/0 (socket says 172.21.15.123:56430) 2024-04-24T05:46:50.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.319+0000 7ffb63400640 1 -- 172.21.15.123:0/2418086409 learned_addr learned my addr 172.21.15.123:0/2418086409 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.319+0000 7ffb63e00640 1 --2- 172.21.15.123:0/2418086409 >> v2:172.21.15.155:3300/0 conn(0x7ffb6415c2f0 0x7ffb64141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.319+0000 7ffb63400640 1 -- 172.21.15.123:0/2418086409 >> v2:172.21.15.155:3300/0 conn(0x7ffb6415c2f0 msgr2=0x7ffb64141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.320+0000 7ffb63400640 1 --2- 172.21.15.123:0/2418086409 >> v2:172.21.15.155:3300/0 conn(0x7ffb6415c2f0 0x7ffb64141ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.320+0000 7ffb63400640 1 -- 172.21.15.123:0/2418086409 >> v1:172.21.15.155:6789/0 conn(0x7ffb640ad510 legacy=0x7ffb6415ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.320+0000 7ffb63400640 1 -- 172.21.15.123:0/2418086409 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb64142530 con 0x7ffb640cc5d0 2024-04-24T05:46:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.321+0000 7ffb63400640 1 --2- 172.21.15.123:0/2418086409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb640cc5d0 0x7ffb640acfd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffb64073fd0 tx=0x7ffb5002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2028ed823d885cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:50.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.321+0000 7ffb62a00640 1 -- 172.21.15.123:0/2418086409 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50037070 con 0x7ffb640cc5d0 2024-04-24T05:46:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.321+0000 7ffb62a00640 1 -- 172.21.15.123:0/2418086409 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb50032440 con 0x7ffb640cc5d0 2024-04-24T05:46:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.321+0000 7ffb62a00640 1 -- 172.21.15.123:0/2418086409 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50032810 con 0x7ffb640cc5d0 2024-04-24T05:46:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.321+0000 7ffb69c00640 1 -- 172.21.15.123:0/2418086409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb640cc5d0 msgr2=0x7ffb640acfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.321+0000 7ffb69c00640 1 --2- 172.21.15.123:0/2418086409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb640cc5d0 0x7ffb640acfd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffb64073fd0 tx=0x7ffb5002a160 comp rx=0 tx=0).stop 2024-04-24T05:46:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.322+0000 7ffb69c00640 1 -- 172.21.15.123:0/2418086409 shutdown_connections 2024-04-24T05:46:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.322+0000 7ffb69c00640 1 --2- 172.21.15.123:0/2418086409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb640cc5d0 0x7ffb640acfd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.322+0000 7ffb69c00640 1 --2- 172.21.15.123:0/2418086409 >> v2:172.21.15.155:3300/0 conn(0x7ffb6415c2f0 0x7ffb64141ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.322+0000 7ffb69c00640 1 -- 172.21.15.123:0/2418086409 >> 172.21.15.123:0/2418086409 conn(0x7ffb640b1cf0 msgr2=0x7ffb640b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.322+0000 7ffb69c00640 1 -- 172.21.15.123:0/2418086409 shutdown_connections 2024-04-24T05:46:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.322+0000 7ffb69c00640 1 -- 172.21.15.123:0/2418086409 wait complete. 2024-04-24T05:46:50.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.323+0000 7ffb69c00640 1 Processor -- start 2024-04-24T05:46:50.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.324+0000 7ffb69c00640 1 -- start start 2024-04-24T05:46:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.324+0000 7ffb69c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb640ad510 0x7ffb641f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.324+0000 7ffb69c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb640cc5d0 0x7ffb64158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.324+0000 7ffb69c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb6415c2f0 0x7ffb640cf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.324+0000 7ffb69c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64142220 con 0x7ffb6415c2f0 2024-04-24T05:46:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.324+0000 7ffb69c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64142af0 con 0x7ffb640ad510 2024-04-24T05:46:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.324+0000 7ffb69c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffb6406d1d0 con 0x7ffb640cc5d0 2024-04-24T05:46:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.325+0000 7ffb69200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb6415c2f0 0x7ffb640cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.325+0000 7ffb69200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb6415c2f0 0x7ffb640cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56444/0 (socket says 172.21.15.123:56444) 2024-04-24T05:46:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.325+0000 7ffb69200640 1 -- 172.21.15.123:0/4034345762 learned_addr learned my addr 172.21.15.123:0/4034345762 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.325+0000 7ffb63400640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb640cc5d0 0x7ffb64158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.325+0000 7ffb63e00640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb640ad510 0x7ffb641f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.325+0000 7ffb69200640 1 -- 172.21.15.123:0/4034345762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb640cc5d0 msgr2=0x7ffb64158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.325+0000 7ffb69200640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb640cc5d0 0x7ffb64158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.325+0000 7ffb69200640 1 -- 172.21.15.123:0/4034345762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb640ad510 msgr2=0x7ffb641f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.325+0000 7ffb69200640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb640ad510 0x7ffb641f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.325+0000 7ffb69200640 1 -- 172.21.15.123:0/4034345762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb640ce0c0 con 0x7ffb6415c2f0 2024-04-24T05:46:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.325+0000 7ffb63400640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb640cc5d0 0x7ffb64158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.325+0000 7ffb63e00640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb640ad510 0x7ffb641f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.326+0000 7ffb69200640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb6415c2f0 0x7ffb640cf160 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00b9e0 tx=0x7ffb5c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.326+0000 7ffb5be00640 1 -- 172.21.15.123:0/4034345762 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c003b70 con 0x7ffb6415c2f0 2024-04-24T05:46:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.326+0000 7ffb5be00640 1 -- 172.21.15.123:0/4034345762 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5c003d10 con 0x7ffb6415c2f0 2024-04-24T05:46:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.326+0000 7ffb69c00640 1 -- 172.21.15.123:0/4034345762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb640ce3a0 con 0x7ffb6415c2f0 2024-04-24T05:46:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.326+0000 7ffb5be00640 1 -- 172.21.15.123:0/4034345762 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c00cae0 con 0x7ffb6415c2f0 2024-04-24T05:46:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.326+0000 7ffb69c00640 1 -- 172.21.15.123:0/4034345762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb640ce960 con 0x7ffb6415c2f0 2024-04-24T05:46:50.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.327+0000 7ffb69c00640 1 -- 172.21.15.123:0/4034345762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb64068a20 con 0x7ffb6415c2f0 2024-04-24T05:46:50.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.328+0000 7ffb5be00640 1 -- 172.21.15.123:0/4034345762 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb5c00cc80 con 0x7ffb6415c2f0 2024-04-24T05:46:50.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.329+0000 7ffb5be00640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ffb4406c8f0 0x7ffb4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:50.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.330+0000 7ffb5be00640 1 -- 172.21.15.123:0/4034345762 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7ffb5c08bda0 con 0x7ffb6415c2f0 2024-04-24T05:46:50.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.330+0000 7ffb63e00640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ffb4406c8f0 0x7ffb4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:50.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.330+0000 7ffb63e00640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ffb4406c8f0 0x7ffb4406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffb54008890 tx=0x7ffb54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:50.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.335+0000 7ffb5be00640 1 -- 172.21.15.123:0/4034345762 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb5c059d20 con 0x7ffb6415c2f0 2024-04-24T05:46:50.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.532+0000 7ffb69c00640 1 -- 172.21.15.123:0/4034345762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffb640b19e0 con 0x7ffb6415c2f0 2024-04-24T05:46:50.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.533+0000 7ffb5be00640 1 -- 172.21.15.123:0/4034345762 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7ffb5c059620 con 0x7ffb6415c2f0 2024-04-24T05:46:50.535 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:46:50.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.536+0000 7ffb69c00640 1 -- 172.21.15.123:0/4034345762 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ffb4406c8f0 msgr2=0x7ffb4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:50.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.536+0000 7ffb69c00640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ffb4406c8f0 0x7ffb4406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffb54008890 tx=0x7ffb54008040 comp rx=0 tx=0).stop 2024-04-24T05:46:50.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.536+0000 7ffb69c00640 1 -- 172.21.15.123:0/4034345762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb6415c2f0 msgr2=0x7ffb640cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:50.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.536+0000 7ffb69c00640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb6415c2f0 0x7ffb640cf160 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00b9e0 tx=0x7ffb5c00bea0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.536+0000 7ffb69c00640 1 -- 172.21.15.123:0/4034345762 shutdown_connections 2024-04-24T05:46:50.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.536+0000 7ffb69c00640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ffb4406c8f0 0x7ffb4406edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.536+0000 7ffb69c00640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb6415c2f0 0x7ffb640cf160 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.536+0000 7ffb69c00640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb640cc5d0 0x7ffb64158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.536+0000 7ffb69c00640 1 --2- 172.21.15.123:0/4034345762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb640ad510 0x7ffb641f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.536+0000 7ffb69c00640 1 -- 172.21.15.123:0/4034345762 >> 172.21.15.123:0/4034345762 conn(0x7ffb640b1cf0 msgr2=0x7ffb6406db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:50.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.536+0000 7ffb69c00640 1 -- 172.21.15.123:0/4034345762 shutdown_connections 2024-04-24T05:46:50.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.536+0000 7ffb69c00640 1 -- 172.21.15.123:0/4034345762 wait complete. 2024-04-24T05:46:50.556 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:46:50.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.763+0000 7f0a97e00640 1 Processor -- start 2024-04-24T05:46:50.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.763+0000 7f0a97e00640 1 -- start start 2024-04-24T05:46:50.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.765+0000 7f0a97e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a9815c2f0 0x7f0a98141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:50.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.765+0000 7f0a97e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0a980ad6e0 0x7f0a9806c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:50.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.765+0000 7f0a97e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a9806ba00 con 0x7f0a980cc5d0 2024-04-24T05:46:50.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.765+0000 7f0a97e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0a9806bd30 con 0x7f0a980ad6e0 2024-04-24T05:46:50.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.765+0000 7f0a97e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0a98142530 con 0x7f0a9815c2f0 2024-04-24T05:46:50.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.766+0000 7f0a97400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0a980ad6e0 0x7f0a9806c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:50.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.766+0000 7f0a8fe00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f0a980cc5d0 0x7f0a980acfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.123:34606/0 (socket says 172.21.15.123:34606) 2024-04-24T05:46:50.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.766+0000 7f0a8fe00640 1 -- 172.21.15.123:0/2570216810 learned_addr learned my addr 172.21.15.123:0/2570216810 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:50.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.766+0000 7f0a96a00640 1 --2- 172.21.15.123:0/2570216810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a9815c2f0 0x7f0a98141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:50.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.766+0000 7f0a97400640 1 -- 172.21.15.123:0/2570216810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a9815c2f0 msgr2=0x7f0a98141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:50.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.766+0000 7f0a97400640 1 --2- 172.21.15.123:0/2570216810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a9815c2f0 0x7f0a98141ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.766+0000 7f0a97400640 1 -- 172.21.15.123:0/2570216810 >> v1:172.21.15.155:6789/0 conn(0x7f0a980cc5d0 legacy=0x7f0a980acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:50.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.766+0000 7f0a97400640 1 -- 172.21.15.123:0/2570216810 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a98142810 con 0x7f0a980ad6e0 2024-04-24T05:46:50.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.768+0000 7f0a97400640 1 --2- 172.21.15.123:0/2570216810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a980ad6e0 0x7f0a9806c1c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a9806beb0 tx=0x7f0a9002a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5920a9a67a429fa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:50.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.768+0000 7f0a96000640 1 -- 172.21.15.123:0/2570216810 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90007740 con 0x7f0a980ad6e0 2024-04-24T05:46:50.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.768+0000 7f0a96000640 1 -- 172.21.15.123:0/2570216810 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a900078e0 con 0x7f0a980ad6e0 2024-04-24T05:46:50.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.768+0000 7f0a96000640 1 -- 172.21.15.123:0/2570216810 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90007cb0 con 0x7f0a980ad6e0 2024-04-24T05:46:50.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.769+0000 7f0a97e00640 1 -- 172.21.15.123:0/2570216810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a980ad6e0 msgr2=0x7f0a9806c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:50.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.769+0000 7f0a97e00640 1 --2- 172.21.15.123:0/2570216810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a980ad6e0 0x7f0a9806c1c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a9806beb0 tx=0x7f0a9002a5b0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.769+0000 7f0a97e00640 1 -- 172.21.15.123:0/2570216810 shutdown_connections 2024-04-24T05:46:50.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.769+0000 7f0a97e00640 1 --2- 172.21.15.123:0/2570216810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a980ad6e0 0x7f0a9806c1c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.769+0000 7f0a97e00640 1 --2- 172.21.15.123:0/2570216810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a9815c2f0 0x7f0a98141ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.769+0000 7f0a97e00640 1 -- 172.21.15.123:0/2570216810 >> 172.21.15.123:0/2570216810 conn(0x7f0a980b1cf0 msgr2=0x7f0a980b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:50.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.769+0000 7f0a97e00640 1 -- 172.21.15.123:0/2570216810 shutdown_connections 2024-04-24T05:46:50.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.769+0000 7f0a97e00640 1 -- 172.21.15.123:0/2570216810 wait complete. 2024-04-24T05:46:50.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.771+0000 7f0a97e00640 1 Processor -- start 2024-04-24T05:46:50.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.771+0000 7f0a97e00640 1 -- start start 2024-04-24T05:46:50.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a97e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a980ad6e0 0x7f0a980c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:50.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a97e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a980cc5d0 0x7f0a980c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:50.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a9815c2f0 0x7f0a981f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:50.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a97e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9815c6d0 con 0x7f0a9815c2f0 2024-04-24T05:46:50.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a97e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0a98142220 con 0x7f0a980ad6e0 2024-04-24T05:46:50.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a97e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9815c850 con 0x7f0a980cc5d0 2024-04-24T05:46:50.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a97400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a9815c2f0 0x7f0a981f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:50.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a97400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a9815c2f0 0x7f0a981f3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56454/0 (socket says 172.21.15.123:56454) 2024-04-24T05:46:50.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a97400640 1 -- 172.21.15.123:0/3311212009 learned_addr learned my addr 172.21.15.123:0/3311212009 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:50.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a96a00640 1 --2- 172.21.15.123:0/3311212009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a980ad6e0 0x7f0a980c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:50.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a8fe00640 1 --2- 172.21.15.123:0/3311212009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a980cc5d0 0x7f0a980c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:50.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a97400640 1 -- 172.21.15.123:0/3311212009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a980cc5d0 msgr2=0x7f0a980c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:50.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a97400640 1 --2- 172.21.15.123:0/3311212009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a980cc5d0 0x7f0a980c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a97400640 1 -- 172.21.15.123:0/3311212009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a980ad6e0 msgr2=0x7f0a980c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:50.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.772+0000 7f0a97400640 1 --2- 172.21.15.123:0/3311212009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a980ad6e0 0x7f0a980c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.773+0000 7f0a97400640 1 -- 172.21.15.123:0/3311212009 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a98071cb0 con 0x7f0a9815c2f0 2024-04-24T05:46:50.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.773+0000 7f0a97400640 1 --2- 172.21.15.123:0/3311212009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a9815c2f0 0x7f0a981f3c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0a9002aba0 tx=0x7f0a90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:50.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.773+0000 7f0a8ea00640 1 -- 172.21.15.123:0/3311212009 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90040070 con 0x7f0a9815c2f0 2024-04-24T05:46:50.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.773+0000 7f0a8ea00640 1 -- 172.21.15.123:0/3311212009 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a90031bc0 con 0x7f0a9815c2f0 2024-04-24T05:46:50.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.773+0000 7f0a97e00640 1 -- 172.21.15.123:0/3311212009 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a98071f90 con 0x7f0a9815c2f0 2024-04-24T05:46:50.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.773+0000 7f0a8ea00640 1 -- 172.21.15.123:0/3311212009 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90009450 con 0x7f0a9815c2f0 2024-04-24T05:46:50.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.773+0000 7f0a97e00640 1 -- 172.21.15.123:0/3311212009 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a980c98b0 con 0x7f0a9815c2f0 2024-04-24T05:46:50.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.774+0000 7f0a97e00640 1 -- 172.21.15.123:0/3311212009 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a98068a20 con 0x7f0a9815c2f0 2024-04-24T05:46:50.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.775+0000 7f0a8ea00640 1 -- 172.21.15.123:0/3311212009 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a90039040 con 0x7f0a9815c2f0 2024-04-24T05:46:50.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.776+0000 7f0a8ea00640 1 --2- 172.21.15.123:0/3311212009 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0a7406c8f0 0x7f0a7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:50.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.776+0000 7f0a8ea00640 1 -- 172.21.15.123:0/3311212009 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f0a900ad980 con 0x7f0a9815c2f0 2024-04-24T05:46:50.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.777+0000 7f0a96a00640 1 --2- 172.21.15.123:0/3311212009 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0a7406c8f0 0x7f0a7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:50.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.777+0000 7f0a96a00640 1 --2- 172.21.15.123:0/3311212009 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0a7406c8f0 0x7f0a7406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009a30 tx=0x7f0a88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:50.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.781+0000 7f0a8ea00640 1 -- 172.21.15.123:0/3311212009 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a9007ba30 con 0x7f0a9815c2f0 2024-04-24T05:46:50.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.981+0000 7f0a97e00640 1 -- 172.21.15.123:0/3311212009 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0a98160db0 con 0x7f0a9815c2f0 2024-04-24T05:46:50.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.982+0000 7f0a8ea00640 1 -- 172.21.15.123:0/3311212009 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f0a900abe20 con 0x7f0a9815c2f0 2024-04-24T05:46:50.983 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:50.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.984+0000 7f0a97e00640 1 -- 172.21.15.123:0/3311212009 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0a7406c8f0 msgr2=0x7f0a7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:50.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.984+0000 7f0a97e00640 1 --2- 172.21.15.123:0/3311212009 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0a7406c8f0 0x7f0a7406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009a30 tx=0x7f0a88008040 comp rx=0 tx=0).stop 2024-04-24T05:46:50.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.984+0000 7f0a97e00640 1 -- 172.21.15.123:0/3311212009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a9815c2f0 msgr2=0x7f0a981f3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:50.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.984+0000 7f0a97e00640 1 --2- 172.21.15.123:0/3311212009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a9815c2f0 0x7f0a981f3c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0a9002aba0 tx=0x7f0a90004a40 comp rx=0 tx=0).stop 2024-04-24T05:46:50.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.985+0000 7f0a97e00640 1 -- 172.21.15.123:0/3311212009 shutdown_connections 2024-04-24T05:46:50.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.985+0000 7f0a97e00640 1 --2- 172.21.15.123:0/3311212009 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0a7406c8f0 0x7f0a7406edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.985+0000 7f0a97e00640 1 --2- 172.21.15.123:0/3311212009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a9815c2f0 0x7f0a981f3c20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.985+0000 7f0a97e00640 1 --2- 172.21.15.123:0/3311212009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a980cc5d0 0x7f0a980c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.985+0000 7f0a97e00640 1 --2- 172.21.15.123:0/3311212009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a980ad6e0 0x7f0a980c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:50.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.985+0000 7f0a97e00640 1 -- 172.21.15.123:0/3311212009 >> 172.21.15.123:0/3311212009 conn(0x7f0a980b1cf0 msgr2=0x7f0a9815fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:50.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.985+0000 7f0a97e00640 1 -- 172.21.15.123:0/3311212009 shutdown_connections 2024-04-24T05:46:50.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:50.985+0000 7f0a97e00640 1 -- 172.21.15.123:0/3311212009 wait complete. 2024-04-24T05:46:51.004 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:46:51.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.202+0000 7f02d2200640 1 Processor -- start 2024-04-24T05:46:51.204 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.203+0000 7f02d2200640 1 -- start start 2024-04-24T05:46:51.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.204+0000 7f02d2200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f02cc06dcb0 0x7f02cc06e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:51.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.204+0000 7f02d2200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02cc06e960 0x7f02cc06ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:51.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.204+0000 7f02d2200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02cc06bb30 con 0x7f02cc158e00 2024-04-24T05:46:51.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.204+0000 7f02d2200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f02cc06f590 con 0x7f02cc06dcb0 2024-04-24T05:46:51.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.204+0000 7f02d2200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f02cc06c460 con 0x7f02cc06e960 2024-04-24T05:46:51.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.204+0000 7f02caa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f02cc06dcb0 0x7f02cc06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:51.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.205+0000 7f02caa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f02cc06dcb0 0x7f02cc06e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56468/0 (socket says 172.21.15.123:56468) 2024-04-24T05:46:51.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.205+0000 7f02caa00640 1 -- 172.21.15.123:0/1520787398 learned_addr learned my addr 172.21.15.123:0/1520787398 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:51.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.205+0000 7f02cbe00640 1 --2- 172.21.15.123:0/1520787398 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02cc06e960 0x7f02cc06ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:51.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.205+0000 7f02caa00640 1 -- 172.21.15.123:0/1520787398 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02cc06e960 msgr2=0x7f02cc06ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:51.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.205+0000 7f02caa00640 1 --2- 172.21.15.123:0/1520787398 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02cc06e960 0x7f02cc06ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.205+0000 7f02caa00640 1 -- 172.21.15.123:0/1520787398 >> v1:172.21.15.155:6789/0 conn(0x7f02cc158e00 legacy=0x7f02cc06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:51.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.205+0000 7f02caa00640 1 -- 172.21.15.123:0/1520787398 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02cc06be10 con 0x7f02cc06dcb0 2024-04-24T05:46:51.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.205+0000 7f02cbe00640 1 --2- 172.21.15.123:0/1520787398 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02cc06e960 0x7f02cc06ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:51.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.206+0000 7f02caa00640 1 --2- 172.21.15.123:0/1520787398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02cc06dcb0 0x7f02cc06e160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009870 tx=0x7f02b802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f652979b7c9cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:51.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.207+0000 7f02ca000640 1 -- 172.21.15.123:0/1520787398 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8037070 con 0x7f02cc06dcb0 2024-04-24T05:46:51.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.207+0000 7f02ca000640 1 -- 172.21.15.123:0/1520787398 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b8032440 con 0x7f02cc06dcb0 2024-04-24T05:46:51.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.207+0000 7f02ca000640 1 -- 172.21.15.123:0/1520787398 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8032810 con 0x7f02cc06dcb0 2024-04-24T05:46:51.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.207+0000 7f02d2200640 1 -- 172.21.15.123:0/1520787398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02cc06dcb0 msgr2=0x7f02cc06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:51.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.207+0000 7f02d2200640 1 --2- 172.21.15.123:0/1520787398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02cc06dcb0 0x7f02cc06e160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009870 tx=0x7f02b802a1c0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.207+0000 7f02d2200640 1 -- 172.21.15.123:0/1520787398 shutdown_connections 2024-04-24T05:46:51.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.207+0000 7f02d2200640 1 --2- 172.21.15.123:0/1520787398 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02cc06e960 0x7f02cc06ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.207+0000 7f02d2200640 1 --2- 172.21.15.123:0/1520787398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02cc06dcb0 0x7f02cc06e160 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.207+0000 7f02d2200640 1 -- 172.21.15.123:0/1520787398 >> 172.21.15.123:0/1520787398 conn(0x7f02cc0b51f0 msgr2=0x7f02cc0b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:51.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.208+0000 7f02d2200640 1 -- 172.21.15.123:0/1520787398 shutdown_connections 2024-04-24T05:46:51.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.208+0000 7f02d2200640 1 -- 172.21.15.123:0/1520787398 wait complete. 2024-04-24T05:46:51.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.209+0000 7f02d2200640 1 Processor -- start 2024-04-24T05:46:51.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.209+0000 7f02d2200640 1 -- start start 2024-04-24T05:46:51.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.210+0000 7f02d2200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02cc06dcb0 0x7f02cc0998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:51.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.210+0000 7f02d2200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02cc06e960 0x7f02cc09acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:51.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.210+0000 7f02d2200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02cc158e00 0x7f02cc09b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:51.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.210+0000 7f02d2200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc06d000 con 0x7f02cc158e00 2024-04-24T05:46:51.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.210+0000 7f02d2200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc06f2b0 con 0x7f02cc06e960 2024-04-24T05:46:51.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.210+0000 7f02d2200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f02cc1591e0 con 0x7f02cc06dcb0 2024-04-24T05:46:51.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.211+0000 7f02cbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02cc158e00 0x7f02cc09b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:51.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.211+0000 7f02cbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02cc158e00 0x7f02cc09b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56480/0 (socket says 172.21.15.123:56480) 2024-04-24T05:46:51.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.211+0000 7f02cbe00640 1 -- 172.21.15.123:0/2743328166 learned_addr learned my addr 172.21.15.123:0/2743328166 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:51.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.211+0000 7f02caa00640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02cc06e960 0x7f02cc09acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:51.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.211+0000 7f02cb400640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02cc06dcb0 0x7f02cc0998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:51.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.211+0000 7f02cbe00640 1 -- 172.21.15.123:0/2743328166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02cc06dcb0 msgr2=0x7f02cc0998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:51.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.211+0000 7f02cbe00640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02cc06dcb0 0x7f02cc0998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.211+0000 7f02cbe00640 1 -- 172.21.15.123:0/2743328166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02cc06e960 msgr2=0x7f02cc09acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:51.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.211+0000 7f02cbe00640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02cc06e960 0x7f02cc09acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.211+0000 7f02cbe00640 1 -- 172.21.15.123:0/2743328166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02cc09ce50 con 0x7f02cc158e00 2024-04-24T05:46:51.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.211+0000 7f02caa00640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02cc06e960 0x7f02cc09acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:51.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.212+0000 7f02cbe00640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02cc158e00 0x7f02cc09b230 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f02c400cd80 tx=0x7f02c400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:51.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.212+0000 7f02c3400640 1 -- 172.21.15.123:0/2743328166 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c400f680 con 0x7f02cc158e00 2024-04-24T05:46:51.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.212+0000 7f02c3400640 1 -- 172.21.15.123:0/2743328166 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c400f840 con 0x7f02cc158e00 2024-04-24T05:46:51.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.212+0000 7f02c3400640 1 -- 172.21.15.123:0/2743328166 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c400fc30 con 0x7f02cc158e00 2024-04-24T05:46:51.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.212+0000 7f02d2200640 1 -- 172.21.15.123:0/2743328166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02cc09d130 con 0x7f02cc158e00 2024-04-24T05:46:51.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.212+0000 7f02d2200640 1 -- 172.21.15.123:0/2743328166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02cc09d6f0 con 0x7f02cc158e00 2024-04-24T05:46:51.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.213+0000 7f02d2200640 1 -- 172.21.15.123:0/2743328166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02cc068a20 con 0x7f02cc158e00 2024-04-24T05:46:51.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.214+0000 7f02c3400640 1 -- 172.21.15.123:0/2743328166 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02c400e3e0 con 0x7f02cc158e00 2024-04-24T05:46:51.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.215+0000 7f02c3400640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f02ac06c8f0 0x7f02ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:51.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.215+0000 7f02c3400640 1 -- 172.21.15.123:0/2743328166 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f02c408ce40 con 0x7f02cc158e00 2024-04-24T05:46:51.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.215+0000 7f02cb400640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f02ac06c8f0 0x7f02ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:51.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.216+0000 7f02cb400640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f02ac06c8f0 0x7f02ac06edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f02bc000f60 tx=0x7f02bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:51.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.219+0000 7f02c3400640 1 -- 172.21.15.123:0/2743328166 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f02c4014030 con 0x7f02cc158e00 2024-04-24T05:46:51.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.421+0000 7f02d2200640 1 -- 172.21.15.123:0/2743328166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f02cc0d0d10 con 0x7f02cc158e00 2024-04-24T05:46:51.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.422+0000 7f02c3400640 1 -- 172.21.15.123:0/2743328166 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f02c405ad30 con 0x7f02cc158e00 2024-04-24T05:46:51.424 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:46:51.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.424+0000 7f02d2200640 1 -- 172.21.15.123:0/2743328166 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f02ac06c8f0 msgr2=0x7f02ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:51.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.424+0000 7f02d2200640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f02ac06c8f0 0x7f02ac06edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f02bc000f60 tx=0x7f02bc009040 comp rx=0 tx=0).stop 2024-04-24T05:46:51.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.424+0000 7f02d2200640 1 -- 172.21.15.123:0/2743328166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02cc158e00 msgr2=0x7f02cc09b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:51.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.424+0000 7f02d2200640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02cc158e00 0x7f02cc09b230 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f02c400cd80 tx=0x7f02c400a650 comp rx=0 tx=0).stop 2024-04-24T05:46:51.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.425+0000 7f02d2200640 1 -- 172.21.15.123:0/2743328166 shutdown_connections 2024-04-24T05:46:51.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.425+0000 7f02d2200640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f02ac06c8f0 0x7f02ac06edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.425+0000 7f02d2200640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02cc158e00 0x7f02cc09b230 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.425+0000 7f02d2200640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02cc06e960 0x7f02cc09acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.425+0000 7f02d2200640 1 --2- 172.21.15.123:0/2743328166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02cc06dcb0 0x7f02cc0998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.425+0000 7f02d2200640 1 -- 172.21.15.123:0/2743328166 >> 172.21.15.123:0/2743328166 conn(0x7f02cc0b51f0 msgr2=0x7f02cc0b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:51.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.425+0000 7f02d2200640 1 -- 172.21.15.123:0/2743328166 shutdown_connections 2024-04-24T05:46:51.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.425+0000 7f02d2200640 1 -- 172.21.15.123:0/2743328166 wait complete. 2024-04-24T05:46:51.444 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:46:51.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.635+0000 7fea40600640 1 Processor -- start 2024-04-24T05:46:51.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.635+0000 7fea40600640 1 -- start start 2024-04-24T05:46:51.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.637+0000 7fea40600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fea3806c510 0x7fea38071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:51.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.637+0000 7fea40600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fea380cd2c0 0x7fea3806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:51.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.637+0000 7fea40600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea380cf170 con 0x7fea3806c8f0 2024-04-24T05:46:51.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.637+0000 7fea40600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fea380cf450 con 0x7fea3806c510 2024-04-24T05:46:51.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.637+0000 7fea40600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fea380cf730 con 0x7fea380cd2c0 2024-04-24T05:46:51.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.637+0000 7fea3dc00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fea3806c510 0x7fea38071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:51.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.637+0000 7fea3dc00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fea3806c510 0x7fea38071c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56490/0 (socket says 172.21.15.123:56490) 2024-04-24T05:46:51.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.637+0000 7fea3dc00640 1 -- 172.21.15.123:0/3654651385 learned_addr learned my addr 172.21.15.123:0/3654651385 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:51.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.637+0000 7fea3dc00640 1 -- 172.21.15.123:0/3654651385 >> v2:172.21.15.155:3300/0 conn(0x7fea380cd2c0 msgr2=0x7fea3806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:51.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.637+0000 7fea3dc00640 1 --2- 172.21.15.123:0/3654651385 >> v2:172.21.15.155:3300/0 conn(0x7fea380cd2c0 0x7fea3806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.637+0000 7fea3dc00640 1 -- 172.21.15.123:0/3654651385 >> v1:172.21.15.155:6789/0 conn(0x7fea3806c8f0 legacy=0x7fea380ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:51.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.637+0000 7fea3dc00640 1 -- 172.21.15.123:0/3654651385 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea380cfa10 con 0x7fea3806c510 2024-04-24T05:46:51.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.638+0000 7fea3dc00640 1 --2- 172.21.15.123:0/3654651385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea3806c510 0x7fea38071c50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fea38071be0 tx=0x7fea2c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5a969723412722 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:51.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.639+0000 7fea37e00640 1 -- 172.21.15.123:0/3654651385 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c037070 con 0x7fea3806c510 2024-04-24T05:46:51.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.639+0000 7fea37e00640 1 -- 172.21.15.123:0/3654651385 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea2c032440 con 0x7fea3806c510 2024-04-24T05:46:51.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.639+0000 7fea37e00640 1 -- 172.21.15.123:0/3654651385 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c032810 con 0x7fea3806c510 2024-04-24T05:46:51.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.639+0000 7fea40600640 1 -- 172.21.15.123:0/3654651385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea3806c510 msgr2=0x7fea38071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:51.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.639+0000 7fea40600640 1 --2- 172.21.15.123:0/3654651385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea3806c510 0x7fea38071c50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fea38071be0 tx=0x7fea2c02a230 comp rx=0 tx=0).stop 2024-04-24T05:46:51.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.639+0000 7fea40600640 1 -- 172.21.15.123:0/3654651385 shutdown_connections 2024-04-24T05:46:51.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.639+0000 7fea40600640 1 --2- 172.21.15.123:0/3654651385 >> v2:172.21.15.155:3300/0 conn(0x7fea380cd2c0 0x7fea3806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.639+0000 7fea40600640 1 --2- 172.21.15.123:0/3654651385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea3806c510 0x7fea38071c50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.639+0000 7fea40600640 1 -- 172.21.15.123:0/3654651385 >> 172.21.15.123:0/3654651385 conn(0x7fea380b51f0 msgr2=0x7fea380b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:51.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.639+0000 7fea40600640 1 -- 172.21.15.123:0/3654651385 shutdown_connections 2024-04-24T05:46:51.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.639+0000 7fea40600640 1 -- 172.21.15.123:0/3654651385 wait complete. 2024-04-24T05:46:51.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.641+0000 7fea40600640 1 Processor -- start 2024-04-24T05:46:51.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.641+0000 7fea40600640 1 -- start start 2024-04-24T05:46:51.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.642+0000 7fea40600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3806c510 0x7fea381f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:51.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.642+0000 7fea40600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3806c8f0 0x7fea3815d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:51.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.642+0000 7fea40600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea380cd2c0 0x7fea380af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:51.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.642+0000 7fea40600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea380cee80 con 0x7fea380cd2c0 2024-04-24T05:46:51.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.642+0000 7fea40600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fea380ced00 con 0x7fea3806c8f0 2024-04-24T05:46:51.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.642+0000 7fea40600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fea380d0210 con 0x7fea3806c510 2024-04-24T05:46:51.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.642+0000 7fea3e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea380cd2c0 0x7fea380af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:51.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.642+0000 7fea3e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea380cd2c0 0x7fea380af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56506/0 (socket says 172.21.15.123:56506) 2024-04-24T05:46:51.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.642+0000 7fea3e600640 1 -- 172.21.15.123:0/1779985126 learned_addr learned my addr 172.21.15.123:0/1779985126 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:51.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.643+0000 7fea3e600640 1 -- 172.21.15.123:0/1779985126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3806c510 msgr2=0x7fea381f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:51.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.643+0000 7fea3e600640 1 --2- 172.21.15.123:0/1779985126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3806c510 0x7fea381f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.643+0000 7fea3e600640 1 -- 172.21.15.123:0/1779985126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3806c8f0 msgr2=0x7fea3815d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:51.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.643+0000 7fea3e600640 1 --2- 172.21.15.123:0/1779985126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3806c8f0 0x7fea3815d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.643+0000 7fea3e600640 1 -- 172.21.15.123:0/1779985126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea380c4e90 con 0x7fea380cd2c0 2024-04-24T05:46:51.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.643+0000 7fea3e600640 1 --2- 172.21.15.123:0/1779985126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea380cd2c0 0x7fea380af050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fea3000b410 tx=0x7fea3000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:51.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.643+0000 7fea35600640 1 -- 172.21.15.123:0/1779985126 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30015020 con 0x7fea380cd2c0 2024-04-24T05:46:51.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.643+0000 7fea35600640 1 -- 172.21.15.123:0/1779985126 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea30011830 con 0x7fea380cd2c0 2024-04-24T05:46:51.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.643+0000 7fea40600640 1 -- 172.21.15.123:0/1779985126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea380c5170 con 0x7fea380cd2c0 2024-04-24T05:46:51.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.643+0000 7fea35600640 1 -- 172.21.15.123:0/1779985126 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30011c20 con 0x7fea380cd2c0 2024-04-24T05:46:51.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.643+0000 7fea40600640 1 -- 172.21.15.123:0/1779985126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea380c5730 con 0x7fea380cd2c0 2024-04-24T05:46:51.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.644+0000 7fea40600640 1 -- 172.21.15.123:0/1779985126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea38068a20 con 0x7fea380cd2c0 2024-04-24T05:46:51.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.645+0000 7fea35600640 1 -- 172.21.15.123:0/1779985126 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea30018020 con 0x7fea380cd2c0 2024-04-24T05:46:51.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.646+0000 7fea35600640 1 --2- 172.21.15.123:0/1779985126 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fea1c06c8f0 0x7fea1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:51.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.646+0000 7fea35600640 1 -- 172.21.15.123:0/1779985126 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fea3008be60 con 0x7fea380cd2c0 2024-04-24T05:46:51.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.647+0000 7fea3dc00640 1 --2- 172.21.15.123:0/1779985126 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fea1c06c8f0 0x7fea1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:51.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.647+0000 7fea3dc00640 1 --2- 172.21.15.123:0/1779985126 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fea1c06c8f0 0x7fea1c06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fea2c034000 tx=0x7fea2c02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:51.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.651+0000 7fea35600640 1 -- 172.21.15.123:0/1779985126 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea3001f020 con 0x7fea380cd2c0 2024-04-24T05:46:51.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.853+0000 7fea40600640 1 -- 172.21.15.123:0/1779985126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fea380ccf80 con 0x7fea380cd2c0 2024-04-24T05:46:51.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.854+0000 7fea35600640 1 -- 172.21.15.123:0/1779985126 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fea300172c0 con 0x7fea380cd2c0 2024-04-24T05:46:51.855 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:51.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.856+0000 7fea40600640 1 -- 172.21.15.123:0/1779985126 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fea1c06c8f0 msgr2=0x7fea1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.856+0000 7fea40600640 1 --2- 172.21.15.123:0/1779985126 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fea1c06c8f0 0x7fea1c06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fea2c034000 tx=0x7fea2c02c000 comp rx=0 tx=0).stop 2024-04-24T05:46:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.856+0000 7fea40600640 1 -- 172.21.15.123:0/1779985126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea380cd2c0 msgr2=0x7fea380af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.856+0000 7fea40600640 1 --2- 172.21.15.123:0/1779985126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea380cd2c0 0x7fea380af050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fea3000b410 tx=0x7fea3000b810 comp rx=0 tx=0).stop 2024-04-24T05:46:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.857+0000 7fea40600640 1 -- 172.21.15.123:0/1779985126 shutdown_connections 2024-04-24T05:46:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.857+0000 7fea40600640 1 --2- 172.21.15.123:0/1779985126 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fea1c06c8f0 0x7fea1c06edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.857+0000 7fea40600640 1 --2- 172.21.15.123:0/1779985126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea380cd2c0 0x7fea380af050 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.857+0000 7fea40600640 1 --2- 172.21.15.123:0/1779985126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3806c8f0 0x7fea3815d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.857+0000 7fea40600640 1 --2- 172.21.15.123:0/1779985126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3806c510 0x7fea381f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.857+0000 7fea40600640 1 -- 172.21.15.123:0/1779985126 >> 172.21.15.123:0/1779985126 conn(0x7fea380b51f0 msgr2=0x7fea380b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.857+0000 7fea40600640 1 -- 172.21.15.123:0/1779985126 shutdown_connections 2024-04-24T05:46:51.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:51.857+0000 7fea40600640 1 -- 172.21.15.123:0/1779985126 wait complete. 2024-04-24T05:46:51.876 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:52.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.060+0000 7f97f9c00640 1 Processor -- start 2024-04-24T05:46:52.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.060+0000 7f97f9c00640 1 -- start start 2024-04-24T05:46:52.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.062+0000 7f97f9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97f406f680 0x7f97f415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.062+0000 7f97f9c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97f406ba00 0x7f97f4071c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.062+0000 7f97f9c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f97f4072170 0x7f97f415b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.062+0000 7f97f9c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f97f406be80 con 0x7f97f406ba00 2024-04-24T05:46:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.062+0000 7f97f9c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f97f40725a0 con 0x7f97f4072170 2024-04-24T05:46:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.062+0000 7f97f9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f97f406c0e0 con 0x7f97f406f680 2024-04-24T05:46:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.062+0000 7f97f2000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97f406ba00 0x7f97f4071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.062+0000 7f97f2000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97f406ba00 0x7f97f4071c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56522/0 (socket says 172.21.15.123:56522) 2024-04-24T05:46:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.062+0000 7f97f2000640 1 -- 172.21.15.123:0/1479597525 learned_addr learned my addr 172.21.15.123:0/1479597525 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.062+0000 7f97f3400640 1 --2- 172.21.15.123:0/1479597525 >> v2:172.21.15.155:3300/0 conn(0x7f97f4072170 0x7f97f415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.062+0000 7f97f2a00640 1 --2- 172.21.15.123:0/1479597525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97f406f680 0x7f97f415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.063+0000 7f97f2000640 1 -- 172.21.15.123:0/1479597525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97f406f680 msgr2=0x7f97f415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:52.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.063+0000 7f97f2000640 1 --2- 172.21.15.123:0/1479597525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97f406f680 0x7f97f415ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.063+0000 7f97f2000640 1 -- 172.21.15.123:0/1479597525 >> v2:172.21.15.155:3300/0 conn(0x7f97f4072170 msgr2=0x7f97f415b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:52.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.063+0000 7f97f2000640 1 --2- 172.21.15.123:0/1479597525 >> v2:172.21.15.155:3300/0 conn(0x7f97f4072170 0x7f97f415b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.063+0000 7f97f2000640 1 -- 172.21.15.123:0/1479597525 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f406c3c0 con 0x7f97f406ba00 2024-04-24T05:46:52.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.064+0000 7f97f2000640 1 --2- 172.21.15.123:0/1479597525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f406ba00 0x7f97f4071c30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f97f40b2e90 tx=0x7f97e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395a6f4cdb195163 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:52.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.064+0000 7f97f1600640 1 -- 172.21.15.123:0/1479597525 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0037070 con 0x7f97f406ba00 2024-04-24T05:46:52.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.064+0000 7f97f1600640 1 -- 172.21.15.123:0/1479597525 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e0032440 con 0x7f97f406ba00 2024-04-24T05:46:52.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.064+0000 7f97f1600640 1 -- 172.21.15.123:0/1479597525 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0032810 con 0x7f97f406ba00 2024-04-24T05:46:52.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.064+0000 7f97f9c00640 1 -- 172.21.15.123:0/1479597525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f406ba00 msgr2=0x7f97f4071c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:52.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.064+0000 7f97f9c00640 1 --2- 172.21.15.123:0/1479597525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f406ba00 0x7f97f4071c30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f97f40b2e90 tx=0x7f97e002a180 comp rx=0 tx=0).stop 2024-04-24T05:46:52.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.065+0000 7f97f9c00640 1 -- 172.21.15.123:0/1479597525 shutdown_connections 2024-04-24T05:46:52.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.065+0000 7f97f9c00640 1 --2- 172.21.15.123:0/1479597525 >> v2:172.21.15.155:3300/0 conn(0x7f97f4072170 0x7f97f415b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.065+0000 7f97f9c00640 1 --2- 172.21.15.123:0/1479597525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f406ba00 0x7f97f4071c30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.065+0000 7f97f9c00640 1 --2- 172.21.15.123:0/1479597525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97f406f680 0x7f97f415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.065+0000 7f97f9c00640 1 -- 172.21.15.123:0/1479597525 >> 172.21.15.123:0/1479597525 conn(0x7f97f40b1e70 msgr2=0x7f97f40b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:52.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.065+0000 7f97f9c00640 1 -- 172.21.15.123:0/1479597525 shutdown_connections 2024-04-24T05:46:52.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.065+0000 7f97f9c00640 1 -- 172.21.15.123:0/1479597525 wait complete. 2024-04-24T05:46:52.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.067+0000 7f97f9c00640 1 Processor -- start 2024-04-24T05:46:52.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.067+0000 7f97f9c00640 1 -- start start 2024-04-24T05:46:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.068+0000 7f97f9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97f406ba00 0x7f97f414d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.068+0000 7f97f9c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f406f680 0x7f97f41e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.068+0000 7f97f9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97f4072170 0x7f97f41f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.068+0000 7f97f9c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97f406cc60 con 0x7f97f406f680 2024-04-24T05:46:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.068+0000 7f97f9c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f97f406c980 con 0x7f97f406ba00 2024-04-24T05:46:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.068+0000 7f97f9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f97f406fa60 con 0x7f97f4072170 2024-04-24T05:46:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.068+0000 7f97f2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f406f680 0x7f97f41e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.068+0000 7f97f2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f406f680 0x7f97f41e8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56530/0 (socket says 172.21.15.123:56530) 2024-04-24T05:46:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.068+0000 7f97f2000640 1 -- 172.21.15.123:0/3325293843 learned_addr learned my addr 172.21.15.123:0/3325293843 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.068+0000 7f97f2a00640 1 --2- 172.21.15.123:0/3325293843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97f406ba00 0x7f97f414d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.068+0000 7f97f3400640 1 --2- 172.21.15.123:0/3325293843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97f4072170 0x7f97f41f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.069+0000 7f97f2000640 1 -- 172.21.15.123:0/3325293843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97f4072170 msgr2=0x7f97f41f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:52.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.069+0000 7f97f2000640 1 --2- 172.21.15.123:0/3325293843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97f4072170 0x7f97f41f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.069+0000 7f97f2000640 1 -- 172.21.15.123:0/3325293843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97f406ba00 msgr2=0x7f97f414d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:52.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.069+0000 7f97f2000640 1 --2- 172.21.15.123:0/3325293843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97f406ba00 0x7f97f414d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.069+0000 7f97f2000640 1 -- 172.21.15.123:0/3325293843 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f4161aa0 con 0x7f97f406f680 2024-04-24T05:46:52.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.069+0000 7f97f2000640 1 --2- 172.21.15.123:0/3325293843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f406f680 0x7f97f41e8b80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f97e002a1c0 tx=0x7f97e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:52.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.070+0000 7f97eaa00640 1 -- 172.21.15.123:0/3325293843 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0041020 con 0x7f97f406f680 2024-04-24T05:46:52.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.070+0000 7f97eaa00640 1 -- 172.21.15.123:0/3325293843 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e00306c0 con 0x7f97f406f680 2024-04-24T05:46:52.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.070+0000 7f97eaa00640 1 -- 172.21.15.123:0/3325293843 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0037040 con 0x7f97f406f680 2024-04-24T05:46:52.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.070+0000 7f97f9c00640 1 -- 172.21.15.123:0/3325293843 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f4161d80 con 0x7f97f406f680 2024-04-24T05:46:52.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.070+0000 7f97f9c00640 1 -- 172.21.15.123:0/3325293843 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f4162340 con 0x7f97f406f680 2024-04-24T05:46:52.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.071+0000 7f97f9c00640 1 -- 172.21.15.123:0/3325293843 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f4068a20 con 0x7f97f406f680 2024-04-24T05:46:52.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.072+0000 7f97eaa00640 1 -- 172.21.15.123:0/3325293843 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97e0044020 con 0x7f97f406f680 2024-04-24T05:46:52.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.073+0000 7f97eaa00640 1 --2- 172.21.15.123:0/3325293843 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f97d406c8f0 0x7f97d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.073+0000 7f97eaa00640 1 -- 172.21.15.123:0/3325293843 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f97e00ac660 con 0x7f97f406f680 2024-04-24T05:46:52.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.073+0000 7f97f2a00640 1 --2- 172.21.15.123:0/3325293843 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f97d406c8f0 0x7f97d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.074+0000 7f97f2a00640 1 --2- 172.21.15.123:0/3325293843 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f97d406c8f0 0x7f97d406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97e4006fd0 tx=0x7f97e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:52.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.077+0000 7f97eaa00640 1 -- 172.21.15.123:0/3325293843 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97e004b030 con 0x7f97f406f680 2024-04-24T05:46:52.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.319+0000 7f97f9c00640 1 -- 172.21.15.123:0/3325293843 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97f4071f50 con 0x7f97f406f680 2024-04-24T05:46:52.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.320+0000 7f97eaa00640 1 -- 172.21.15.123:0/3325293843 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7f97e007a030 con 0x7f97f406f680 2024-04-24T05:46:52.322 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:52.322 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:52.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.323+0000 7f97f9c00640 1 -- 172.21.15.123:0/3325293843 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f97d406c8f0 msgr2=0x7f97d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:52.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.323+0000 7f97f9c00640 1 --2- 172.21.15.123:0/3325293843 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f97d406c8f0 0x7f97d406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97e4006fd0 tx=0x7f97e4008040 comp rx=0 tx=0).stop 2024-04-24T05:46:52.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.323+0000 7f97f9c00640 1 -- 172.21.15.123:0/3325293843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f406f680 msgr2=0x7f97f41e8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:52.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.323+0000 7f97f9c00640 1 --2- 172.21.15.123:0/3325293843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f406f680 0x7f97f41e8b80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f97e002a1c0 tx=0x7f97e00046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.323+0000 7f97f9c00640 1 -- 172.21.15.123:0/3325293843 shutdown_connections 2024-04-24T05:46:52.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.323+0000 7f97f9c00640 1 --2- 172.21.15.123:0/3325293843 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f97d406c8f0 0x7f97d406edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.323+0000 7f97f9c00640 1 --2- 172.21.15.123:0/3325293843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97f4072170 0x7f97f41f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.323+0000 7f97f9c00640 1 --2- 172.21.15.123:0/3325293843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97f406f680 0x7f97f41e8b80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.323+0000 7f97f9c00640 1 --2- 172.21.15.123:0/3325293843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97f406ba00 0x7f97f414d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.323+0000 7f97f9c00640 1 -- 172.21.15.123:0/3325293843 >> 172.21.15.123:0/3325293843 conn(0x7f97f40b1e70 msgr2=0x7f97f415cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:52.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.324+0000 7f97f9c00640 1 -- 172.21.15.123:0/3325293843 shutdown_connections 2024-04-24T05:46:52.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.324+0000 7f97f9c00640 1 -- 172.21.15.123:0/3325293843 wait complete. 2024-04-24T05:46:52.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:46:52.344 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:46:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.543+0000 7feef7200640 1 Processor -- start 2024-04-24T05:46:52.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.543+0000 7feef7200640 1 -- start start 2024-04-24T05:46:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.545+0000 7feef7200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feef8155480 0x7feef8072c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.545+0000 7feef7200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feef815f660 0x7feef80731a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.545+0000 7feef7200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef80ad850 con 0x7feef8155860 2024-04-24T05:46:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.545+0000 7feef7200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7feef80ae8d0 con 0x7feef8155480 2024-04-24T05:46:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.545+0000 7feef7200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7feef80adb10 con 0x7feef815f660 2024-04-24T05:46:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.545+0000 7feef5e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feef8155480 0x7feef8072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.545+0000 7feef5e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feef8155480 0x7feef8072c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56538/0 (socket says 172.21.15.123:56538) 2024-04-24T05:46:52.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.545+0000 7feef5e00640 1 -- 172.21.15.123:0/3906187452 learned_addr learned my addr 172.21.15.123:0/3906187452 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:52.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.546+0000 7feef5e00640 1 -- 172.21.15.123:0/3906187452 >> v2:172.21.15.155:3300/0 conn(0x7feef815f660 msgr2=0x7feef80731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:52.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.546+0000 7feef5e00640 1 --2- 172.21.15.123:0/3906187452 >> v2:172.21.15.155:3300/0 conn(0x7feef815f660 0x7feef80731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.546+0000 7feef5e00640 1 -- 172.21.15.123:0/3906187452 >> v1:172.21.15.155:6789/0 conn(0x7feef8155860 legacy=0x7feef815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:52.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.546+0000 7feef5e00640 1 -- 172.21.15.123:0/3906187452 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef80addf0 con 0x7feef8155480 2024-04-24T05:46:52.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.547+0000 7feef5e00640 1 --2- 172.21.15.123:0/3906187452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef8155480 0x7feef8072c60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feef80b6350 tx=0x7feee802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aba354843183fff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:52.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.547+0000 7feef4a00640 1 -- 172.21.15.123:0/3906187452 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8036070 con 0x7feef8155480 2024-04-24T05:46:52.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.547+0000 7feef4a00640 1 -- 172.21.15.123:0/3906187452 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee8031600 con 0x7feef8155480 2024-04-24T05:46:52.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.547+0000 7feef4a00640 1 -- 172.21.15.123:0/3906187452 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee80319d0 con 0x7feef8155480 2024-04-24T05:46:52.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.548+0000 7feef7200640 1 -- 172.21.15.123:0/3906187452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef8155480 msgr2=0x7feef8072c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:52.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.548+0000 7feef7200640 1 --2- 172.21.15.123:0/3906187452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef8155480 0x7feef8072c60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feef80b6350 tx=0x7feee802a6f0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.548+0000 7feef7200640 1 -- 172.21.15.123:0/3906187452 shutdown_connections 2024-04-24T05:46:52.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.548+0000 7feef7200640 1 --2- 172.21.15.123:0/3906187452 >> v2:172.21.15.155:3300/0 conn(0x7feef815f660 0x7feef80731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.548+0000 7feef7200640 1 --2- 172.21.15.123:0/3906187452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef8155480 0x7feef8072c60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.548+0000 7feef7200640 1 -- 172.21.15.123:0/3906187452 >> 172.21.15.123:0/3906187452 conn(0x7feef80b2ef0 msgr2=0x7feef80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:52.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.548+0000 7feef7200640 1 -- 172.21.15.123:0/3906187452 shutdown_connections 2024-04-24T05:46:52.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.548+0000 7feef7200640 1 -- 172.21.15.123:0/3906187452 wait complete. 2024-04-24T05:46:52.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.550+0000 7feef7200640 1 Processor -- start 2024-04-24T05:46:52.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.550+0000 7feef7200640 1 -- start start 2024-04-24T05:46:52.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.551+0000 7feef7200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feef8155480 0x7feef813ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.551+0000 7feef7200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feef8155860 0x7feef8150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.551+0000 7feef7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef815f660 0x7feef81527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.551+0000 7feef7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feef80af4b0 con 0x7feef815f660 2024-04-24T05:46:52.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.551+0000 7feef7200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feef80ae3b0 con 0x7feef8155860 2024-04-24T05:46:52.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.551+0000 7feef7200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feef80af630 con 0x7feef8155480 2024-04-24T05:46:52.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.551+0000 7feef6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef815f660 0x7feef81527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.551+0000 7feef6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef815f660 0x7feef81527a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56544/0 (socket says 172.21.15.123:56544) 2024-04-24T05:46:52.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.551+0000 7feef6800640 1 -- 172.21.15.123:0/2577484190 learned_addr learned my addr 172.21.15.123:0/2577484190 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:52.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.551+0000 7feef5e00640 1 --2- 172.21.15.123:0/2577484190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feef8155480 0x7feef813ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.551+0000 7feef5400640 1 --2- 172.21.15.123:0/2577484190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feef8155860 0x7feef8150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.552+0000 7feef6800640 1 -- 172.21.15.123:0/2577484190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feef8155480 msgr2=0x7feef813ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:52.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.552+0000 7feef6800640 1 --2- 172.21.15.123:0/2577484190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feef8155480 0x7feef813ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.552+0000 7feef6800640 1 -- 172.21.15.123:0/2577484190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feef8155860 msgr2=0x7feef8150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:52.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.552+0000 7feef6800640 1 --2- 172.21.15.123:0/2577484190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feef8155860 0x7feef8150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.552+0000 7feef6800640 1 -- 172.21.15.123:0/2577484190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef813f140 con 0x7feef815f660 2024-04-24T05:46:52.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.552+0000 7feef6800640 1 --2- 172.21.15.123:0/2577484190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef815f660 0x7feef81527a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feef000b410 tx=0x7feef000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:52.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.552+0000 7feeee000640 1 -- 172.21.15.123:0/2577484190 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0015020 con 0x7feef815f660 2024-04-24T05:46:52.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.552+0000 7feeee000640 1 -- 172.21.15.123:0/2577484190 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef0011850 con 0x7feef815f660 2024-04-24T05:46:52.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.552+0000 7feeee000640 1 -- 172.21.15.123:0/2577484190 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0011c40 con 0x7feef815f660 2024-04-24T05:46:52.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.552+0000 7feef7200640 1 -- 172.21.15.123:0/2577484190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef8151700 con 0x7feef815f660 2024-04-24T05:46:52.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.552+0000 7feef7200640 1 -- 172.21.15.123:0/2577484190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef8151cc0 con 0x7feef815f660 2024-04-24T05:46:52.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.553+0000 7feef7200640 1 -- 172.21.15.123:0/2577484190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef806f310 con 0x7feef815f660 2024-04-24T05:46:52.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.554+0000 7feeee000640 1 -- 172.21.15.123:0/2577484190 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feef0018020 con 0x7feef815f660 2024-04-24T05:46:52.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.556+0000 7feeee000640 1 --2- 172.21.15.123:0/2577484190 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feed406c8f0 0x7feed406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.556+0000 7feeee000640 1 -- 172.21.15.123:0/2577484190 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7feef008bef0 con 0x7feef815f660 2024-04-24T05:46:52.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.556+0000 7feef5e00640 1 --2- 172.21.15.123:0/2577484190 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feed406c8f0 0x7feed406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.557+0000 7feef5e00640 1 --2- 172.21.15.123:0/2577484190 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feed406c8f0 0x7feed406edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feee8033000 tx=0x7feee8009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:52.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.560+0000 7feeee000640 1 -- 172.21.15.123:0/2577484190 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feef001f020 con 0x7feef815f660 2024-04-24T05:46:52.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.761+0000 7feef7200640 1 -- 172.21.15.123:0/2577484190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feef8074e70 con 0x7feef815f660 2024-04-24T05:46:52.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.762+0000 7feeee000640 1 -- 172.21.15.123:0/2577484190 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7feef813f140 con 0x7feef815f660 2024-04-24T05:46:52.763 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:46:52.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.765+0000 7feef7200640 1 -- 172.21.15.123:0/2577484190 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feed406c8f0 msgr2=0x7feed406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:52.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.765+0000 7feef7200640 1 --2- 172.21.15.123:0/2577484190 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feed406c8f0 0x7feed406edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feee8033000 tx=0x7feee8009000 comp rx=0 tx=0).stop 2024-04-24T05:46:52.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.765+0000 7feef7200640 1 -- 172.21.15.123:0/2577484190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef815f660 msgr2=0x7feef81527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:52.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.765+0000 7feef7200640 1 --2- 172.21.15.123:0/2577484190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef815f660 0x7feef81527a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feef000b410 tx=0x7feef000b8d0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.765+0000 7feef7200640 1 -- 172.21.15.123:0/2577484190 shutdown_connections 2024-04-24T05:46:52.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.765+0000 7feef7200640 1 --2- 172.21.15.123:0/2577484190 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feed406c8f0 0x7feed406edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.765+0000 7feef7200640 1 --2- 172.21.15.123:0/2577484190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef815f660 0x7feef81527a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.765+0000 7feef7200640 1 --2- 172.21.15.123:0/2577484190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feef8155860 0x7feef8150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.765+0000 7feef7200640 1 --2- 172.21.15.123:0/2577484190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feef8155480 0x7feef813ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.765+0000 7feef7200640 1 -- 172.21.15.123:0/2577484190 >> 172.21.15.123:0/2577484190 conn(0x7feef80b2ef0 msgr2=0x7feef80b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:52.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.765+0000 7feef7200640 1 -- 172.21.15.123:0/2577484190 shutdown_connections 2024-04-24T05:46:52.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.765+0000 7feef7200640 1 -- 172.21.15.123:0/2577484190 wait complete. 2024-04-24T05:46:52.785 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:46:52.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.987+0000 7f4d07000640 1 Processor -- start 2024-04-24T05:46:52.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.987+0000 7f4d07000640 1 -- start start 2024-04-24T05:46:52.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.988+0000 7f4d07000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4d000ae990 0x7f4d000d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.988+0000 7f4d07000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4d000ce7e0 0x7f4d0006e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.988+0000 7f4d07000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d000d0740 con 0x7f4d0006c410 2024-04-24T05:46:52.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.988+0000 7f4d07000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4d0006dcb0 con 0x7f4d000ce7e0 2024-04-24T05:46:52.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.988+0000 7f4d07000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4d0006e270 con 0x7f4d000ae990 2024-04-24T05:46:52.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.989+0000 7f4d05000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4d000ce7e0 0x7f4d0006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.989+0000 7f4d05000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4d000ce7e0 0x7f4d0006e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56552/0 (socket says 172.21.15.123:56552) 2024-04-24T05:46:52.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.989+0000 7f4d05000640 1 -- 172.21.15.123:0/3926775184 learned_addr learned my addr 172.21.15.123:0/3926775184 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:52.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.989+0000 7f4d05000640 1 -- 172.21.15.123:0/3926775184 >> v2:172.21.15.155:3300/0 conn(0x7f4d000ae990 msgr2=0x7f4d000d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:52.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.989+0000 7f4cffe00640 1 --2- 172.21.15.123:0/3926775184 >> v2:172.21.15.155:3300/0 conn(0x7f4d000ae990 0x7f4d000d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.989+0000 7f4d05000640 1 --2- 172.21.15.123:0/3926775184 >> v2:172.21.15.155:3300/0 conn(0x7f4d000ae990 0x7f4d000d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.989+0000 7f4d05000640 1 -- 172.21.15.123:0/3926775184 >> v1:172.21.15.155:6789/0 conn(0x7f4d0006c410 legacy=0x7f4d000ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:52.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.989+0000 7f4d05000640 1 -- 172.21.15.123:0/3926775184 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d000d0a20 con 0x7f4d000ce7e0 2024-04-24T05:46:52.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.989+0000 7f4cffe00640 1 --2- 172.21.15.123:0/3926775184 >> v2:172.21.15.155:3300/0 conn(0x7f4d000ae990 0x7f4d000d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:52.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.990+0000 7f4d05000640 1 --2- 172.21.15.123:0/3926775184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d000ce7e0 0x7f4d0006e700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4d0006ff40 tx=0x7f4cf802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1370479016bf0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:52.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.991+0000 7f4cfea00640 1 -- 172.21.15.123:0/3926775184 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8036070 con 0x7f4d000ce7e0 2024-04-24T05:46:52.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.991+0000 7f4cfea00640 1 -- 172.21.15.123:0/3926775184 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf8031600 con 0x7f4d000ce7e0 2024-04-24T05:46:52.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.991+0000 7f4cfea00640 1 -- 172.21.15.123:0/3926775184 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf80319d0 con 0x7f4d000ce7e0 2024-04-24T05:46:52.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.991+0000 7f4d07000640 1 -- 172.21.15.123:0/3926775184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d000ce7e0 msgr2=0x7f4d0006e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:52.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.991+0000 7f4d07000640 1 --2- 172.21.15.123:0/3926775184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d000ce7e0 0x7f4d0006e700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4d0006ff40 tx=0x7f4cf802a6f0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.992+0000 7f4d07000640 1 -- 172.21.15.123:0/3926775184 shutdown_connections 2024-04-24T05:46:52.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.992+0000 7f4d07000640 1 --2- 172.21.15.123:0/3926775184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d000ce7e0 0x7f4d0006e700 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.992+0000 7f4d07000640 1 --2- 172.21.15.123:0/3926775184 >> v2:172.21.15.155:3300/0 conn(0x7f4d000ae990 0x7f4d000d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.992+0000 7f4d07000640 1 -- 172.21.15.123:0/3926775184 >> 172.21.15.123:0/3926775184 conn(0x7f4d000b51f0 msgr2=0x7f4d000b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:52.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.992+0000 7f4d07000640 1 -- 172.21.15.123:0/3926775184 shutdown_connections 2024-04-24T05:46:52.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.992+0000 7f4d07000640 1 -- 172.21.15.123:0/3926775184 wait complete. 2024-04-24T05:46:52.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.993+0000 7f4d07000640 1 Processor -- start 2024-04-24T05:46:52.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.994+0000 7f4d07000640 1 -- start start 2024-04-24T05:46:52.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.994+0000 7f4d07000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d0006c410 0x7f4d00072050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.994+0000 7f4d07000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d000ae990 0x7f4d0015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.995+0000 7f4d07000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d000ce7e0 0x7f4d000b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:52.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.995+0000 7f4d07000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0006c870 con 0x7f4d0006c410 2024-04-24T05:46:52.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.995+0000 7f4d07000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4d000aed70 con 0x7f4d000ae990 2024-04-24T05:46:52.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.995+0000 7f4d07000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4d000b09b0 con 0x7f4d000ce7e0 2024-04-24T05:46:52.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.995+0000 7f4cffe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d0006c410 0x7f4d00072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.995+0000 7f4cffe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d0006c410 0x7f4d00072050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56556/0 (socket says 172.21.15.123:56556) 2024-04-24T05:46:52.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.995+0000 7f4cffe00640 1 -- 172.21.15.123:0/2204428524 learned_addr learned my addr 172.21.15.123:0/2204428524 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:52.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.995+0000 7f4cff400640 1 --2- 172.21.15.123:0/2204428524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d000ae990 0x7f4d0015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:52.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.995+0000 7f4cffe00640 1 -- 172.21.15.123:0/2204428524 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d000ce7e0 msgr2=0x7f4d000b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:52.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.995+0000 7f4cffe00640 1 --2- 172.21.15.123:0/2204428524 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d000ce7e0 0x7f4d000b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.995+0000 7f4cffe00640 1 -- 172.21.15.123:0/2204428524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d000ae990 msgr2=0x7f4d0015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:52.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.995+0000 7f4cffe00640 1 --2- 172.21.15.123:0/2204428524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d000ae990 0x7f4d0015ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:52.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.995+0000 7f4cffe00640 1 -- 172.21.15.123:0/2204428524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d000727a0 con 0x7f4d0006c410 2024-04-24T05:46:52.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.996+0000 7f4cffe00640 1 --2- 172.21.15.123:0/2204428524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d0006c410 0x7f4d00072050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000c820 tx=0x7f4cf000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:52.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.996+0000 7f4cf7e00640 1 -- 172.21.15.123:0/2204428524 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0015070 con 0x7f4d0006c410 2024-04-24T05:46:52.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.996+0000 7f4cf7e00640 1 -- 172.21.15.123:0/2204428524 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf00079d0 con 0x7f4d0006c410 2024-04-24T05:46:52.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.996+0000 7f4cf7e00640 1 -- 172.21.15.123:0/2204428524 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0007da0 con 0x7f4d0006c410 2024-04-24T05:46:52.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.996+0000 7f4d07000640 1 -- 172.21.15.123:0/2204428524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d00072a80 con 0x7f4d0006c410 2024-04-24T05:46:52.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.996+0000 7f4d07000640 1 -- 172.21.15.123:0/2204428524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d00073040 con 0x7f4d0006c410 2024-04-24T05:46:52.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.997+0000 7f4d07000640 1 -- 172.21.15.123:0/2204428524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d00068a20 con 0x7f4d0006c410 2024-04-24T05:46:52.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.998+0000 7f4cf7e00640 1 -- 172.21.15.123:0/2204428524 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4cf0018070 con 0x7f4d0006c410 2024-04-24T05:46:53.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.999+0000 7f4cf7e00640 1 --2- 172.21.15.123:0/2204428524 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ce006c8f0 0x7f4ce006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:53.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.999+0000 7f4cf7e00640 1 -- 172.21.15.123:0/2204428524 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f4cf008bdf0 con 0x7f4d0006c410 2024-04-24T05:46:53.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:52.999+0000 7f4cff400640 1 --2- 172.21.15.123:0/2204428524 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ce006c8f0 0x7f4ce006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:53.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.000+0000 7f4cff400640 1 --2- 172.21.15.123:0/2204428524 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ce006c8f0 0x7f4ce006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4d0009a020 tx=0x7f4cec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:53.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.003+0000 7f4cf7e00640 1 -- 172.21.15.123:0/2204428524 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4cf0059ea0 con 0x7f4d0006c410 2024-04-24T05:46:53.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.206+0000 7f4d07000640 1 -- 172.21.15.123:0/2204428524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4d0015b170 con 0x7f4d0006c410 2024-04-24T05:46:53.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.207+0000 7f4cf7e00640 1 -- 172.21.15.123:0/2204428524 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f4cf00597a0 con 0x7f4d0006c410 2024-04-24T05:46:53.208 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.209+0000 7f4d07000640 1 -- 172.21.15.123:0/2204428524 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ce006c8f0 msgr2=0x7f4ce006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.209+0000 7f4d07000640 1 --2- 172.21.15.123:0/2204428524 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ce006c8f0 0x7f4ce006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4d0009a020 tx=0x7f4cec009040 comp rx=0 tx=0).stop 2024-04-24T05:46:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.209+0000 7f4d07000640 1 -- 172.21.15.123:0/2204428524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d0006c410 msgr2=0x7f4d00072050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.209+0000 7f4d07000640 1 --2- 172.21.15.123:0/2204428524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d0006c410 0x7f4d00072050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000c820 tx=0x7f4cf000cc20 comp rx=0 tx=0).stop 2024-04-24T05:46:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.209+0000 7f4d07000640 1 -- 172.21.15.123:0/2204428524 shutdown_connections 2024-04-24T05:46:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.209+0000 7f4d07000640 1 --2- 172.21.15.123:0/2204428524 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ce006c8f0 0x7f4ce006edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.209+0000 7f4d07000640 1 --2- 172.21.15.123:0/2204428524 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d000ce7e0 0x7f4d000b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.209+0000 7f4d07000640 1 --2- 172.21.15.123:0/2204428524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d000ae990 0x7f4d0015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.209+0000 7f4d07000640 1 --2- 172.21.15.123:0/2204428524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d0006c410 0x7f4d00072050 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.209+0000 7f4d07000640 1 -- 172.21.15.123:0/2204428524 >> 172.21.15.123:0/2204428524 conn(0x7f4d000b51f0 msgr2=0x7f4d000ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.210+0000 7f4d07000640 1 -- 172.21.15.123:0/2204428524 shutdown_connections 2024-04-24T05:46:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.210+0000 7f4d07000640 1 -- 172.21.15.123:0/2204428524 wait complete. 2024-04-24T05:46:53.230 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:46:53.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.434+0000 7f871d200640 1 Processor -- start 2024-04-24T05:46:53.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.434+0000 7f871d200640 1 -- start start 2024-04-24T05:46:53.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.435+0000 7f871d200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f871806c510 0x7f8718071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:53.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.435+0000 7f871d200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f871806c8f0 0x7f871806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:53.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.435+0000 7f871d200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87180cf170 con 0x7f87180ccbb0 2024-04-24T05:46:53.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.435+0000 7f871d200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f87180cf450 con 0x7f871806c510 2024-04-24T05:46:53.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.435+0000 7f871d200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f87180cf730 con 0x7f871806c8f0 2024-04-24T05:46:53.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.436+0000 7f8715e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f871806c510 0x7f8718071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:53.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.436+0000 7f8716800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f87180ccbb0 0x7f87180d0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:44004/0 (socket says 172.21.15.123:44004) 2024-04-24T05:46:53.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.436+0000 7f8716800640 1 -- 172.21.15.123:0/1912958861 learned_addr learned my addr 172.21.15.123:0/1912958861 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:53.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.436+0000 7f8715400640 1 --2- 172.21.15.123:0/1912958861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f871806c8f0 0x7f871806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:53.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.436+0000 7f8715e00640 1 -- 172.21.15.123:0/1912958861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f871806c8f0 msgr2=0x7f871806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:53.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.436+0000 7f8715e00640 1 --2- 172.21.15.123:0/1912958861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f871806c8f0 0x7f871806cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.436+0000 7f8714a00640 1 -- 172.21.15.123:0/1912958861 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 732639185 0 0) 0x7f87180cf170 con 0x7f87180ccbb0 2024-04-24T05:46:53.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.436+0000 7f8715e00640 1 -- 172.21.15.123:0/1912958861 >> v1:172.21.15.123:6789/0 conn(0x7f87180ccbb0 legacy=0x7f87180d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:53.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.436+0000 7f8715e00640 1 -- 172.21.15.123:0/1912958861 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87180cfa10 con 0x7f871806c510 2024-04-24T05:46:53.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.437+0000 7f8715e00640 1 --2- 172.21.15.123:0/1912958861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f871806c510 0x7f8718071c50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f87080097a0 tx=0x7f870802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3dcb26e4d1d99a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:53.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.438+0000 7f8714a00640 1 -- 172.21.15.123:0/1912958861 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708037070 con 0x7f871806c510 2024-04-24T05:46:53.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.438+0000 7f8714a00640 1 -- 172.21.15.123:0/1912958861 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8708032440 con 0x7f871806c510 2024-04-24T05:46:53.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.438+0000 7f8714a00640 1 -- 172.21.15.123:0/1912958861 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708032810 con 0x7f871806c510 2024-04-24T05:46:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.438+0000 7f871d200640 1 -- 172.21.15.123:0/1912958861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f871806c510 msgr2=0x7f8718071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.438+0000 7f871d200640 1 --2- 172.21.15.123:0/1912958861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f871806c510 0x7f8718071c50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f87080097a0 tx=0x7f870802a180 comp rx=0 tx=0).stop 2024-04-24T05:46:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.438+0000 7f871d200640 1 -- 172.21.15.123:0/1912958861 shutdown_connections 2024-04-24T05:46:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.438+0000 7f871d200640 1 --2- 172.21.15.123:0/1912958861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f871806c8f0 0x7f871806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.438+0000 7f871d200640 1 --2- 172.21.15.123:0/1912958861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f871806c510 0x7f8718071c50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.438+0000 7f871d200640 1 -- 172.21.15.123:0/1912958861 >> 172.21.15.123:0/1912958861 conn(0x7f87180b51f0 msgr2=0x7f87180b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.439+0000 7f871d200640 1 -- 172.21.15.123:0/1912958861 shutdown_connections 2024-04-24T05:46:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.439+0000 7f871d200640 1 -- 172.21.15.123:0/1912958861 wait complete. 2024-04-24T05:46:53.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.440+0000 7f871d200640 1 Processor -- start 2024-04-24T05:46:53.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.440+0000 7f871d200640 1 -- start start 2024-04-24T05:46:53.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.441+0000 7f871d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f871806c510 0x7f87181512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:53.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.441+0000 7f871d200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f871806c8f0 0x7f871815d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:53.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.441+0000 7f871d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87180ccbb0 0x7f871815e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.441+0000 7f871d200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87180cee80 con 0x7f871806c510 2024-04-24T05:46:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.441+0000 7f871d200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f87180ced00 con 0x7f87180ccbb0 2024-04-24T05:46:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.441+0000 7f871d200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f87180cd040 con 0x7f871806c8f0 2024-04-24T05:46:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.442+0000 7f8715e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f871806c510 0x7f87181512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.442+0000 7f8715e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f871806c510 0x7f87181512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56564/0 (socket says 172.21.15.123:56564) 2024-04-24T05:46:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.442+0000 7f8715e00640 1 -- 172.21.15.123:0/2004191250 learned_addr learned my addr 172.21.15.123:0/2004191250 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.442+0000 7f8715400640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f871806c8f0 0x7f871815d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.442+0000 7f8716800640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87180ccbb0 0x7f871815e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.442+0000 7f8715e00640 1 -- 172.21.15.123:0/2004191250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f871806c8f0 msgr2=0x7f871815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.442+0000 7f8715e00640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f871806c8f0 0x7f871815d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.442+0000 7f8715e00640 1 -- 172.21.15.123:0/2004191250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87180ccbb0 msgr2=0x7f871815e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.442+0000 7f8715e00640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87180ccbb0 0x7f871815e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.442+0000 7f8715e00640 1 -- 172.21.15.123:0/2004191250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871815d940 con 0x7f871806c510 2024-04-24T05:46:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.442+0000 7f8716800640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87180ccbb0 0x7f871815e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:53.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.442+0000 7f8715e00640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f871806c510 0x7f87181512a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8708000c00 tx=0x7f87080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:53.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.443+0000 7f870e000640 1 -- 172.21.15.123:0/2004191250 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708041070 con 0x7f871806c510 2024-04-24T05:46:53.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.443+0000 7f870e000640 1 -- 172.21.15.123:0/2004191250 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870803c930 con 0x7f871806c510 2024-04-24T05:46:53.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.443+0000 7f870e000640 1 -- 172.21.15.123:0/2004191250 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708037040 con 0x7f871806c510 2024-04-24T05:46:53.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.443+0000 7f871d200640 1 -- 172.21.15.123:0/2004191250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87181e4f90 con 0x7f871806c510 2024-04-24T05:46:53.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.443+0000 7f871d200640 1 -- 172.21.15.123:0/2004191250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87180c4570 con 0x7f871806c510 2024-04-24T05:46:53.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.444+0000 7f871d200640 1 -- 172.21.15.123:0/2004191250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8718068a20 con 0x7f871806c510 2024-04-24T05:46:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.445+0000 7f870e000640 1 -- 172.21.15.123:0/2004191250 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8708044070 con 0x7f871806c510 2024-04-24T05:46:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.446+0000 7f870e000640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f86f806c8f0 0x7f86f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.446+0000 7f870e000640 1 -- 172.21.15.123:0/2004191250 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f87080acb40 con 0x7f871806c510 2024-04-24T05:46:53.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.446+0000 7f8715400640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f86f806c8f0 0x7f86f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:53.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.447+0000 7f8715400640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f86f806c8f0 0x7f86f806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f871815bd40 tx=0x7f8704008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:53.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.451+0000 7f870e000640 1 -- 172.21.15.123:0/2004191250 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f870807ac70 con 0x7f871806c510 2024-04-24T05:46:53.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.652+0000 7f871d200640 1 -- 172.21.15.123:0/2004191250 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f87181e79a0 con 0x7f871806c510 2024-04-24T05:46:53.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.652+0000 7f870e000640 1 -- 172.21.15.123:0/2004191250 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f870807a570 con 0x7f871806c510 2024-04-24T05:46:53.654 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:46:53.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.655+0000 7f871d200640 1 -- 172.21.15.123:0/2004191250 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f86f806c8f0 msgr2=0x7f86f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:53.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.655+0000 7f871d200640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f86f806c8f0 0x7f86f806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f871815bd40 tx=0x7f8704008040 comp rx=0 tx=0).stop 2024-04-24T05:46:53.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.655+0000 7f871d200640 1 -- 172.21.15.123:0/2004191250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f871806c510 msgr2=0x7f87181512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:53.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.655+0000 7f871d200640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f871806c510 0x7f87181512a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8708000c00 tx=0x7f87080046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.655+0000 7f871d200640 1 -- 172.21.15.123:0/2004191250 shutdown_connections 2024-04-24T05:46:53.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.655+0000 7f871d200640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f86f806c8f0 0x7f86f806edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.655+0000 7f871d200640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87180ccbb0 0x7f871815e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.655+0000 7f871d200640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f871806c8f0 0x7f871815d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.655+0000 7f871d200640 1 --2- 172.21.15.123:0/2004191250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f871806c510 0x7f87181512a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.655+0000 7f871d200640 1 -- 172.21.15.123:0/2004191250 >> 172.21.15.123:0/2004191250 conn(0x7f87180b51f0 msgr2=0x7f87180b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:53.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.655+0000 7f871d200640 1 -- 172.21.15.123:0/2004191250 shutdown_connections 2024-04-24T05:46:53.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.655+0000 7f871d200640 1 -- 172.21.15.123:0/2004191250 wait complete. 2024-04-24T05:46:53.673 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:46:53.675 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:46:53.678 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:46:53.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.864+0000 7ff92fe00640 1 Processor -- start 2024-04-24T05:46:53.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.864+0000 7ff92fe00640 1 -- start start 2024-04-24T05:46:53.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.865+0000 7ff92fe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff93015e790 0x7ff9300d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:53.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.865+0000 7ff92fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff9300b5f10 0x7ff9300b6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:53.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.865+0000 7ff92fe00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9300b6870 con 0x7ff9300d2120 2024-04-24T05:46:53.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.865+0000 7ff92fe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff9300d3a80 con 0x7ff9300b5f10 2024-04-24T05:46:53.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.865+0000 7ff92fe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff93015eb70 con 0x7ff93015e790 2024-04-24T05:46:53.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.865+0000 7ff92e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff9300b5f10 0x7ff9300b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:53.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.865+0000 7ff92e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff9300b5f10 0x7ff9300b6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56580/0 (socket says 172.21.15.123:56580) 2024-04-24T05:46:53.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.865+0000 7ff92e000640 1 -- 172.21.15.123:0/189900415 learned_addr learned my addr 172.21.15.123:0/189900415 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:53.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.866+0000 7ff92e000640 1 -- 172.21.15.123:0/189900415 >> v2:172.21.15.155:3300/0 conn(0x7ff93015e790 msgr2=0x7ff9300d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:53.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.866+0000 7ff92e000640 1 --2- 172.21.15.123:0/189900415 >> v2:172.21.15.155:3300/0 conn(0x7ff93015e790 0x7ff9300d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.866+0000 7ff92e000640 1 -- 172.21.15.123:0/189900415 >> v1:172.21.15.155:6789/0 conn(0x7ff9300d2120 legacy=0x7ff9300d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:53.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.866+0000 7ff92e000640 1 -- 172.21.15.123:0/189900415 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9300d2e70 con 0x7ff9300b5f10 2024-04-24T05:46:53.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.867+0000 7ff92e000640 1 --2- 172.21.15.123:0/189900415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9300b5f10 0x7ff9300b6330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff9300b7020 tx=0x7ff91802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3698d823ff028fb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:53.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.867+0000 7ff92d600640 1 -- 172.21.15.123:0/189900415 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918037070 con 0x7ff9300b5f10 2024-04-24T05:46:53.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.867+0000 7ff92d600640 1 -- 172.21.15.123:0/189900415 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff918032440 con 0x7ff9300b5f10 2024-04-24T05:46:53.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.867+0000 7ff92d600640 1 -- 172.21.15.123:0/189900415 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918032810 con 0x7ff9300b5f10 2024-04-24T05:46:53.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.868+0000 7ff92fe00640 1 -- 172.21.15.123:0/189900415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9300b5f10 msgr2=0x7ff9300b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:53.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.868+0000 7ff92fe00640 1 --2- 172.21.15.123:0/189900415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9300b5f10 0x7ff9300b6330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff9300b7020 tx=0x7ff91802a230 comp rx=0 tx=0).stop 2024-04-24T05:46:53.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.868+0000 7ff92fe00640 1 -- 172.21.15.123:0/189900415 shutdown_connections 2024-04-24T05:46:53.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.868+0000 7ff92fe00640 1 --2- 172.21.15.123:0/189900415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9300b5f10 0x7ff9300b6330 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.868+0000 7ff92fe00640 1 --2- 172.21.15.123:0/189900415 >> v2:172.21.15.155:3300/0 conn(0x7ff93015e790 0x7ff9300d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.868+0000 7ff92fe00640 1 -- 172.21.15.123:0/189900415 >> 172.21.15.123:0/189900415 conn(0x7ff93009d830 msgr2=0x7ff9300b73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:53.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.868+0000 7ff92fe00640 1 -- 172.21.15.123:0/189900415 shutdown_connections 2024-04-24T05:46:53.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.868+0000 7ff92fe00640 1 -- 172.21.15.123:0/189900415 wait complete. 2024-04-24T05:46:53.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.870+0000 7ff92fe00640 1 Processor -- start 2024-04-24T05:46:53.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.870+0000 7ff92fe00640 1 -- start start 2024-04-24T05:46:53.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.871+0000 7ff92fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9300b5f10 0x7ff93015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:53.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.871+0000 7ff92fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9300d2120 0x7ff93015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:53.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.871+0000 7ff92fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff93015e790 0x7ff93015d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:53.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.871+0000 7ff92fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9300d3710 con 0x7ff9300d2120 2024-04-24T05:46:53.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.871+0000 7ff92fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff9300d3430 con 0x7ff93015e790 2024-04-24T05:46:53.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.871+0000 7ff92fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff9301685a0 con 0x7ff9300b5f10 2024-04-24T05:46:53.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.871+0000 7ff92e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9300d2120 0x7ff93015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:53.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.871+0000 7ff92e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9300d2120 0x7ff93015cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56582/0 (socket says 172.21.15.123:56582) 2024-04-24T05:46:53.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.871+0000 7ff92e000640 1 -- 172.21.15.123:0/3709592445 learned_addr learned my addr 172.21.15.123:0/3709592445 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:53.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.872+0000 7ff92e000640 1 -- 172.21.15.123:0/3709592445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9300b5f10 msgr2=0x7ff93015cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:53.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.872+0000 7ff92e000640 1 --2- 172.21.15.123:0/3709592445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9300b5f10 0x7ff93015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.872+0000 7ff92e000640 1 -- 172.21.15.123:0/3709592445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff93015e790 msgr2=0x7ff93015d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.872+0000 7ff92f400640 1 --2- 172.21.15.123:0/3709592445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff93015e790 0x7ff93015d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.872+0000 7ff92e000640 1 --2- 172.21.15.123:0/3709592445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff93015e790 0x7ff93015d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.872+0000 7ff92e000640 1 -- 172.21.15.123:0/3709592445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9301560a0 con 0x7ff9300d2120 2024-04-24T05:46:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.872+0000 7ff92e000640 1 --2- 172.21.15.123:0/3709592445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9300d2120 0x7ff93015cff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff918000c00 tx=0x7ff9180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.872+0000 7ff926a00640 1 -- 172.21.15.123:0/3709592445 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918041070 con 0x7ff9300d2120 2024-04-24T05:46:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.872+0000 7ff926a00640 1 -- 172.21.15.123:0/3709592445 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff91803c870 con 0x7ff9300d2120 2024-04-24T05:46:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.872+0000 7ff926a00640 1 -- 172.21.15.123:0/3709592445 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918037040 con 0x7ff9300d2120 2024-04-24T05:46:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.872+0000 7ff92fe00640 1 -- 172.21.15.123:0/3709592445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff930156380 con 0x7ff9300d2120 2024-04-24T05:46:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.873+0000 7ff92fe00640 1 -- 172.21.15.123:0/3709592445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff93014dfc0 con 0x7ff9300d2120 2024-04-24T05:46:53.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.874+0000 7ff92fe00640 1 -- 172.21.15.123:0/3709592445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff930069130 con 0x7ff9300d2120 2024-04-24T05:46:53.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.875+0000 7ff926a00640 1 -- 172.21.15.123:0/3709592445 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff918044070 con 0x7ff9300d2120 2024-04-24T05:46:53.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.875+0000 7ff926a00640 1 --2- 172.21.15.123:0/3709592445 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff90c06c8f0 0x7ff90c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:53.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.876+0000 7ff926a00640 1 -- 172.21.15.123:0/3709592445 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7ff9180acb80 con 0x7ff9300d2120 2024-04-24T05:46:53.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.876+0000 7ff92ea00640 1 --2- 172.21.15.123:0/3709592445 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff90c06c8f0 0x7ff90c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:53.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.876+0000 7ff92ea00640 1 --2- 172.21.15.123:0/3709592445 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff90c06c8f0 0x7ff90c06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff92000ab90 tx=0x7ff920009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:53.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:53.880+0000 7ff926a00640 1 -- 172.21.15.123:0/3709592445 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff91807ab90 con 0x7ff9300d2120 2024-04-24T05:46:54.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.080+0000 7ff92fe00640 1 -- 172.21.15.123:0/3709592445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff9300b6bd0 con 0x7ff9300d2120 2024-04-24T05:46:54.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.081+0000 7ff926a00640 1 -- 172.21.15.123:0/3709592445 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7ff91807a490 con 0x7ff9300d2120 2024-04-24T05:46:54.082 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.083+0000 7ff92fe00640 1 -- 172.21.15.123:0/3709592445 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff90c06c8f0 msgr2=0x7ff90c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.083+0000 7ff92fe00640 1 --2- 172.21.15.123:0/3709592445 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff90c06c8f0 0x7ff90c06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff92000ab90 tx=0x7ff920009250 comp rx=0 tx=0).stop 2024-04-24T05:46:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.083+0000 7ff92fe00640 1 -- 172.21.15.123:0/3709592445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9300d2120 msgr2=0x7ff93015cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.083+0000 7ff92fe00640 1 --2- 172.21.15.123:0/3709592445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9300d2120 0x7ff93015cff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff918000c00 tx=0x7ff9180046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.083+0000 7ff92fe00640 1 -- 172.21.15.123:0/3709592445 shutdown_connections 2024-04-24T05:46:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.083+0000 7ff92fe00640 1 --2- 172.21.15.123:0/3709592445 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff90c06c8f0 0x7ff90c06edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.083+0000 7ff92fe00640 1 --2- 172.21.15.123:0/3709592445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff93015e790 0x7ff93015d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.083+0000 7ff92fe00640 1 --2- 172.21.15.123:0/3709592445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9300d2120 0x7ff93015cff0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.083+0000 7ff92fe00640 1 --2- 172.21.15.123:0/3709592445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9300b5f10 0x7ff93015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.083+0000 7ff92fe00640 1 -- 172.21.15.123:0/3709592445 >> 172.21.15.123:0/3709592445 conn(0x7ff93009d830 msgr2=0x7ff9301581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:54.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.084+0000 7ff92fe00640 1 -- 172.21.15.123:0/3709592445 shutdown_connections 2024-04-24T05:46:54.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.084+0000 7ff92fe00640 1 -- 172.21.15.123:0/3709592445 wait complete. 2024-04-24T05:46:54.104 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:54.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.298+0000 7f4ccb400640 1 Processor -- start 2024-04-24T05:46:54.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.298+0000 7f4ccb400640 1 -- start start 2024-04-24T05:46:54.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.299+0000 7f4ccb400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4cc406f680 0x7f4cc415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.299+0000 7f4ccb400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4cc406ba00 0x7f4cc4071c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.299+0000 7f4ccb400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cc406c0e0 con 0x7f4cc4072170 2024-04-24T05:46:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.299+0000 7f4ccb400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4cc406be80 con 0x7f4cc406ba00 2024-04-24T05:46:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.299+0000 7f4ccb400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4cc40725a0 con 0x7f4cc406f680 2024-04-24T05:46:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.300+0000 7f4cc9400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4cc4072170 0x7f4cc415b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:44018/0 (socket says 172.21.15.123:44018) 2024-04-24T05:46:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.300+0000 7f4cc9400640 1 -- 172.21.15.123:0/3674526080 learned_addr learned my addr 172.21.15.123:0/3674526080 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.300+0000 7f4cc8a00640 1 --2- 172.21.15.123:0/3674526080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4cc406f680 0x7f4cc415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.300+0000 7f4cc3e00640 1 --2- 172.21.15.123:0/3674526080 >> v2:172.21.15.155:3300/0 conn(0x7f4cc406ba00 0x7f4cc4071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.300+0000 7f4cc3400640 1 -- 172.21.15.123:0/3674526080 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 957726999 0 0) 0x7f4cc406c0e0 con 0x7f4cc4072170 2024-04-24T05:46:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.300+0000 7f4cc3400640 1 -- 172.21.15.123:0/3674526080 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cac003520 con 0x7f4cc4072170 2024-04-24T05:46:54.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3400640 1 -- 172.21.15.123:0/3674526080 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 928222336 0 0) 0x7f4cac003520 con 0x7f4cc4072170 2024-04-24T05:46:54.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3400640 1 -- 172.21.15.123:0/3674526080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4cc406f680 msgr2=0x7f4cc415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3400640 1 --2- 172.21.15.123:0/3674526080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4cc406f680 0x7f4cc415ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3400640 1 -- 172.21.15.123:0/3674526080 >> v2:172.21.15.155:3300/0 conn(0x7f4cc406ba00 msgr2=0x7f4cc4071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3400640 1 --2- 172.21.15.123:0/3674526080 >> v2:172.21.15.155:3300/0 conn(0x7f4cc406ba00 0x7f4cc4071c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3400640 1 -- 172.21.15.123:0/3674526080 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc406c3c0 con 0x7f4cc4072170 2024-04-24T05:46:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3400640 1 -- 172.21.15.123:0/3674526080 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f4cbc0019f0 con 0x7f4cc4072170 2024-04-24T05:46:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3400640 1 -- 172.21.15.123:0/3674526080 >> v1:172.21.15.123:6789/0 conn(0x7f4cc4072170 legacy=0x7f4cc415b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3400640 1 --2- 172.21.15.123:0/3674526080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cac0039e0 0x7f4cac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3400640 1 --2- 172.21.15.123:0/3674526080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4cc406f680 0x7f4cac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3400640 1 --2- 172.21.15.123:0/3674526080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cac004d40 0x7f4cac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3400640 1 -- 172.21.15.123:0/3674526080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc406be80 con 0x7f4cac0039e0 2024-04-24T05:46:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3400640 1 -- 172.21.15.123:0/3674526080 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc40725a0 con 0x7f4cac004d40 2024-04-24T05:46:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3400640 1 -- 172.21.15.123:0/3674526080 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4cac00b9f0 con 0x7f4cc406f680 2024-04-24T05:46:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc8a00640 1 --2- 172.21.15.123:0/3674526080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cac0039e0 0x7f4cac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:54.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.301+0000 7f4cc3e00640 1 --2- 172.21.15.123:0/3674526080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4cc406f680 0x7f4cac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:54.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.302+0000 7f4cc9400640 1 --2- 172.21.15.123:0/3674526080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cac004d40 0x7f4cac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:54.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.302+0000 7f4cc8a00640 1 -- 172.21.15.123:0/3674526080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4cc406f680 msgr2=0x7f4cac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:54.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.302+0000 7f4cc8a00640 1 --2- 172.21.15.123:0/3674526080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4cc406f680 0x7f4cac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.302+0000 7f4cc8a00640 1 -- 172.21.15.123:0/3674526080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cac004d40 msgr2=0x7f4cac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:54.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.302+0000 7f4cc8a00640 1 --2- 172.21.15.123:0/3674526080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cac004d40 0x7f4cac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.302+0000 7f4cc8a00640 1 -- 172.21.15.123:0/3674526080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac003520 con 0x7f4cac0039e0 2024-04-24T05:46:54.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.303+0000 7f4cc8a00640 1 --2- 172.21.15.123:0/3674526080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cac0039e0 0x7f4cac003e00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009ac0 tx=0x7f4cac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab353387b6c54df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:54.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.303+0000 7f4cc3400640 1 -- 172.21.15.123:0/3674526080 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8036070 con 0x7f4cac0039e0 2024-04-24T05:46:54.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.303+0000 7f4cc3400640 1 -- 172.21.15.123:0/3674526080 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cb8031400 con 0x7f4cac0039e0 2024-04-24T05:46:54.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.303+0000 7f4cc3400640 1 -- 172.21.15.123:0/3674526080 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8009690 con 0x7f4cac0039e0 2024-04-24T05:46:54.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.303+0000 7f4ccb400640 1 -- 172.21.15.123:0/3674526080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cac0039e0 msgr2=0x7f4cac003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:54.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.303+0000 7f4ccb400640 1 --2- 172.21.15.123:0/3674526080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cac0039e0 0x7f4cac003e00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009ac0 tx=0x7f4cac005830 comp rx=0 tx=0).stop 2024-04-24T05:46:54.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.303+0000 7f4cc9400640 1 -- 172.21.15.123:0/3674526080 reap_dead start 2024-04-24T05:46:54.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.304+0000 7f4ccb400640 1 -- 172.21.15.123:0/3674526080 shutdown_connections 2024-04-24T05:46:54.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.304+0000 7f4ccb400640 1 -- 172.21.15.123:0/3674526080 >> 172.21.15.123:0/3674526080 conn(0x7f4cc40b1e70 msgr2=0x7f4cc40b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:54.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.304+0000 7f4ccb400640 1 -- 172.21.15.123:0/3674526080 shutdown_connections 2024-04-24T05:46:54.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.304+0000 7f4ccb400640 1 -- 172.21.15.123:0/3674526080 wait complete. 2024-04-24T05:46:54.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.305+0000 7f4ccb400640 1 Processor -- start 2024-04-24T05:46:54.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.305+0000 7f4ccb400640 1 -- start start 2024-04-24T05:46:54.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.306+0000 7f4ccb400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4cac0039e0 0x7f4cc41616c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.306+0000 7f4ccb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cc41f83c0 0x7f4cc41f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.306+0000 7f4ccb400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cc41599d0 0x7f4cc40ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.306+0000 7f4ccb400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc406cc60 con 0x7f4cc41f83c0 2024-04-24T05:46:54.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.306+0000 7f4ccb400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc406c980 con 0x7f4cc41599d0 2024-04-24T05:46:54.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.306+0000 7f4ccb400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4cc406fa60 con 0x7f4cac0039e0 2024-04-24T05:46:54.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.307+0000 7f4cc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cc41f83c0 0x7f4cc41f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:54.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.307+0000 7f4cc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cc41f83c0 0x7f4cc41f3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56598/0 (socket says 172.21.15.123:56598) 2024-04-24T05:46:54.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.307+0000 7f4cc3e00640 1 -- 172.21.15.123:0/2798633300 learned_addr learned my addr 172.21.15.123:0/2798633300 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.307+0000 7f4cc8a00640 1 --2- 172.21.15.123:0/2798633300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4cac0039e0 0x7f4cc41616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.307+0000 7f4cc9400640 1 --2- 172.21.15.123:0/2798633300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cc41599d0 0x7f4cc40ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.307+0000 7f4cc3e00640 1 -- 172.21.15.123:0/2798633300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4cac0039e0 msgr2=0x7f4cc41616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.307+0000 7f4cc3e00640 1 --2- 172.21.15.123:0/2798633300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4cac0039e0 0x7f4cc41616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.307+0000 7f4cc3e00640 1 -- 172.21.15.123:0/2798633300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cc41599d0 msgr2=0x7f4cc40ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.307+0000 7f4cc3e00640 1 --2- 172.21.15.123:0/2798633300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cc41599d0 0x7f4cc40ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.307+0000 7f4cc3e00640 1 -- 172.21.15.123:0/2798633300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc415a2d0 con 0x7f4cc41f83c0 2024-04-24T05:46:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.307+0000 7f4cc3e00640 1 --2- 172.21.15.123:0/2798633300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cc41f83c0 0x7f4cc41f3a90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0011a70 tx=0x7f4cb0011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.308+0000 7f4cc0c00640 1 -- 172.21.15.123:0/2798633300 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb000fb50 con 0x7f4cc41f83c0 2024-04-24T05:46:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.308+0000 7f4cc0c00640 1 -- 172.21.15.123:0/2798633300 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cb000fcf0 con 0x7f4cc41f83c0 2024-04-24T05:46:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.308+0000 7f4cc0c00640 1 -- 172.21.15.123:0/2798633300 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb000fea0 con 0x7f4cc41f83c0 2024-04-24T05:46:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.308+0000 7f4ccb400640 1 -- 172.21.15.123:0/2798633300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc415a5b0 con 0x7f4cc41f83c0 2024-04-24T05:46:54.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.308+0000 7f4ccb400640 1 -- 172.21.15.123:0/2798633300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc41579e0 con 0x7f4cc41f83c0 2024-04-24T05:46:54.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.309+0000 7f4ccb400640 1 -- 172.21.15.123:0/2798633300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc4072a80 con 0x7f4cc41f83c0 2024-04-24T05:46:54.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.310+0000 7f4cc0c00640 1 -- 172.21.15.123:0/2798633300 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4cb0012040 con 0x7f4cc41f83c0 2024-04-24T05:46:54.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.311+0000 7f4cc0c00640 1 --2- 172.21.15.123:0/2798633300 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ca406c920 0x7f4ca406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.311+0000 7f4cc0c00640 1 -- 172.21.15.123:0/2798633300 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f4cb008c590 con 0x7f4cc41f83c0 2024-04-24T05:46:54.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.311+0000 7f4cc8a00640 1 --2- 172.21.15.123:0/2798633300 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ca406c920 0x7f4ca406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:54.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.312+0000 7f4cc8a00640 1 --2- 172.21.15.123:0/2798633300 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ca406c920 0x7f4ca406ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8033000 tx=0x7f4cb8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:54.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.316+0000 7f4cc0c00640 1 -- 172.21.15.123:0/2798633300 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4cc4072a80 con 0x7f4cc41f83c0 2024-04-24T05:46:54.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.504+0000 7f4ccb400640 1 -- 172.21.15.123:0/2798633300 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cc4068a20 con 0x7f4cc41f83c0 2024-04-24T05:46:54.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.506+0000 7f4cc0c00640 1 -- 172.21.15.123:0/2798633300 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7f4cb0019030 con 0x7f4cc41f83c0 2024-04-24T05:46:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:54.507 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:54.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.508+0000 7f4ccb400640 1 -- 172.21.15.123:0/2798633300 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ca406c920 msgr2=0x7f4ca406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:54.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.508+0000 7f4ccb400640 1 --2- 172.21.15.123:0/2798633300 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ca406c920 0x7f4ca406ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8033000 tx=0x7f4cb8008030 comp rx=0 tx=0).stop 2024-04-24T05:46:54.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.508+0000 7f4ccb400640 1 -- 172.21.15.123:0/2798633300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cc41f83c0 msgr2=0x7f4cc41f3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:54.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.508+0000 7f4ccb400640 1 --2- 172.21.15.123:0/2798633300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cc41f83c0 0x7f4cc41f3a90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0011a70 tx=0x7f4cb0011f30 comp rx=0 tx=0).stop 2024-04-24T05:46:54.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.509+0000 7f4ccb400640 1 -- 172.21.15.123:0/2798633300 shutdown_connections 2024-04-24T05:46:54.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.509+0000 7f4ccb400640 1 --2- 172.21.15.123:0/2798633300 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4ca406c920 0x7f4ca406ede0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.509+0000 7f4ccb400640 1 --2- 172.21.15.123:0/2798633300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cc41599d0 0x7f4cc40ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.509+0000 7f4ccb400640 1 --2- 172.21.15.123:0/2798633300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cc41f83c0 0x7f4cc41f3a90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.509+0000 7f4ccb400640 1 --2- 172.21.15.123:0/2798633300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4cac0039e0 0x7f4cc41616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.509+0000 7f4ccb400640 1 -- 172.21.15.123:0/2798633300 >> 172.21.15.123:0/2798633300 conn(0x7f4cc40b1e70 msgr2=0x7f4cc406dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:54.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.509+0000 7f4ccb400640 1 -- 172.21.15.123:0/2798633300 shutdown_connections 2024-04-24T05:46:54.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.509+0000 7f4ccb400640 1 -- 172.21.15.123:0/2798633300 wait complete. 2024-04-24T05:46:54.528 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:46:54.528 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:46:54.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.730+0000 7ff5de600640 1 Processor -- start 2024-04-24T05:46:54.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.730+0000 7ff5de600640 1 -- start start 2024-04-24T05:46:54.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.731+0000 7ff5de600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5d80ae990 0x7ff5d80d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.732+0000 7ff5de600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff5d806c410 0x7ff5d80ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.732+0000 7ff5de600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5d80d0740 con 0x7ff5d80ce610 2024-04-24T05:46:54.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.732+0000 7ff5de600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff5d80cea70 con 0x7ff5d806c410 2024-04-24T05:46:54.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.732+0000 7ff5de600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d806dcb0 con 0x7ff5d80ae990 2024-04-24T05:46:54.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.732+0000 7ff5d6a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff5d806c410 0x7ff5d80ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:54.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.732+0000 7ff5d6a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff5d806c410 0x7ff5d80ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56610/0 (socket says 172.21.15.123:56610) 2024-04-24T05:46:54.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.732+0000 7ff5d7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff5d80ce610 0x7ff5d815f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:44032/0 (socket says 172.21.15.123:44032) 2024-04-24T05:46:54.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.732+0000 7ff5d6a00640 1 -- 172.21.15.123:0/649816646 learned_addr learned my addr 172.21.15.123:0/649816646 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:54.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.732+0000 7ff5d7400640 1 --2- 172.21.15.123:0/649816646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5d80ae990 0x7ff5d80d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:54.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.732+0000 7ff5d6a00640 1 -- 172.21.15.123:0/649816646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5d80ae990 msgr2=0x7ff5d80d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:54.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.732+0000 7ff5d6a00640 1 --2- 172.21.15.123:0/649816646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5d80ae990 0x7ff5d80d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.732+0000 7ff5d6a00640 1 -- 172.21.15.123:0/649816646 >> v1:172.21.15.123:6789/0 conn(0x7ff5d80ce610 legacy=0x7ff5d815f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:54.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.732+0000 7ff5d6a00640 1 -- 172.21.15.123:0/649816646 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d80d0a20 con 0x7ff5d806c410 2024-04-24T05:46:54.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.732+0000 7ff5d6000640 1 -- 172.21.15.123:0/649816646 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2933117016 0 0) 0x7ff5d80d0740 con 0x7ff5d80ce610 2024-04-24T05:46:54.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.734+0000 7ff5d6a00640 1 --2- 172.21.15.123:0/649816646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d806c410 0x7ff5d80ce0d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009870 tx=0x7ff5c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e24437c453d02c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:54.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.734+0000 7ff5d6000640 1 -- 172.21.15.123:0/649816646 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4037070 con 0x7ff5d806c410 2024-04-24T05:46:54.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.734+0000 7ff5d6000640 1 -- 172.21.15.123:0/649816646 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5c4032440 con 0x7ff5d806c410 2024-04-24T05:46:54.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.734+0000 7ff5d6000640 1 -- 172.21.15.123:0/649816646 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4032810 con 0x7ff5d806c410 2024-04-24T05:46:54.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.734+0000 7ff5de600640 1 -- 172.21.15.123:0/649816646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d806c410 msgr2=0x7ff5d80ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:54.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.734+0000 7ff5de600640 1 --2- 172.21.15.123:0/649816646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d806c410 0x7ff5d80ce0d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009870 tx=0x7ff5c402a270 comp rx=0 tx=0).stop 2024-04-24T05:46:54.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.735+0000 7ff5de600640 1 -- 172.21.15.123:0/649816646 shutdown_connections 2024-04-24T05:46:54.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.735+0000 7ff5de600640 1 --2- 172.21.15.123:0/649816646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d806c410 0x7ff5d80ce0d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.735+0000 7ff5de600640 1 --2- 172.21.15.123:0/649816646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5d80ae990 0x7ff5d80d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.735+0000 7ff5de600640 1 -- 172.21.15.123:0/649816646 >> 172.21.15.123:0/649816646 conn(0x7ff5d80b51f0 msgr2=0x7ff5d80b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:54.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.735+0000 7ff5de600640 1 -- 172.21.15.123:0/649816646 shutdown_connections 2024-04-24T05:46:54.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.735+0000 7ff5de600640 1 -- 172.21.15.123:0/649816646 wait complete. 2024-04-24T05:46:54.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.736+0000 7ff5de600640 1 Processor -- start 2024-04-24T05:46:54.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.737+0000 7ff5de600640 1 -- start start 2024-04-24T05:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.737+0000 7ff5de600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5d806c410 0x7ff5d815b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.737+0000 7ff5de600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5d80ae990 0x7ff5d809d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.737+0000 7ff5de600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d80ce610 0x7ff5d80c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.737+0000 7ff5de600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d80aed70 con 0x7ff5d80ce610 2024-04-24T05:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.738+0000 7ff5de600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d806e270 con 0x7ff5d80ae990 2024-04-24T05:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.738+0000 7ff5de600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d806c870 con 0x7ff5d806c410 2024-04-24T05:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.738+0000 7ff5d7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d80ce610 0x7ff5d80c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.738+0000 7ff5d7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d80ce610 0x7ff5d80c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56614/0 (socket says 172.21.15.123:56614) 2024-04-24T05:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.738+0000 7ff5d7e00640 1 -- 172.21.15.123:0/2937925027 learned_addr learned my addr 172.21.15.123:0/2937925027 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.738+0000 7ff5d7e00640 1 -- 172.21.15.123:0/2937925027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5d806c410 msgr2=0x7ff5d815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.738+0000 7ff5d7e00640 1 --2- 172.21.15.123:0/2937925027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5d806c410 0x7ff5d815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.738+0000 7ff5d7e00640 1 -- 172.21.15.123:0/2937925027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5d80ae990 msgr2=0x7ff5d809d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:54.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.738+0000 7ff5d7e00640 1 --2- 172.21.15.123:0/2937925027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5d80ae990 0x7ff5d809d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.738+0000 7ff5d7e00640 1 -- 172.21.15.123:0/2937925027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d815dc20 con 0x7ff5d80ce610 2024-04-24T05:46:54.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.738+0000 7ff5d7e00640 1 --2- 172.21.15.123:0/2937925027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d80ce610 0x7ff5d80c9470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000c7a0 tx=0x7ff5d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:54.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.739+0000 7ff5cf400640 1 -- 172.21.15.123:0/2937925027 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0018070 con 0x7ff5d80ce610 2024-04-24T05:46:54.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.739+0000 7ff5cf400640 1 -- 172.21.15.123:0/2937925027 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d000a640 con 0x7ff5d80ce610 2024-04-24T05:46:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.739+0000 7ff5cf400640 1 -- 172.21.15.123:0/2937925027 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000aab0 con 0x7ff5d80ce610 2024-04-24T05:46:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.739+0000 7ff5de600640 1 -- 172.21.15.123:0/2937925027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d815df00 con 0x7ff5d80ce610 2024-04-24T05:46:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.739+0000 7ff5de600640 1 -- 172.21.15.123:0/2937925027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d815b740 con 0x7ff5d80ce610 2024-04-24T05:46:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.740+0000 7ff5de600640 1 -- 172.21.15.123:0/2937925027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d8068a20 con 0x7ff5d80ce610 2024-04-24T05:46:54.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.741+0000 7ff5cf400640 1 -- 172.21.15.123:0/2937925027 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5d000ac50 con 0x7ff5d80ce610 2024-04-24T05:46:54.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.742+0000 7ff5cf400640 1 --2- 172.21.15.123:0/2937925027 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5b806c8f0 0x7ff5b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:54.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.742+0000 7ff5cf400640 1 -- 172.21.15.123:0/2937925027 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7ff5d008b610 con 0x7ff5d80ce610 2024-04-24T05:46:54.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.742+0000 7ff5d7400640 1 --2- 172.21.15.123:0/2937925027 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5b806c8f0 0x7ff5b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:54.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.743+0000 7ff5d7400640 1 --2- 172.21.15.123:0/2937925027 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5b806c8f0 0x7ff5b806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80098f0 tx=0x7ff5c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:54.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.746+0000 7ff5cf400640 1 -- 172.21.15.123:0/2937925027 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5d0059620 con 0x7ff5d80ce610 2024-04-24T05:46:54.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.945+0000 7ff5de600640 1 -- 172.21.15.123:0/2937925027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff5d806cb80 con 0x7ff5d80ce610 2024-04-24T05:46:54.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.946+0000 7ff5cf400640 1 -- 172.21.15.123:0/2937925027 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7ff5d0058f20 con 0x7ff5d80ce610 2024-04-24T05:46:54.946 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:46:54.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.948+0000 7ff5de600640 1 -- 172.21.15.123:0/2937925027 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5b806c8f0 msgr2=0x7ff5b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:54.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.948+0000 7ff5de600640 1 --2- 172.21.15.123:0/2937925027 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5b806c8f0 0x7ff5b806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80098f0 tx=0x7ff5c8008040 comp rx=0 tx=0).stop 2024-04-24T05:46:54.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.949+0000 7ff5de600640 1 -- 172.21.15.123:0/2937925027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d80ce610 msgr2=0x7ff5d80c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:54.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.949+0000 7ff5de600640 1 --2- 172.21.15.123:0/2937925027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d80ce610 0x7ff5d80c9470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000c7a0 tx=0x7ff5d000cc60 comp rx=0 tx=0).stop 2024-04-24T05:46:54.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.949+0000 7ff5de600640 1 -- 172.21.15.123:0/2937925027 shutdown_connections 2024-04-24T05:46:54.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.949+0000 7ff5de600640 1 --2- 172.21.15.123:0/2937925027 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5b806c8f0 0x7ff5b806edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.949+0000 7ff5de600640 1 --2- 172.21.15.123:0/2937925027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5d80ce610 0x7ff5d80c9470 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.949+0000 7ff5de600640 1 --2- 172.21.15.123:0/2937925027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5d80ae990 0x7ff5d809d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.949+0000 7ff5de600640 1 --2- 172.21.15.123:0/2937925027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5d806c410 0x7ff5d815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:54.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.949+0000 7ff5de600640 1 -- 172.21.15.123:0/2937925027 >> 172.21.15.123:0/2937925027 conn(0x7ff5d80b51f0 msgr2=0x7ff5d80cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:54.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.949+0000 7ff5de600640 1 -- 172.21.15.123:0/2937925027 shutdown_connections 2024-04-24T05:46:54.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:54.949+0000 7ff5de600640 1 -- 172.21.15.123:0/2937925027 wait complete. 2024-04-24T05:46:54.968 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:46:55.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.172+0000 7fa9fd000640 1 Processor -- start 2024-04-24T05:46:55.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.172+0000 7fa9fd000640 1 -- start start 2024-04-24T05:46:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.174+0000 7fa9fd000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f8158fe0 0x7fa9f80b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.174+0000 7fa9fd000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa9f815ee30 0x7fa9f806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.174+0000 7fa9fd000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9f815f3d0 con 0x7fa9f806d7e0 2024-04-24T05:46:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.174+0000 7fa9fd000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa9f806e540 con 0x7fa9f815ee30 2024-04-24T05:46:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.174+0000 7fa9fd000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f815f6b0 con 0x7fa9f8158fe0 2024-04-24T05:46:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.174+0000 7fa9f6800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa9f806d7e0 0x7fa9f8162420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:44040/0 (socket says 172.21.15.123:44040) 2024-04-24T05:46:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.174+0000 7fa9f6800640 1 -- 172.21.15.123:0/3966864933 learned_addr learned my addr 172.21.15.123:0/3966864933 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:55.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.175+0000 7fa9ed400640 1 --2- 172.21.15.123:0/3966864933 >> v2:172.21.15.155:3300/0 conn(0x7fa9f815ee30 0x7fa9f806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:55.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.175+0000 7fa9f5e00640 1 --2- 172.21.15.123:0/3966864933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f8158fe0 0x7fa9f80b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:55.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.175+0000 7fa9f5400640 1 -- 172.21.15.123:0/3966864933 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 453843134 0 0) 0x7fa9f815f3d0 con 0x7fa9f806d7e0 2024-04-24T05:46:55.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.175+0000 7fa9f5400640 1 -- 172.21.15.123:0/3966864933 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9dc003520 con 0x7fa9f806d7e0 2024-04-24T05:46:55.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.175+0000 7fa9ed400640 1 -- 172.21.15.123:0/3966864933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f8158fe0 msgr2=0x7fa9f80b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:55.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.175+0000 7fa9ed400640 1 --2- 172.21.15.123:0/3966864933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f8158fe0 0x7fa9f80b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.175+0000 7fa9ed400640 1 -- 172.21.15.123:0/3966864933 >> v1:172.21.15.123:6789/0 conn(0x7fa9f806d7e0 legacy=0x7fa9f8162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:55.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.175+0000 7fa9ed400640 1 -- 172.21.15.123:0/3966864933 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f815f990 con 0x7fa9f815ee30 2024-04-24T05:46:55.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.176+0000 7fa9ed400640 1 --2- 172.21.15.123:0/3966864933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f815ee30 0x7fa9f806d2a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00097a0 tx=0x7fa9e002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc69ef7b2c2ebe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:55.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.177+0000 7fa9f5400640 1 -- 172.21.15.123:0/3966864933 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0037070 con 0x7fa9f815ee30 2024-04-24T05:46:55.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.177+0000 7fa9f5400640 1 -- 172.21.15.123:0/3966864933 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e0032440 con 0x7fa9f815ee30 2024-04-24T05:46:55.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.177+0000 7fa9f5400640 1 -- 172.21.15.123:0/3966864933 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e00327f0 con 0x7fa9f815ee30 2024-04-24T05:46:55.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.177+0000 7fa9fd000640 1 -- 172.21.15.123:0/3966864933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f815ee30 msgr2=0x7fa9f806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:55.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.177+0000 7fa9fd000640 1 --2- 172.21.15.123:0/3966864933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f815ee30 0x7fa9f806d2a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00097a0 tx=0x7fa9e002a180 comp rx=0 tx=0).stop 2024-04-24T05:46:55.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.177+0000 7fa9fd000640 1 -- 172.21.15.123:0/3966864933 shutdown_connections 2024-04-24T05:46:55.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.177+0000 7fa9fd000640 1 --2- 172.21.15.123:0/3966864933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f815ee30 0x7fa9f806d2a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.177+0000 7fa9fd000640 1 --2- 172.21.15.123:0/3966864933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f8158fe0 0x7fa9f80b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.177+0000 7fa9fd000640 1 -- 172.21.15.123:0/3966864933 >> 172.21.15.123:0/3966864933 conn(0x7fa9f80724e0 msgr2=0x7fa9f80728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:55.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.178+0000 7fa9fd000640 1 -- 172.21.15.123:0/3966864933 shutdown_connections 2024-04-24T05:46:55.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.178+0000 7fa9fd000640 1 -- 172.21.15.123:0/3966864933 wait complete. 2024-04-24T05:46:55.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.179+0000 7fa9fd000640 1 Processor -- start 2024-04-24T05:46:55.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.179+0000 7fa9fd000640 1 -- start start 2024-04-24T05:46:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.180+0000 7fa9fd000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f806d7e0 0x7fa9f81593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.180+0000 7fa9fd000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f8158fe0 0x7fa9f8158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.180+0000 7fa9fd000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9f815ee30 0x7fa9f81e9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.180+0000 7fa9fd000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f806f9e0 con 0x7fa9f815ee30 2024-04-24T05:46:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.180+0000 7fa9fd000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f80b3fe0 con 0x7fa9f8158fe0 2024-04-24T05:46:55.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.180+0000 7fa9fd000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f806fb60 con 0x7fa9f806d7e0 2024-04-24T05:46:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.181+0000 7fa9f6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9f815ee30 0x7fa9f81e9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.181+0000 7fa9f6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9f815ee30 0x7fa9f81e9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56620/0 (socket says 172.21.15.123:56620) 2024-04-24T05:46:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.181+0000 7fa9f6800640 1 -- 172.21.15.123:0/1533214722 learned_addr learned my addr 172.21.15.123:0/1533214722 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.181+0000 7fa9ed400640 1 --2- 172.21.15.123:0/1533214722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f8158fe0 0x7fa9f8158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.181+0000 7fa9f5e00640 1 --2- 172.21.15.123:0/1533214722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f806d7e0 0x7fa9f81593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.181+0000 7fa9f6800640 1 -- 172.21.15.123:0/1533214722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f806d7e0 msgr2=0x7fa9f81593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.181+0000 7fa9f6800640 1 --2- 172.21.15.123:0/1533214722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f806d7e0 0x7fa9f81593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.181+0000 7fa9f6800640 1 -- 172.21.15.123:0/1533214722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f8158fe0 msgr2=0x7fa9f8158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.181+0000 7fa9f6800640 1 --2- 172.21.15.123:0/1533214722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f8158fe0 0x7fa9f8158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.181+0000 7fa9f6800640 1 -- 172.21.15.123:0/1533214722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f8159af0 con 0x7fa9f815ee30 2024-04-24T05:46:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.181+0000 7fa9f6800640 1 --2- 172.21.15.123:0/1533214722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9f815ee30 0x7fa9f81e9cc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000caa0 tx=0x7fa9f000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:55.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.182+0000 7fa9eea00640 1 -- 172.21.15.123:0/1533214722 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0017070 con 0x7fa9f815ee30 2024-04-24T05:46:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.182+0000 7fa9eea00640 1 -- 172.21.15.123:0/1533214722 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f000ac00 con 0x7fa9f815ee30 2024-04-24T05:46:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.182+0000 7fa9eea00640 1 -- 172.21.15.123:0/1533214722 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0012400 con 0x7fa9f815ee30 2024-04-24T05:46:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.182+0000 7fa9fd000640 1 -- 172.21.15.123:0/1533214722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f8159dd0 con 0x7fa9f815ee30 2024-04-24T05:46:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.182+0000 7fa9fd000640 1 -- 172.21.15.123:0/1533214722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f815a390 con 0x7fa9f815ee30 2024-04-24T05:46:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.183+0000 7fa9fd000640 1 -- 172.21.15.123:0/1533214722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f8068a20 con 0x7fa9f815ee30 2024-04-24T05:46:55.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.184+0000 7fa9eea00640 1 -- 172.21.15.123:0/1533214722 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9f001a070 con 0x7fa9f815ee30 2024-04-24T05:46:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.185+0000 7fa9eea00640 1 --2- 172.21.15.123:0/1533214722 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa9d406c8f0 0x7fa9d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:55.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.185+0000 7fa9eea00640 1 -- 172.21.15.123:0/1533214722 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fa9f008bd80 con 0x7fa9f815ee30 2024-04-24T05:46:55.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.185+0000 7fa9f5e00640 1 --2- 172.21.15.123:0/1533214722 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa9d406c8f0 0x7fa9d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:55.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.186+0000 7fa9f5e00640 1 --2- 172.21.15.123:0/1533214722 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa9d406c8f0 0x7fa9d406edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006fd0 tx=0x7fa9e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:55.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.190+0000 7fa9eea00640 1 -- 172.21.15.123:0/1533214722 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9f00596b0 con 0x7fa9f815ee30 2024-04-24T05:46:55.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.390+0000 7fa9fd000640 1 -- 172.21.15.123:0/1533214722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa9f80b3ae0 con 0x7fa9f815ee30 2024-04-24T05:46:55.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.391+0000 7fa9eea00640 1 -- 172.21.15.123:0/1533214722 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fa9f005d9b0 con 0x7fa9f815ee30 2024-04-24T05:46:55.392 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:55.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.393+0000 7fa9fd000640 1 -- 172.21.15.123:0/1533214722 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa9d406c8f0 msgr2=0x7fa9d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:55.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.393+0000 7fa9fd000640 1 --2- 172.21.15.123:0/1533214722 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa9d406c8f0 0x7fa9d406edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006fd0 tx=0x7fa9e8008040 comp rx=0 tx=0).stop 2024-04-24T05:46:55.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.393+0000 7fa9fd000640 1 -- 172.21.15.123:0/1533214722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9f815ee30 msgr2=0x7fa9f81e9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:55.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.393+0000 7fa9fd000640 1 --2- 172.21.15.123:0/1533214722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9f815ee30 0x7fa9f81e9cc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000caa0 tx=0x7fa9f000cf60 comp rx=0 tx=0).stop 2024-04-24T05:46:55.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.394+0000 7fa9fd000640 1 -- 172.21.15.123:0/1533214722 shutdown_connections 2024-04-24T05:46:55.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.394+0000 7fa9fd000640 1 --2- 172.21.15.123:0/1533214722 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa9d406c8f0 0x7fa9d406edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.394+0000 7fa9fd000640 1 --2- 172.21.15.123:0/1533214722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9f815ee30 0x7fa9f81e9cc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.394+0000 7fa9fd000640 1 --2- 172.21.15.123:0/1533214722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f8158fe0 0x7fa9f8158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.394+0000 7fa9fd000640 1 --2- 172.21.15.123:0/1533214722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9f806d7e0 0x7fa9f81593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.394+0000 7fa9fd000640 1 -- 172.21.15.123:0/1533214722 >> 172.21.15.123:0/1533214722 conn(0x7fa9f80724e0 msgr2=0x7fa9f80b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:55.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.394+0000 7fa9fd000640 1 -- 172.21.15.123:0/1533214722 shutdown_connections 2024-04-24T05:46:55.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.394+0000 7fa9fd000640 1 -- 172.21.15.123:0/1533214722 wait complete. 2024-04-24T05:46:55.413 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:46:55.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.621+0000 7f6c90800640 1 Processor -- start 2024-04-24T05:46:55.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.622+0000 7f6c90800640 1 -- start start 2024-04-24T05:46:55.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.623+0000 7f6c90800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c8815c270 0x7f6c8815c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:55.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.623+0000 7f6c90800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c880d0840 con 0x7f6c880b7dd0 2024-04-24T05:46:55.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.623+0000 7f6c90800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c880d0b20 con 0x7f6c880d02f0 2024-04-24T05:46:55.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.623+0000 7f6c90800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6c880d0e00 con 0x7f6c8815c270 2024-04-24T05:46:55.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.624+0000 7f6c8e800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6c880b7dd0 0x7f6c880d18e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:44050/0 (socket says 172.21.15.123:44050) 2024-04-24T05:46:55.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.624+0000 7f6c8e800640 1 -- 172.21.15.123:0/921023152 learned_addr learned my addr 172.21.15.123:0/921023152 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:55.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.624+0000 7f6c8d400640 1 --1- 172.21.15.123:0/921023152 >> v1:172.21.15.155:6789/0 conn(0x7f6c880d02f0 0x7f6c880b52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.123:34664/0 (socket says 172.21.15.123:34664) 2024-04-24T05:46:55.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.624+0000 7f6c8de00640 1 --2- 172.21.15.123:0/921023152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c8815c270 0x7f6c8815c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:55.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.624+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1888342582 0 0) 0x7f6c880d0840 con 0x7f6c880b7dd0 2024-04-24T05:46:55.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.624+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c74003520 con 0x7f6c880b7dd0 2024-04-24T05:46:55.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.624+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 923591059 0 0) 0x7f6c880d0b20 con 0x7f6c880d02f0 2024-04-24T05:46:55.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.624+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c880d0840 con 0x7f6c880d02f0 2024-04-24T05:46:55.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.624+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 113333356 0 0) 0x7f6c74003520 con 0x7f6c880b7dd0 2024-04-24T05:46:55.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.624+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c8815c270 msgr2=0x7f6c8815c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:55.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.624+0000 7f6c8ca00640 1 --2- 172.21.15.123:0/921023152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c8815c270 0x7f6c8815c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.624+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 >> v1:172.21.15.155:6789/0 conn(0x7f6c880d02f0 legacy=0x7f6c880b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:55.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.624+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c880d10e0 con 0x7f6c880b7dd0 2024-04-24T05:46:55.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.624+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f6c84001670 con 0x7f6c880b7dd0 2024-04-24T05:46:55.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.625+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 >> v1:172.21.15.123:6789/0 conn(0x7f6c880b7dd0 legacy=0x7f6c880d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:55.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.625+0000 7f6c8ca00640 1 --2- 172.21.15.123:0/921023152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c8815c270 0x7f6c74003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:55.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.625+0000 7f6c8ca00640 1 --2- 172.21.15.123:0/921023152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c740040d0 0x7f6c74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:55.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.625+0000 7f6c8ca00640 1 --2- 172.21.15.123:0/921023152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c74004d40 0x7f6c740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:55.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.625+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c880d0e00 con 0x7f6c74004d40 2024-04-24T05:46:55.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.625+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7400baa0 con 0x7f6c740040d0 2024-04-24T05:46:55.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.625+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7400bca0 con 0x7f6c8815c270 2024-04-24T05:46:55.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.625+0000 7f6c8e800640 1 --2- 172.21.15.123:0/921023152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c74004d40 0x7f6c740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:55.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.625+0000 7f6c8e800640 1 -- 172.21.15.123:0/921023152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c8815c270 msgr2=0x7f6c74003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:55.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.625+0000 7f6c8e800640 1 --2- 172.21.15.123:0/921023152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c8815c270 0x7f6c74003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.625+0000 7f6c8e800640 1 -- 172.21.15.123:0/921023152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c740040d0 msgr2=0x7f6c74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:55.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.625+0000 7f6c8e800640 1 --2- 172.21.15.123:0/921023152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c740040d0 0x7f6c74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.625+0000 7f6c8e800640 1 -- 172.21.15.123:0/921023152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c74003520 con 0x7f6c74004d40 2024-04-24T05:46:55.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.626+0000 7f6c8e800640 1 --2- 172.21.15.123:0/921023152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c74004d40 0x7f6c740051c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400c210 tx=0x7f6c74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20dfe1d81174072 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:55.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.627+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84007ce0 con 0x7f6c74004d40 2024-04-24T05:46:55.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.627+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c84005ad0 con 0x7f6c74004d40 2024-04-24T05:46:55.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.627+0000 7f6c8ca00640 1 -- 172.21.15.123:0/921023152 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84007070 con 0x7f6c74004d40 2024-04-24T05:46:55.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.627+0000 7f6c90800640 1 -- 172.21.15.123:0/921023152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c74004d40 msgr2=0x7f6c740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:55.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.627+0000 7f6c90800640 1 --2- 172.21.15.123:0/921023152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c74004d40 0x7f6c740051c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400c210 tx=0x7f6c74005830 comp rx=0 tx=0).stop 2024-04-24T05:46:55.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.627+0000 7f6c8e800640 1 -- 172.21.15.123:0/921023152 reap_dead start 2024-04-24T05:46:55.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.627+0000 7f6c90800640 1 -- 172.21.15.123:0/921023152 shutdown_connections 2024-04-24T05:46:55.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.627+0000 7f6c90800640 1 -- 172.21.15.123:0/921023152 >> 172.21.15.123:0/921023152 conn(0x7f6c880d23f0 msgr2=0x7f6c880d27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:55.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.628+0000 7f6c90800640 1 -- 172.21.15.123:0/921023152 shutdown_connections 2024-04-24T05:46:55.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.628+0000 7f6c90800640 1 -- 172.21.15.123:0/921023152 wait complete. 2024-04-24T05:46:55.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.629+0000 7f6c90800640 1 Processor -- start 2024-04-24T05:46:55.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.629+0000 7f6c90800640 1 -- start start 2024-04-24T05:46:55.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.630+0000 7f6c90800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c74004d40 0x7f6c881495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:55.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.630+0000 7f6c90800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c881f8290 0x7f6c881f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:55.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.630+0000 7f6c90800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c8814f7c0 0x7f6c8814fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:55.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.630+0000 7f6c90800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c880b59b0 con 0x7f6c881f8290 2024-04-24T05:46:55.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.630+0000 7f6c90800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c880b3c80 con 0x7f6c8814f7c0 2024-04-24T05:46:55.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.630+0000 7f6c90800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6c8806e970 con 0x7f6c74004d40 2024-04-24T05:46:55.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.631+0000 7f6c8d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c881f8290 0x7f6c881f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:55.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.631+0000 7f6c8d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c881f8290 0x7f6c881f8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56640/0 (socket says 172.21.15.123:56640) 2024-04-24T05:46:55.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.631+0000 7f6c8d400640 1 -- 172.21.15.123:0/4209050427 learned_addr learned my addr 172.21.15.123:0/4209050427 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:55.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.631+0000 7f6c8de00640 1 --2- 172.21.15.123:0/4209050427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c74004d40 0x7f6c881495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:55.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.631+0000 7f6c8e800640 1 --2- 172.21.15.123:0/4209050427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c8814f7c0 0x7f6c8814fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:55.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.631+0000 7f6c8d400640 1 -- 172.21.15.123:0/4209050427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c74004d40 msgr2=0x7f6c881495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:55.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.631+0000 7f6c8d400640 1 --2- 172.21.15.123:0/4209050427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c74004d40 0x7f6c881495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.631+0000 7f6c8d400640 1 -- 172.21.15.123:0/4209050427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c8814f7c0 msgr2=0x7f6c8814fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:55.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.631+0000 7f6c8d400640 1 --2- 172.21.15.123:0/4209050427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c8814f7c0 0x7f6c8814fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.631+0000 7f6c8d400640 1 -- 172.21.15.123:0/4209050427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c881f8d50 con 0x7f6c881f8290 2024-04-24T05:46:55.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.632+0000 7f6c8d400640 1 --2- 172.21.15.123:0/4209050427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c881f8290 0x7f6c881f8710 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800c700 tx=0x7f6c7800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:55.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.632+0000 7f6c7e000640 1 -- 172.21.15.123:0/4209050427 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78018070 con 0x7f6c881f8290 2024-04-24T05:46:55.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.632+0000 7f6c7e000640 1 -- 172.21.15.123:0/4209050427 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c78007270 con 0x7f6c881f8290 2024-04-24T05:46:55.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.632+0000 7f6c7e000640 1 -- 172.21.15.123:0/4209050427 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78007660 con 0x7f6c881f8290 2024-04-24T05:46:55.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.632+0000 7f6c90800640 1 -- 172.21.15.123:0/4209050427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c881f9030 con 0x7f6c881f8290 2024-04-24T05:46:55.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.632+0000 7f6c90800640 1 -- 172.21.15.123:0/4209050427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c88157db0 con 0x7f6c881f8290 2024-04-24T05:46:55.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.633+0000 7f6c90800640 1 -- 172.21.15.123:0/4209050427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c8806e3f0 con 0x7f6c881f8290 2024-04-24T05:46:55.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.634+0000 7f6c7e000640 1 -- 172.21.15.123:0/4209050427 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c7801b070 con 0x7f6c881f8290 2024-04-24T05:46:55.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.635+0000 7f6c7e000640 1 --2- 172.21.15.123:0/4209050427 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6c6c06c8f0 0x7f6c6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:55.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.635+0000 7f6c7e000640 1 -- 172.21.15.123:0/4209050427 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f6c7808b350 con 0x7f6c881f8290 2024-04-24T05:46:55.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.635+0000 7f6c8de00640 1 --2- 172.21.15.123:0/4209050427 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6c6c06c8f0 0x7f6c6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:55.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.636+0000 7f6c8de00640 1 --2- 172.21.15.123:0/4209050427 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6c6c06c8f0 0x7f6c6c06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6c80004b30 tx=0x7f6c8000d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:55.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.639+0000 7f6c7e000640 1 -- 172.21.15.123:0/4209050427 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c78059360 con 0x7f6c881f8290 2024-04-24T05:46:55.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.839+0000 7f6c90800640 1 -- 172.21.15.123:0/4209050427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6c8806eaf0 con 0x7f6c881f8290 2024-04-24T05:46:55.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.840+0000 7f6c7e000640 1 -- 172.21.15.123:0/4209050427 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f6c78058c60 con 0x7f6c881f8290 2024-04-24T05:46:55.841 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:46:55.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.843+0000 7f6c90800640 1 -- 172.21.15.123:0/4209050427 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6c6c06c8f0 msgr2=0x7f6c6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:55.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.843+0000 7f6c90800640 1 --2- 172.21.15.123:0/4209050427 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6c6c06c8f0 0x7f6c6c06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6c80004b30 tx=0x7f6c8000d040 comp rx=0 tx=0).stop 2024-04-24T05:46:55.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.843+0000 7f6c90800640 1 -- 172.21.15.123:0/4209050427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c881f8290 msgr2=0x7f6c881f8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:55.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.843+0000 7f6c90800640 1 --2- 172.21.15.123:0/4209050427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c881f8290 0x7f6c881f8710 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800c700 tx=0x7f6c7800cbc0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.843+0000 7f6c90800640 1 -- 172.21.15.123:0/4209050427 shutdown_connections 2024-04-24T05:46:55.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.843+0000 7f6c90800640 1 --2- 172.21.15.123:0/4209050427 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6c6c06c8f0 0x7f6c6c06edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.843+0000 7f6c90800640 1 --2- 172.21.15.123:0/4209050427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c8814f7c0 0x7f6c8814fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.843+0000 7f6c90800640 1 --2- 172.21.15.123:0/4209050427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c881f8290 0x7f6c881f8710 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.843+0000 7f6c90800640 1 --2- 172.21.15.123:0/4209050427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c74004d40 0x7f6c881495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:55.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.843+0000 7f6c90800640 1 -- 172.21.15.123:0/4209050427 >> 172.21.15.123:0/4209050427 conn(0x7f6c880d23f0 msgr2=0x7f6c880c8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:55.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.843+0000 7f6c90800640 1 -- 172.21.15.123:0/4209050427 shutdown_connections 2024-04-24T05:46:55.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:55.843+0000 7f6c90800640 1 -- 172.21.15.123:0/4209050427 wait complete. 2024-04-24T05:46:55.864 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:46:56.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.069+0000 7f7249c00640 1 Processor -- start 2024-04-24T05:46:56.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.069+0000 7f7249c00640 1 -- start start 2024-04-24T05:46:56.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.070+0000 7f7249c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f724406dcb0 0x7f724406e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.070+0000 7f7249c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f724406e960 0x7f724406ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.070+0000 7f7249c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f724406f2d0 con 0x7f7244158e00 2024-04-24T05:46:56.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.070+0000 7f7249c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f724406f590 con 0x7f724406e960 2024-04-24T05:46:56.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.070+0000 7f7249c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f724406c460 con 0x7f724406dcb0 2024-04-24T05:46:56.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.071+0000 7f7243400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f724406e960 0x7f724406ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.071+0000 7f7243400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f724406e960 0x7f724406ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56652/0 (socket says 172.21.15.123:56652) 2024-04-24T05:46:56.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.071+0000 7f7243400640 1 -- 172.21.15.123:0/2492112483 learned_addr learned my addr 172.21.15.123:0/2492112483 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:56.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.071+0000 7f7242000640 1 --2- 172.21.15.123:0/2492112483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f724406dcb0 0x7f724406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.072+0000 7f7243400640 1 -- 172.21.15.123:0/2492112483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f724406dcb0 msgr2=0x7f724406e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:56.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.072+0000 7f7243400640 1 --2- 172.21.15.123:0/2492112483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f724406dcb0 0x7f724406e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.072+0000 7f7243400640 1 -- 172.21.15.123:0/2492112483 >> v1:172.21.15.123:6789/0 conn(0x7f7244158e00 legacy=0x7f724406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.072+0000 7f7243400640 1 -- 172.21.15.123:0/2492112483 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724406bb30 con 0x7f724406e960 2024-04-24T05:46:56.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.072+0000 7f7241600640 1 -- 172.21.15.123:0/2492112483 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3761818174 0 0) 0x7f724406f2d0 con 0x7f7244158e00 2024-04-24T05:46:56.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.073+0000 7f7243400640 1 --2- 172.21.15.123:0/2492112483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f724406e960 0x7f724406ed90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f723c009e50 tx=0x7f723c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de194425214ac17 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:56.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.073+0000 7f7241600640 1 -- 172.21.15.123:0/2492112483 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c036070 con 0x7f724406e960 2024-04-24T05:46:56.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.073+0000 7f7241600640 1 -- 172.21.15.123:0/2492112483 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f723c031600 con 0x7f724406e960 2024-04-24T05:46:56.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.073+0000 7f7241600640 1 -- 172.21.15.123:0/2492112483 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c0319d0 con 0x7f724406e960 2024-04-24T05:46:56.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.073+0000 7f7249c00640 1 -- 172.21.15.123:0/2492112483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f724406e960 msgr2=0x7f724406ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.073+0000 7f7249c00640 1 --2- 172.21.15.123:0/2492112483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f724406e960 0x7f724406ed90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f723c009e50 tx=0x7f723c02a680 comp rx=0 tx=0).stop 2024-04-24T05:46:56.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.074+0000 7f7249c00640 1 -- 172.21.15.123:0/2492112483 shutdown_connections 2024-04-24T05:46:56.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.074+0000 7f7249c00640 1 --2- 172.21.15.123:0/2492112483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f724406e960 0x7f724406ed90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.074+0000 7f7249c00640 1 --2- 172.21.15.123:0/2492112483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f724406dcb0 0x7f724406e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.074+0000 7f7249c00640 1 -- 172.21.15.123:0/2492112483 >> 172.21.15.123:0/2492112483 conn(0x7f72440b51f0 msgr2=0x7f72440b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:56.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.074+0000 7f7249c00640 1 -- 172.21.15.123:0/2492112483 shutdown_connections 2024-04-24T05:46:56.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.074+0000 7f7249c00640 1 -- 172.21.15.123:0/2492112483 wait complete. 2024-04-24T05:46:56.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.075+0000 7f7249c00640 1 Processor -- start 2024-04-24T05:46:56.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.076+0000 7f7249c00640 1 -- start start 2024-04-24T05:46:56.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.076+0000 7f7249c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f724406dcb0 0x7f724414a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.076+0000 7f7249c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f724406e960 0x7f7244157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7249c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7244158e00 0x7f7244158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7249c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f724406d000 con 0x7f7244158e00 2024-04-24T05:46:56.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7249c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f724406c0f0 con 0x7f724406dcb0 2024-04-24T05:46:56.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7249c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f72441591e0 con 0x7f724406e960 2024-04-24T05:46:56.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7243400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7244158e00 0x7f7244158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7243400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7244158e00 0x7f7244158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56658/0 (socket says 172.21.15.123:56658) 2024-04-24T05:46:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7243400640 1 -- 172.21.15.123:0/707379909 learned_addr learned my addr 172.21.15.123:0/707379909 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7242000640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f724406e960 0x7f7244157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7242a00640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f724406dcb0 0x7f724414a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7243400640 1 -- 172.21.15.123:0/707379909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f724406e960 msgr2=0x7f7244157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7243400640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f724406e960 0x7f7244157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7243400640 1 -- 172.21.15.123:0/707379909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f724406dcb0 msgr2=0x7f724414a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7243400640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f724406dcb0 0x7f724414a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7243400640 1 -- 172.21.15.123:0/707379909 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7244156930 con 0x7f7244158e00 2024-04-24T05:46:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.077+0000 7f7242a00640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f724406dcb0 0x7f724414a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:56.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.078+0000 7f7243400640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7244158e00 0x7f7244158130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f723c009e50 tx=0x7f723c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:56.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.078+0000 7f723aa00640 1 -- 172.21.15.123:0/707379909 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c036050 con 0x7f7244158e00 2024-04-24T05:46:56.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.078+0000 7f723aa00640 1 -- 172.21.15.123:0/707379909 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f723c042070 con 0x7f7244158e00 2024-04-24T05:46:56.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.078+0000 7f723aa00640 1 -- 172.21.15.123:0/707379909 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c009510 con 0x7f7244158e00 2024-04-24T05:46:56.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.078+0000 7f7249c00640 1 -- 172.21.15.123:0/707379909 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7244156c10 con 0x7f7244158e00 2024-04-24T05:46:56.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.078+0000 7f7249c00640 1 -- 172.21.15.123:0/707379909 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72441571d0 con 0x7f7244158e00 2024-04-24T05:46:56.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.079+0000 7f7249c00640 1 -- 172.21.15.123:0/707379909 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f724406cdf0 con 0x7f7244158e00 2024-04-24T05:46:56.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.080+0000 7f723aa00640 1 -- 172.21.15.123:0/707379909 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f723c007bd0 con 0x7f7244158e00 2024-04-24T05:46:56.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.081+0000 7f723aa00640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f722406c8f0 0x7f722406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.081+0000 7f723aa00640 1 -- 172.21.15.123:0/707379909 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f723c0acd90 con 0x7f7244158e00 2024-04-24T05:46:56.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.081+0000 7f7242a00640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f722406c8f0 0x7f722406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.082+0000 7f7242a00640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f722406c8f0 0x7f722406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7234000f60 tx=0x7f7234009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:56.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.085+0000 7f723aa00640 1 -- 172.21.15.123:0/707379909 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f724406cdf0 con 0x7f7244158e00 2024-04-24T05:46:56.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.294+0000 7f7249c00640 1 -- 172.21.15.123:0/707379909 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7244068a20 con 0x7f7244158e00 2024-04-24T05:46:56.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.295+0000 7f723aa00640 1 -- 172.21.15.123:0/707379909 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f723c07ada0 con 0x7f7244158e00 2024-04-24T05:46:56.295 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:56.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.297+0000 7f7249c00640 1 -- 172.21.15.123:0/707379909 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f722406c8f0 msgr2=0x7f722406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.297+0000 7f7249c00640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f722406c8f0 0x7f722406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7234000f60 tx=0x7f7234009040 comp rx=0 tx=0).stop 2024-04-24T05:46:56.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.297+0000 7f7249c00640 1 -- 172.21.15.123:0/707379909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7244158e00 msgr2=0x7f7244158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.297+0000 7f7249c00640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7244158e00 0x7f7244158130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f723c009e50 tx=0x7f723c004a40 comp rx=0 tx=0).stop 2024-04-24T05:46:56.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.298+0000 7f7249c00640 1 -- 172.21.15.123:0/707379909 shutdown_connections 2024-04-24T05:46:56.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.298+0000 7f7249c00640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f722406c8f0 0x7f722406edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.298+0000 7f7249c00640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7244158e00 0x7f7244158130 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.298+0000 7f7249c00640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f724406e960 0x7f7244157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.298+0000 7f7249c00640 1 --2- 172.21.15.123:0/707379909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f724406dcb0 0x7f724414a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.298+0000 7f7249c00640 1 -- 172.21.15.123:0/707379909 >> 172.21.15.123:0/707379909 conn(0x7f72440b51f0 msgr2=0x7f72440b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:56.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.298+0000 7f7249c00640 1 -- 172.21.15.123:0/707379909 shutdown_connections 2024-04-24T05:46:56.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.298+0000 7f7249c00640 1 -- 172.21.15.123:0/707379909 wait complete. 2024-04-24T05:46:56.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:56.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.516+0000 7fe31de00640 1 Processor -- start 2024-04-24T05:46:56.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.516+0000 7fe31de00640 1 -- start start 2024-04-24T05:46:56.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.518+0000 7fe31de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe31806f680 0x7fe31815ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.518+0000 7fe31de00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe318072340 0x7fe31815b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.518+0000 7fe31de00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe31806c0e0 con 0x7fe31806ba00 2024-04-24T05:46:56.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.518+0000 7fe31de00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe31806beb0 con 0x7fe31806f680 2024-04-24T05:46:56.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.518+0000 7fe31de00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe31806c3c0 con 0x7fe318072340 2024-04-24T05:46:56.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.518+0000 7fe316a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe31806f680 0x7fe31815ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.518+0000 7fe316a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe31806f680 0x7fe31815ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56660/0 (socket says 172.21.15.123:56660) 2024-04-24T05:46:56.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.518+0000 7fe316a00640 1 -- 172.21.15.123:0/4283179153 learned_addr learned my addr 172.21.15.123:0/4283179153 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:56.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.519+0000 7fe317400640 1 --2- 172.21.15.123:0/4283179153 >> v2:172.21.15.155:3300/0 conn(0x7fe318072340 0x7fe31815b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.519+0000 7fe316a00640 1 -- 172.21.15.123:0/4283179153 >> v2:172.21.15.155:3300/0 conn(0x7fe318072340 msgr2=0x7fe31815b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:56.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.519+0000 7fe316a00640 1 --2- 172.21.15.123:0/4283179153 >> v2:172.21.15.155:3300/0 conn(0x7fe318072340 0x7fe31815b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.519+0000 7fe316a00640 1 -- 172.21.15.123:0/4283179153 >> v1:172.21.15.155:6789/0 conn(0x7fe31806ba00 legacy=0x7fe318071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:56.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.519+0000 7fe316a00640 1 -- 172.21.15.123:0/4283179153 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31806c6a0 con 0x7fe31806f680 2024-04-24T05:46:56.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.520+0000 7fe316a00640 1 --2- 172.21.15.123:0/4283179153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe31806f680 0x7fe31815ed80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe31815fe50 tx=0x7fe30802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53d2b4b30d9f404 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:56.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.520+0000 7fe316000640 1 -- 172.21.15.123:0/4283179153 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308037070 con 0x7fe31806f680 2024-04-24T05:46:56.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.520+0000 7fe316000640 1 -- 172.21.15.123:0/4283179153 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe308032440 con 0x7fe31806f680 2024-04-24T05:46:56.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.520+0000 7fe316000640 1 -- 172.21.15.123:0/4283179153 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308032810 con 0x7fe31806f680 2024-04-24T05:46:56.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.521+0000 7fe31de00640 1 -- 172.21.15.123:0/4283179153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe31806f680 msgr2=0x7fe31815ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.521+0000 7fe31de00640 1 --2- 172.21.15.123:0/4283179153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe31806f680 0x7fe31815ed80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe31815fe50 tx=0x7fe30802a230 comp rx=0 tx=0).stop 2024-04-24T05:46:56.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.521+0000 7fe31de00640 1 -- 172.21.15.123:0/4283179153 shutdown_connections 2024-04-24T05:46:56.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.521+0000 7fe31de00640 1 --2- 172.21.15.123:0/4283179153 >> v2:172.21.15.155:3300/0 conn(0x7fe318072340 0x7fe31815b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.521+0000 7fe31de00640 1 --2- 172.21.15.123:0/4283179153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe31806f680 0x7fe31815ed80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.521+0000 7fe31de00640 1 -- 172.21.15.123:0/4283179153 >> 172.21.15.123:0/4283179153 conn(0x7fe3180b1e70 msgr2=0x7fe3180b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:56.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.521+0000 7fe31de00640 1 -- 172.21.15.123:0/4283179153 shutdown_connections 2024-04-24T05:46:56.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.521+0000 7fe31de00640 1 -- 172.21.15.123:0/4283179153 wait complete. 2024-04-24T05:46:56.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.523+0000 7fe31de00640 1 Processor -- start 2024-04-24T05:46:56.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.523+0000 7fe31de00640 1 -- start start 2024-04-24T05:46:56.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.524+0000 7fe31de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe31806ba00 0x7fe31815ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.524+0000 7fe31de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe31806f680 0x7fe318156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.524+0000 7fe31de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318072340 0x7fe3181ea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.524+0000 7fe31de00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe31806fa60 con 0x7fe318072340 2024-04-24T05:46:56.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.524+0000 7fe31de00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe31806cc60 con 0x7fe31806f680 2024-04-24T05:46:56.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.524+0000 7fe31de00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe318072a80 con 0x7fe31806ba00 2024-04-24T05:46:56.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.524+0000 7fe317400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318072340 0x7fe3181ea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.524+0000 7fe317400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318072340 0x7fe3181ea5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56666/0 (socket says 172.21.15.123:56666) 2024-04-24T05:46:56.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.524+0000 7fe317400640 1 -- 172.21.15.123:0/627060993 learned_addr learned my addr 172.21.15.123:0/627060993 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:56.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.525+0000 7fe317400640 1 -- 172.21.15.123:0/627060993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe31806ba00 msgr2=0x7fe31815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.525+0000 7fe317400640 1 --2- 172.21.15.123:0/627060993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe31806ba00 0x7fe31815ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.525+0000 7fe317400640 1 -- 172.21.15.123:0/627060993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe31806f680 msgr2=0x7fe318156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.525+0000 7fe317400640 1 --2- 172.21.15.123:0/627060993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe31806f680 0x7fe318156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.525+0000 7fe317400640 1 -- 172.21.15.123:0/627060993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe318158360 con 0x7fe318072340 2024-04-24T05:46:56.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.525+0000 7fe317400640 1 --2- 172.21.15.123:0/627060993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318072340 0x7fe3181ea5c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe31000cc80 tx=0x7fe31000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:56.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.525+0000 7fe30fe00640 1 -- 172.21.15.123:0/627060993 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31000ce30 con 0x7fe318072340 2024-04-24T05:46:56.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.525+0000 7fe30fe00640 1 -- 172.21.15.123:0/627060993 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe310003990 con 0x7fe318072340 2024-04-24T05:46:56.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.525+0000 7fe30fe00640 1 -- 172.21.15.123:0/627060993 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310003d60 con 0x7fe318072340 2024-04-24T05:46:56.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.525+0000 7fe31de00640 1 -- 172.21.15.123:0/627060993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe318160d90 con 0x7fe318072340 2024-04-24T05:46:56.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.526+0000 7fe31de00640 1 -- 172.21.15.123:0/627060993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe318158900 con 0x7fe318072340 2024-04-24T05:46:56.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.527+0000 7fe31de00640 1 -- 172.21.15.123:0/627060993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe318068a20 con 0x7fe318072340 2024-04-24T05:46:56.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.528+0000 7fe30fe00640 1 -- 172.21.15.123:0/627060993 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe310003f00 con 0x7fe318072340 2024-04-24T05:46:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.529+0000 7fe30fe00640 1 --2- 172.21.15.123:0/627060993 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe2f406c8f0 0x7fe2f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.529+0000 7fe30fe00640 1 -- 172.21.15.123:0/627060993 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fe31008c350 con 0x7fe318072340 2024-04-24T05:46:56.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.529+0000 7fe316a00640 1 --2- 172.21.15.123:0/627060993 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe2f406c8f0 0x7fe2f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.530+0000 7fe316a00640 1 --2- 172.21.15.123:0/627060993 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe2f406c8f0 0x7fe2f406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe308034000 tx=0x7fe30802c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:56.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.533+0000 7fe30fe00640 1 -- 172.21.15.123:0/627060993 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe310014030 con 0x7fe318072340 2024-04-24T05:46:56.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.739+0000 7fe31de00640 1 -- 172.21.15.123:0/627060993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3180b1410 con 0x7fe318072340 2024-04-24T05:46:56.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.741+0000 7fe30fe00640 1 -- 172.21.15.123:0/627060993 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7fe31005a360 con 0x7fe318072340 2024-04-24T05:46:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:56.742 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:56.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.744+0000 7fe31de00640 1 -- 172.21.15.123:0/627060993 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe2f406c8f0 msgr2=0x7fe2f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.744+0000 7fe31de00640 1 --2- 172.21.15.123:0/627060993 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe2f406c8f0 0x7fe2f406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe308034000 tx=0x7fe30802c000 comp rx=0 tx=0).stop 2024-04-24T05:46:56.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.744+0000 7fe31de00640 1 -- 172.21.15.123:0/627060993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318072340 msgr2=0x7fe3181ea5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.744+0000 7fe31de00640 1 --2- 172.21.15.123:0/627060993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318072340 0x7fe3181ea5c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe31000cc80 tx=0x7fe31000a5c0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.744+0000 7fe31de00640 1 -- 172.21.15.123:0/627060993 shutdown_connections 2024-04-24T05:46:56.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.744+0000 7fe31de00640 1 --2- 172.21.15.123:0/627060993 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe2f406c8f0 0x7fe2f406edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.744+0000 7fe31de00640 1 --2- 172.21.15.123:0/627060993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318072340 0x7fe3181ea5c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.744+0000 7fe31de00640 1 --2- 172.21.15.123:0/627060993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe31806f680 0x7fe318156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.744+0000 7fe31de00640 1 --2- 172.21.15.123:0/627060993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe31806ba00 0x7fe31815ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.744+0000 7fe31de00640 1 -- 172.21.15.123:0/627060993 >> 172.21.15.123:0/627060993 conn(0x7fe3180b1e70 msgr2=0x7fe3180ad150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:56.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.744+0000 7fe31de00640 1 -- 172.21.15.123:0/627060993 shutdown_connections 2024-04-24T05:46:56.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.744+0000 7fe31de00640 1 -- 172.21.15.123:0/627060993 wait complete. 2024-04-24T05:46:56.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:46:56.764 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:46:56.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.968+0000 7f8c06400640 1 Processor -- start 2024-04-24T05:46:56.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.969+0000 7f8c06400640 1 -- start start 2024-04-24T05:46:56.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.970+0000 7f8c06400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8c000b35e0 0x7f8c000d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.970+0000 7f8c06400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0015ee50 con 0x7f8c000cfac0 2024-04-24T05:46:56.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.970+0000 7f8c06400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c000cff80 con 0x7f8c000d45a0 2024-04-24T05:46:56.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.970+0000 7f8c06400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8c000d0260 con 0x7f8c000b35e0 2024-04-24T05:46:56.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.970+0000 7f8bfea00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8c000cfac0 0x7f8c000d21e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:44062/0 (socket says 172.21.15.123:44062) 2024-04-24T05:46:56.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.970+0000 7f8bfea00640 1 -- 172.21.15.123:0/3590738101 learned_addr learned my addr 172.21.15.123:0/3590738101 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:56.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.971+0000 7f8bffe00640 1 --2- 172.21.15.123:0/3590738101 >> v2:172.21.15.155:3300/0 conn(0x7f8c000b35e0 0x7f8c000d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.971+0000 7f8bfe000640 1 -- 172.21.15.123:0/3590738101 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1229473691 0 0) 0x7f8c0015ee50 con 0x7f8c000cfac0 2024-04-24T05:46:56.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.971+0000 7f8bfe000640 1 -- 172.21.15.123:0/3590738101 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8be4003520 con 0x7f8c000cfac0 2024-04-24T05:46:56.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.971+0000 7f8bfe000640 1 -- 172.21.15.123:0/3590738101 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2092391236 0 0) 0x7f8be4003520 con 0x7f8c000cfac0 2024-04-24T05:46:56.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.971+0000 7f8bfe000640 1 -- 172.21.15.123:0/3590738101 >> v2:172.21.15.155:3300/0 conn(0x7f8c000b35e0 msgr2=0x7f8c000d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:56.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.971+0000 7f8bfe000640 1 --2- 172.21.15.123:0/3590738101 >> v2:172.21.15.155:3300/0 conn(0x7f8c000b35e0 0x7f8c000d28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.971+0000 7f8bfe000640 1 -- 172.21.15.123:0/3590738101 >> v1:172.21.15.155:6789/0 conn(0x7f8c000d45a0 legacy=0x7f8c000d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.971+0000 7f8bfe000640 1 -- 172.21.15.123:0/3590738101 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c000d0540 con 0x7f8c000cfac0 2024-04-24T05:46:56.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.971+0000 7f8bfe000640 1 -- 172.21.15.123:0/3590738101 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f8bec001670 con 0x7f8c000cfac0 2024-04-24T05:46:56.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.971+0000 7f8bfe000640 1 -- 172.21.15.123:0/3590738101 >> v1:172.21.15.123:6789/0 conn(0x7f8c000cfac0 legacy=0x7f8c000d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.971+0000 7f8bfe000640 1 --2- 172.21.15.123:0/3590738101 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8be40039b0 0x7f8be4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.972+0000 7f8bfe000640 1 --2- 172.21.15.123:0/3590738101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8be4004800 0x7f8be4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.972+0000 7f8bfe000640 1 --2- 172.21.15.123:0/3590738101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8be40051c0 0x7f8be400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.972+0000 7f8bfe000640 1 -- 172.21.15.123:0/3590738101 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c000d0260 con 0x7f8be40051c0 2024-04-24T05:46:56.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.972+0000 7f8bfe000640 1 -- 172.21.15.123:0/3590738101 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8be400c060 con 0x7f8be4004800 2024-04-24T05:46:56.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.972+0000 7f8bfe000640 1 -- 172.21.15.123:0/3590738101 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8be400c260 con 0x7f8be40039b0 2024-04-24T05:46:56.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.972+0000 7f8bffe00640 1 --2- 172.21.15.123:0/3590738101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8be40051c0 0x7f8be400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.972+0000 7f8bff400640 1 --2- 172.21.15.123:0/3590738101 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8be40039b0 0x7f8be4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.972+0000 7f8bffe00640 1 -- 172.21.15.123:0/3590738101 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8be40039b0 msgr2=0x7f8be4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:56.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.972+0000 7f8bffe00640 1 --2- 172.21.15.123:0/3590738101 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8be40039b0 0x7f8be4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.972+0000 7f8bffe00640 1 -- 172.21.15.123:0/3590738101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8be4004800 msgr2=0x7f8be4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:56.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.972+0000 7f8bffe00640 1 --2- 172.21.15.123:0/3590738101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8be4004800 0x7f8be4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.972+0000 7f8bffe00640 1 -- 172.21.15.123:0/3590738101 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be4003520 con 0x7f8be40051c0 2024-04-24T05:46:56.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.973+0000 7f8bffe00640 1 -- 172.21.15.123:0/3590738101 reap_dead start 2024-04-24T05:46:56.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.973+0000 7f8bffe00640 1 --2- 172.21.15.123:0/3590738101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8be40051c0 0x7f8be400b9c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009fc0 tx=0x7f8be4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9cb5d3c05971e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:56.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.974+0000 7f8bfe000640 1 -- 172.21.15.123:0/3590738101 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8035020 con 0x7f8be40051c0 2024-04-24T05:46:56.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.974+0000 7f8bfe000640 1 -- 172.21.15.123:0/3590738101 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf8003bd0 con 0x7f8be40051c0 2024-04-24T05:46:56.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.974+0000 7f8bfe000640 1 -- 172.21.15.123:0/3590738101 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8009c10 con 0x7f8be40051c0 2024-04-24T05:46:56.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.974+0000 7f8c06400640 1 -- 172.21.15.123:0/3590738101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8be40051c0 msgr2=0x7f8be400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.974+0000 7f8c06400640 1 --2- 172.21.15.123:0/3590738101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8be40051c0 0x7f8be400b9c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009fc0 tx=0x7f8be4005830 comp rx=0 tx=0).stop 2024-04-24T05:46:56.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.974+0000 7f8c06400640 1 -- 172.21.15.123:0/3590738101 shutdown_connections 2024-04-24T05:46:56.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.974+0000 7f8c06400640 1 --2- 172.21.15.123:0/3590738101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8be40051c0 0x7f8be400b9c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.974+0000 7f8c06400640 1 -- 172.21.15.123:0/3590738101 >> 172.21.15.123:0/3590738101 conn(0x7f8c0009d970 msgr2=0x7f8c000b95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:56.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.974+0000 7f8c06400640 1 -- 172.21.15.123:0/3590738101 shutdown_connections 2024-04-24T05:46:56.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.974+0000 7f8c06400640 1 -- 172.21.15.123:0/3590738101 wait complete. 2024-04-24T05:46:56.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.976+0000 7f8c06400640 1 Processor -- start 2024-04-24T05:46:56.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.976+0000 7f8c06400640 1 -- start start 2024-04-24T05:46:56.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.977+0000 7f8c06400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8be40051c0 0x7f8c000cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.977+0000 7f8c06400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0014c900 0x7f8c001e8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.977+0000 7f8c06400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c001e6a00 0x7f8c001e6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.977+0000 7f8c06400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c000b3ba0 con 0x7f8be40051c0 2024-04-24T05:46:56.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.977+0000 7f8c06400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c000d0b00 con 0x7f8c001e6a00 2024-04-24T05:46:56.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.977+0000 7f8c06400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c000b3d20 con 0x7f8c0014c900 2024-04-24T05:46:56.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.977+0000 7f8bff400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8be40051c0 0x7f8c000cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.977+0000 7f8bff400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8be40051c0 0x7f8c000cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56680/0 (socket says 172.21.15.123:56680) 2024-04-24T05:46:56.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.977+0000 7f8bff400640 1 -- 172.21.15.123:0/3509850727 learned_addr learned my addr 172.21.15.123:0/3509850727 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:56.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.978+0000 7f8bffe00640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c001e6a00 0x7f8c001e6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.978+0000 7f8bff400640 1 -- 172.21.15.123:0/3509850727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0014c900 msgr2=0x7f8c001e8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.978+0000 7f8bfea00640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0014c900 0x7f8c001e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.978+0000 7f8bff400640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0014c900 0x7f8c001e8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.978+0000 7f8bff400640 1 -- 172.21.15.123:0/3509850727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c001e6a00 msgr2=0x7f8c001e6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:56.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.978+0000 7f8bff400640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c001e6a00 0x7f8c001e6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:56.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.978+0000 7f8bff400640 1 -- 172.21.15.123:0/3509850727 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c000d2080 con 0x7f8be40051c0 2024-04-24T05:46:56.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.978+0000 7f8bffe00640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c001e6a00 0x7f8c001e6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:56.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.978+0000 7f8bff400640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8be40051c0 0x7f8c000cb6e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000d4c0 tx=0x7f8bf000d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:56.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.978+0000 7f8bf7400640 1 -- 172.21.15.123:0/3509850727 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0005480 con 0x7f8be40051c0 2024-04-24T05:46:56.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.978+0000 7f8bf7400640 1 -- 172.21.15.123:0/3509850727 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf0017070 con 0x7f8be40051c0 2024-04-24T05:46:56.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.978+0000 7f8bf7400640 1 -- 172.21.15.123:0/3509850727 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0003830 con 0x7f8be40051c0 2024-04-24T05:46:56.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.979+0000 7f8c06400640 1 -- 172.21.15.123:0/3509850727 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c000d2360 con 0x7f8be40051c0 2024-04-24T05:46:56.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.979+0000 7f8c06400640 1 -- 172.21.15.123:0/3509850727 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c000d2920 con 0x7f8be40051c0 2024-04-24T05:46:56.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.980+0000 7f8c06400640 1 -- 172.21.15.123:0/3509850727 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c000b8aa0 con 0x7f8be40051c0 2024-04-24T05:46:56.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.981+0000 7f8bf7400640 1 -- 172.21.15.123:0/3509850727 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8bf00039d0 con 0x7f8be40051c0 2024-04-24T05:46:56.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.982+0000 7f8bf7400640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8be006c920 0x7f8be006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:56.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.982+0000 7f8bf7400640 1 -- 172.21.15.123:0/3509850727 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f8bf008bde0 con 0x7f8be40051c0 2024-04-24T05:46:56.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.982+0000 7f8bfea00640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8be006c920 0x7f8be006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:56.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.983+0000 7f8bfea00640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8be006c920 0x7f8be006ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0011e0 tx=0x7f8bec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:56.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:56.986+0000 7f8bf7400640 1 -- 172.21.15.123:0/3509850727 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c000b8aa0 con 0x7f8be40051c0 2024-04-24T05:46:57.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.184+0000 7f8c06400640 1 -- 172.21.15.123:0/3509850727 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8c00068a20 con 0x7f8be40051c0 2024-04-24T05:46:57.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.185+0000 7f8bf7400640 1 -- 172.21.15.123:0/3509850727 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f8bf0059d60 con 0x7f8be40051c0 2024-04-24T05:46:57.185 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:46:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.187+0000 7f8c06400640 1 -- 172.21.15.123:0/3509850727 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8be006c920 msgr2=0x7f8be006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.187+0000 7f8c06400640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8be006c920 0x7f8be006ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0011e0 tx=0x7f8bec009040 comp rx=0 tx=0).stop 2024-04-24T05:46:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.187+0000 7f8c06400640 1 -- 172.21.15.123:0/3509850727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8be40051c0 msgr2=0x7f8c000cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.187+0000 7f8c06400640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8be40051c0 0x7f8c000cb6e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000d4c0 tx=0x7f8bf000d980 comp rx=0 tx=0).stop 2024-04-24T05:46:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.187+0000 7f8c06400640 1 -- 172.21.15.123:0/3509850727 shutdown_connections 2024-04-24T05:46:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.187+0000 7f8c06400640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f8be006c920 0x7f8be006ede0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.187+0000 7f8c06400640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c001e6a00 0x7f8c001e6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.187+0000 7f8c06400640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c0014c900 0x7f8c001e8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.187+0000 7f8c06400640 1 --2- 172.21.15.123:0/3509850727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8be40051c0 0x7f8c000cb6e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.187+0000 7f8c06400640 1 -- 172.21.15.123:0/3509850727 >> 172.21.15.123:0/3509850727 conn(0x7f8c0009d970 msgr2=0x7f8c0015e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.187+0000 7f8c06400640 1 -- 172.21.15.123:0/3509850727 shutdown_connections 2024-04-24T05:46:57.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.187+0000 7f8c06400640 1 -- 172.21.15.123:0/3509850727 wait complete. 2024-04-24T05:46:57.208 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:46:57.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.410+0000 7f3ad1200640 1 Processor -- start 2024-04-24T05:46:57.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.411+0000 7f3ad1200640 1 -- start start 2024-04-24T05:46:57.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.412+0000 7f3ad1200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3acc0d3d70 0x7f3acc0b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:57.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.412+0000 7f3ad1200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3acc0cfc70 con 0x7f3acc0d4fb0 2024-04-24T05:46:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.412+0000 7f3ad1200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3acc0d2700 con 0x7f3acc15eb20 2024-04-24T05:46:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.412+0000 7f3ad1200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3acc0d29e0 con 0x7f3acc0d3d70 2024-04-24T05:46:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.413+0000 7f3ac9400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3acc0d3d70 0x7f3acc0b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.413+0000 7f3ac9e00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f3acc15eb20 0x7f3acc0d48a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.123:34680/0 (socket says 172.21.15.123:34680) 2024-04-24T05:46:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.413+0000 7f3ac9e00640 1 -- 172.21.15.123:0/3964033628 learned_addr learned my addr 172.21.15.123:0/3964033628 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:57.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.413+0000 7f3ac9400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3acc0d3d70 0x7f3acc0b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56694/0 (socket says 172.21.15.123:56694) 2024-04-24T05:46:57.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.413+0000 7f3ac9400640 1 -- 172.21.15.123:0/3964033628 >> v1:172.21.15.155:6789/0 conn(0x7f3acc15eb20 legacy=0x7f3acc0d48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:57.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.413+0000 7f3ac9400640 1 -- 172.21.15.123:0/3964033628 >> v1:172.21.15.123:6789/0 conn(0x7f3acc0d4fb0 legacy=0x7f3acc0d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:57.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.413+0000 7f3ac9400640 1 -- 172.21.15.123:0/3964033628 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc0d2cc0 con 0x7f3acc0d3d70 2024-04-24T05:46:57.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.414+0000 7f3ac9400640 1 --2- 172.21.15.123:0/3964033628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3acc0d3d70 0x7f3acc0b1290 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80097a0 tx=0x7f3ab802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325c6936d88e45db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:57.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.415+0000 7f3ac8a00640 1 -- 172.21.15.123:0/3964033628 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8037070 con 0x7f3acc0d3d70 2024-04-24T05:46:57.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.415+0000 7f3ac8a00640 1 -- 172.21.15.123:0/3964033628 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab8032640 con 0x7f3acc0d3d70 2024-04-24T05:46:57.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.415+0000 7f3ac8a00640 1 -- 172.21.15.123:0/3964033628 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8032a10 con 0x7f3acc0d3d70 2024-04-24T05:46:57.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.415+0000 7f3ad1200640 1 -- 172.21.15.123:0/3964033628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3acc0d3d70 msgr2=0x7f3acc0b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:57.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.415+0000 7f3ad1200640 1 --2- 172.21.15.123:0/3964033628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3acc0d3d70 0x7f3acc0b1290 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80097a0 tx=0x7f3ab802a2c0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.415+0000 7f3ad1200640 1 -- 172.21.15.123:0/3964033628 shutdown_connections 2024-04-24T05:46:57.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.415+0000 7f3ad1200640 1 --2- 172.21.15.123:0/3964033628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3acc0d3d70 0x7f3acc0b1290 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.415+0000 7f3ad1200640 1 -- 172.21.15.123:0/3964033628 >> 172.21.15.123:0/3964033628 conn(0x7f3acc09d9a0 msgr2=0x7f3acc0b95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:57.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.415+0000 7f3ad1200640 1 -- 172.21.15.123:0/3964033628 shutdown_connections 2024-04-24T05:46:57.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.415+0000 7f3ad1200640 1 -- 172.21.15.123:0/3964033628 wait complete. 2024-04-24T05:46:57.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.417+0000 7f3ad1200640 1 Processor -- start 2024-04-24T05:46:57.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.417+0000 7f3ad1200640 1 -- start start 2024-04-24T05:46:57.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ad1200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3acc0d3d70 0x7f3acc1f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:57.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ad1200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3acc0d4fb0 0x7f3acc0b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:57.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ad1200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3acc15eb20 0x7f3acc0b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ad1200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc0d2330 con 0x7f3acc0d3d70 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ad1200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc0d21b0 con 0x7f3acc0d4fb0 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ad1200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3acc0d24b0 con 0x7f3acc15eb20 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ac9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3acc0d3d70 0x7f3acc1f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ac9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3acc0d3d70 0x7f3acc1f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56708/0 (socket says 172.21.15.123:56708) 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ac9e00640 1 -- 172.21.15.123:0/832470396 learned_addr learned my addr 172.21.15.123:0/832470396 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3aca800640 1 --2- 172.21.15.123:0/832470396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3acc15eb20 0x7f3acc0b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ac9400640 1 --2- 172.21.15.123:0/832470396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3acc0d4fb0 0x7f3acc0b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ac9e00640 1 -- 172.21.15.123:0/832470396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3acc15eb20 msgr2=0x7f3acc0b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ac9e00640 1 --2- 172.21.15.123:0/832470396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3acc15eb20 0x7f3acc0b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ac9e00640 1 -- 172.21.15.123:0/832470396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3acc0d4fb0 msgr2=0x7f3acc0b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ac9e00640 1 --2- 172.21.15.123:0/832470396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3acc0d4fb0 0x7f3acc0b1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.419+0000 7f3ac9e00640 1 -- 172.21.15.123:0/832470396 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc1f4030 con 0x7f3acc0d3d70 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.420+0000 7f3ac9e00640 1 --2- 172.21.15.123:0/832470396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3acc0d3d70 0x7f3acc1f3af0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00c4b0 tx=0x7f3abc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:57.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.420+0000 7f3ac2000640 1 -- 172.21.15.123:0/832470396 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc017070 con 0x7f3acc0d3d70 2024-04-24T05:46:57.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.420+0000 7f3ac2000640 1 -- 172.21.15.123:0/832470396 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3abc005ce0 con 0x7f3acc0d3d70 2024-04-24T05:46:57.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.420+0000 7f3ac2000640 1 -- 172.21.15.123:0/832470396 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc007290 con 0x7f3acc0d3d70 2024-04-24T05:46:57.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.420+0000 7f3ad1200640 1 -- 172.21.15.123:0/832470396 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3acc15fc10 con 0x7f3acc0d3d70 2024-04-24T05:46:57.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.420+0000 7f3ad1200640 1 -- 172.21.15.123:0/832470396 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3acc1601d0 con 0x7f3acc0d3d70 2024-04-24T05:46:57.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.421+0000 7f3ad1200640 1 -- 172.21.15.123:0/832470396 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3acc069130 con 0x7f3acc0d3d70 2024-04-24T05:46:57.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.422+0000 7f3ac2000640 1 -- 172.21.15.123:0/832470396 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3abc01a070 con 0x7f3acc0d3d70 2024-04-24T05:46:57.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.423+0000 7f3ac2000640 1 --2- 172.21.15.123:0/832470396 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3aac06c8f0 0x7f3aac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:57.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.423+0000 7f3ac2000640 1 -- 172.21.15.123:0/832470396 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f3abc08b1e0 con 0x7f3acc0d3d70 2024-04-24T05:46:57.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.423+0000 7f3ac9400640 1 --2- 172.21.15.123:0/832470396 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3aac06c8f0 0x7f3aac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:57.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.424+0000 7f3ac9400640 1 --2- 172.21.15.123:0/832470396 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3aac06c8f0 0x7f3aac06edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8034000 tx=0x7f3ab802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:57.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.428+0000 7f3ac2000640 1 -- 172.21.15.123:0/832470396 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3abc059290 con 0x7f3acc0d3d70 2024-04-24T05:46:57.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.626+0000 7f3ad1200640 1 -- 172.21.15.123:0/832470396 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3acc074e70 con 0x7f3acc0d3d70 2024-04-24T05:46:57.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.627+0000 7f3ac2000640 1 -- 172.21.15.123:0/832470396 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f3abc058b90 con 0x7f3acc0d3d70 2024-04-24T05:46:57.627 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:57.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.629+0000 7f3ad1200640 1 -- 172.21.15.123:0/832470396 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3aac06c8f0 msgr2=0x7f3aac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:57.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.629+0000 7f3ad1200640 1 --2- 172.21.15.123:0/832470396 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3aac06c8f0 0x7f3aac06edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8034000 tx=0x7f3ab802e000 comp rx=0 tx=0).stop 2024-04-24T05:46:57.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.629+0000 7f3ad1200640 1 -- 172.21.15.123:0/832470396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3acc0d3d70 msgr2=0x7f3acc1f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:57.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.629+0000 7f3ad1200640 1 --2- 172.21.15.123:0/832470396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3acc0d3d70 0x7f3acc1f3af0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00c4b0 tx=0x7f3abc00c970 comp rx=0 tx=0).stop 2024-04-24T05:46:57.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.630+0000 7f3ad1200640 1 -- 172.21.15.123:0/832470396 shutdown_connections 2024-04-24T05:46:57.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.630+0000 7f3ad1200640 1 --2- 172.21.15.123:0/832470396 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3aac06c8f0 0x7f3aac06edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.630+0000 7f3ad1200640 1 --2- 172.21.15.123:0/832470396 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3acc15eb20 0x7f3acc0b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.630+0000 7f3ad1200640 1 --2- 172.21.15.123:0/832470396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3acc0d4fb0 0x7f3acc0b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.630+0000 7f3ad1200640 1 --2- 172.21.15.123:0/832470396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3acc0d3d70 0x7f3acc1f3af0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.630+0000 7f3ad1200640 1 -- 172.21.15.123:0/832470396 >> 172.21.15.123:0/832470396 conn(0x7f3acc09d9a0 msgr2=0x7f3acc0b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:57.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.630+0000 7f3ad1200640 1 -- 172.21.15.123:0/832470396 shutdown_connections 2024-04-24T05:46:57.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.630+0000 7f3ad1200640 1 -- 172.21.15.123:0/832470396 wait complete. 2024-04-24T05:46:57.648 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:46:57.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.843+0000 7f32ad800640 1 Processor -- start 2024-04-24T05:46:57.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.843+0000 7f32ad800640 1 -- start start 2024-04-24T05:46:57.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.845+0000 7f32ad800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f32a80d3e10 0x7f32a80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:57.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.845+0000 7f32ad800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f32a80d4c00 0x7f32a80d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:57.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.845+0000 7f32ad800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32a80cfdf0 con 0x7f32a815e970 2024-04-24T05:46:57.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.845+0000 7f32ad800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f32a80d09c0 con 0x7f32a80d4c00 2024-04-24T05:46:57.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.845+0000 7f32ad800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f32a80d0090 con 0x7f32a80d3e10 2024-04-24T05:46:57.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.845+0000 7f32a6a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f32a815e970 0x7f32a80d44f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:44078/0 (socket says 172.21.15.123:44078) 2024-04-24T05:46:57.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.845+0000 7f32a7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f32a80d4c00 0x7f32a80d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:57.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.845+0000 7f32a6a00640 1 -- 172.21.15.123:0/2182870884 learned_addr learned my addr 172.21.15.123:0/2182870884 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:46:57.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.846+0000 7f32a7400640 1 -- 172.21.15.123:0/2182870884 >> v2:172.21.15.155:3300/0 conn(0x7f32a80d3e10 msgr2=0x7f32a80d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:57.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.846+0000 7f32a6000640 1 --2- 172.21.15.123:0/2182870884 >> v2:172.21.15.155:3300/0 conn(0x7f32a80d3e10 0x7f32a80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:57.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.846+0000 7f32a7400640 1 --2- 172.21.15.123:0/2182870884 >> v2:172.21.15.155:3300/0 conn(0x7f32a80d3e10 0x7f32a80d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.846+0000 7f32a7400640 1 -- 172.21.15.123:0/2182870884 >> v1:172.21.15.123:6789/0 conn(0x7f32a815e970 legacy=0x7f32a80d44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:57.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.846+0000 7f32a5600640 1 -- 172.21.15.123:0/2182870884 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 882757899 0 0) 0x7f32a80cfdf0 con 0x7f32a815e970 2024-04-24T05:46:57.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.846+0000 7f32a7400640 1 -- 172.21.15.123:0/2182870884 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a80d0370 con 0x7f32a80d4c00 2024-04-24T05:46:57.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.847+0000 7f32a7400640 1 --2- 172.21.15.123:0/2182870884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a80d4c00 0x7f32a80d1390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f32a80c7f20 tx=0x7f32a002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7120abe36b68d68c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:57.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.847+0000 7f32a5600640 1 -- 172.21.15.123:0/2182870884 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0037070 con 0x7f32a80d4c00 2024-04-24T05:46:57.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.847+0000 7f32a5600640 1 -- 172.21.15.123:0/2182870884 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a0032440 con 0x7f32a80d4c00 2024-04-24T05:46:57.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.847+0000 7f32a5600640 1 -- 172.21.15.123:0/2182870884 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a002ae60 con 0x7f32a80d4c00 2024-04-24T05:46:57.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.848+0000 7f32ad800640 1 -- 172.21.15.123:0/2182870884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a80d4c00 msgr2=0x7f32a80d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:57.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.848+0000 7f32ad800640 1 --2- 172.21.15.123:0/2182870884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a80d4c00 0x7f32a80d1390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f32a80c7f20 tx=0x7f32a002a110 comp rx=0 tx=0).stop 2024-04-24T05:46:57.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.848+0000 7f32ad800640 1 -- 172.21.15.123:0/2182870884 shutdown_connections 2024-04-24T05:46:57.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.848+0000 7f32ad800640 1 --2- 172.21.15.123:0/2182870884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a80d4c00 0x7f32a80d1390 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.848+0000 7f32ad800640 1 --2- 172.21.15.123:0/2182870884 >> v2:172.21.15.155:3300/0 conn(0x7f32a80d3e10 0x7f32a80d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.848+0000 7f32ad800640 1 -- 172.21.15.123:0/2182870884 >> 172.21.15.123:0/2182870884 conn(0x7f32a80b9a40 msgr2=0x7f32a80b75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:57.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.848+0000 7f32ad800640 1 -- 172.21.15.123:0/2182870884 shutdown_connections 2024-04-24T05:46:57.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.848+0000 7f32ad800640 1 -- 172.21.15.123:0/2182870884 wait complete. 2024-04-24T05:46:57.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.850+0000 7f32ad800640 1 Processor -- start 2024-04-24T05:46:57.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.850+0000 7f32ad800640 1 -- start start 2024-04-24T05:46:57.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.851+0000 7f32ad800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32a80d3e10 0x7f32a814b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:57.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.851+0000 7f32ad800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32a80d4c00 0x7f32a806ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:57.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.851+0000 7f32ad800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a815e970 0x7f32a80b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:57.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.851+0000 7f32ad800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32a80b12b0 con 0x7f32a815e970 2024-04-24T05:46:57.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.851+0000 7f32ad800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f32a815ed50 con 0x7f32a80d3e10 2024-04-24T05:46:57.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.851+0000 7f32ad800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f32a80d53a0 con 0x7f32a80d4c00 2024-04-24T05:46:57.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.851+0000 7f32a7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a815e970 0x7f32a80b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:57.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.851+0000 7f32a7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a815e970 0x7f32a80b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56724/0 (socket says 172.21.15.123:56724) 2024-04-24T05:46:57.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.851+0000 7f32a7400640 1 -- 172.21.15.123:0/78388511 learned_addr learned my addr 172.21.15.123:0/78388511 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:57.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.851+0000 7f32a6000640 1 --2- 172.21.15.123:0/78388511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32a80d4c00 0x7f32a806ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:57.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.851+0000 7f32a6a00640 1 --2- 172.21.15.123:0/78388511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32a80d3e10 0x7f32a814b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:57.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.851+0000 7f32a7400640 1 -- 172.21.15.123:0/78388511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32a80d4c00 msgr2=0x7f32a806ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:57.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.851+0000 7f32a7400640 1 --2- 172.21.15.123:0/78388511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32a80d4c00 0x7f32a806ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.852+0000 7f32a7400640 1 -- 172.21.15.123:0/78388511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32a80d3e10 msgr2=0x7f32a814b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:57.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.852+0000 7f32a7400640 1 --2- 172.21.15.123:0/78388511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32a80d3e10 0x7f32a814b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:57.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.852+0000 7f32a7400640 1 -- 172.21.15.123:0/78388511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a814bcf0 con 0x7f32a815e970 2024-04-24T05:46:57.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.852+0000 7f32a7400640 1 --2- 172.21.15.123:0/78388511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a815e970 0x7f32a80b39b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f32a0035040 tx=0x7f32a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:57.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.852+0000 7f329ea00640 1 -- 172.21.15.123:0/78388511 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0037050 con 0x7f32a815e970 2024-04-24T05:46:57.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.852+0000 7f329ea00640 1 -- 172.21.15.123:0/78388511 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a0038040 con 0x7f32a815e970 2024-04-24T05:46:57.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.852+0000 7f329ea00640 1 -- 172.21.15.123:0/78388511 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0004490 con 0x7f32a815e970 2024-04-24T05:46:57.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.852+0000 7f32ad800640 1 -- 172.21.15.123:0/78388511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a806d430 con 0x7f32a815e970 2024-04-24T05:46:57.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.852+0000 7f32ad800640 1 -- 172.21.15.123:0/78388511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a806d9f0 con 0x7f32a815e970 2024-04-24T05:46:57.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.853+0000 7f32ad800640 1 -- 172.21.15.123:0/78388511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a8068a20 con 0x7f32a815e970 2024-04-24T05:46:57.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.854+0000 7f329ea00640 1 -- 172.21.15.123:0/78388511 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32a0030980 con 0x7f32a815e970 2024-04-24T05:46:57.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.855+0000 7f329ea00640 1 --2- 172.21.15.123:0/78388511 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f328806c8f0 0x7f328806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:57.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.855+0000 7f329ea00640 1 -- 172.21.15.123:0/78388511 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f32a00acc50 con 0x7f32a815e970 2024-04-24T05:46:57.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.856+0000 7f32a6a00640 1 --2- 172.21.15.123:0/78388511 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f328806c8f0 0x7f328806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:57.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.856+0000 7f32a6a00640 1 --2- 172.21.15.123:0/78388511 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f328806c8f0 0x7f328806edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f32980012e0 tx=0x7f3298009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:57.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:57.860+0000 7f329ea00640 1 -- 172.21.15.123:0/78388511 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32a0032e20 con 0x7f32a815e970 2024-04-24T05:46:58.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.054+0000 7f32ad800640 1 -- 172.21.15.123:0/78388511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f32a8161220 con 0x7f32a815e970 2024-04-24T05:46:58.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.054+0000 7f329ea00640 1 -- 172.21.15.123:0/78388511 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f32a007abd0 con 0x7f32a815e970 2024-04-24T05:46:58.055 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:46:58.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.057+0000 7f32ad800640 1 -- 172.21.15.123:0/78388511 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f328806c8f0 msgr2=0x7f328806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:58.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.057+0000 7f32ad800640 1 --2- 172.21.15.123:0/78388511 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f328806c8f0 0x7f328806edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f32980012e0 tx=0x7f3298009040 comp rx=0 tx=0).stop 2024-04-24T05:46:58.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.057+0000 7f32ad800640 1 -- 172.21.15.123:0/78388511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a815e970 msgr2=0x7f32a80b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:58.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.057+0000 7f32ad800640 1 --2- 172.21.15.123:0/78388511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a815e970 0x7f32a80b39b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f32a0035040 tx=0x7f32a00046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.057+0000 7f32ad800640 1 -- 172.21.15.123:0/78388511 shutdown_connections 2024-04-24T05:46:58.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.057+0000 7f32ad800640 1 --2- 172.21.15.123:0/78388511 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f328806c8f0 0x7f328806edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.057+0000 7f32ad800640 1 --2- 172.21.15.123:0/78388511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32a815e970 0x7f32a80b39b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.057+0000 7f32ad800640 1 --2- 172.21.15.123:0/78388511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32a80d4c00 0x7f32a806ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.057+0000 7f32ad800640 1 --2- 172.21.15.123:0/78388511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32a80d3e10 0x7f32a814b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.057+0000 7f32ad800640 1 -- 172.21.15.123:0/78388511 >> 172.21.15.123:0/78388511 conn(0x7f32a80b9a40 msgr2=0x7f32a815ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:58.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.057+0000 7f32ad800640 1 -- 172.21.15.123:0/78388511 shutdown_connections 2024-04-24T05:46:58.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.057+0000 7f32ad800640 1 -- 172.21.15.123:0/78388511 wait complete. 2024-04-24T05:46:58.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:46:58.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.273+0000 7fd015800640 1 Processor -- start 2024-04-24T05:46:58.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.273+0000 7fd015800640 1 -- start start 2024-04-24T05:46:58.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.275+0000 7fd015800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd01015ee30 0x7fd01006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:58.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.275+0000 7fd015800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd01015f3d0 con 0x7fd01006d7e0 2024-04-24T05:46:58.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.275+0000 7fd015800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd01015f6b0 con 0x7fd010158fe0 2024-04-24T05:46:58.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.275+0000 7fd015800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd01006e540 con 0x7fd01015ee30 2024-04-24T05:46:58.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.275+0000 7fd00e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd01015ee30 0x7fd01006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:58.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.275+0000 7fd00e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd01015ee30 0x7fd01006d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56730/0 (socket says 172.21.15.123:56730) 2024-04-24T05:46:58.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.275+0000 7fd00e000640 1 -- 172.21.15.123:0/3086991667 learned_addr learned my addr 172.21.15.123:0/3086991667 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:58.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.276+0000 7fd00e000640 1 -- 172.21.15.123:0/3086991667 >> v1:172.21.15.155:6789/0 conn(0x7fd010158fe0 legacy=0x7fd0100b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:58.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.276+0000 7fd00e000640 1 -- 172.21.15.123:0/3086991667 >> v1:172.21.15.123:6789/0 conn(0x7fd01006d7e0 legacy=0x7fd01006e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:58.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.276+0000 7fd00e000640 1 -- 172.21.15.123:0/3086991667 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01015f990 con 0x7fd01015ee30 2024-04-24T05:46:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.277+0000 7fd00e000640 1 --2- 172.21.15.123:0/3086991667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd01015ee30 0x7fd01006d2a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009e50 tx=0x7fcff802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64472ba6e7ea80bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.277+0000 7fd00d600640 1 -- 172.21.15.123:0/3086991667 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8036070 con 0x7fd01015ee30 2024-04-24T05:46:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.277+0000 7fd00d600640 1 -- 172.21.15.123:0/3086991667 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff8031600 con 0x7fd01015ee30 2024-04-24T05:46:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.277+0000 7fd00d600640 1 -- 172.21.15.123:0/3086991667 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff80319d0 con 0x7fd01015ee30 2024-04-24T05:46:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.278+0000 7fd015800640 1 -- 172.21.15.123:0/3086991667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd01015ee30 msgr2=0x7fd01006d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.278+0000 7fd015800640 1 --2- 172.21.15.123:0/3086991667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd01015ee30 0x7fd01006d2a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009e50 tx=0x7fcff802a760 comp rx=0 tx=0).stop 2024-04-24T05:46:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.278+0000 7fd015800640 1 -- 172.21.15.123:0/3086991667 shutdown_connections 2024-04-24T05:46:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.278+0000 7fd015800640 1 --2- 172.21.15.123:0/3086991667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd01015ee30 0x7fd01006d2a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.278+0000 7fd015800640 1 -- 172.21.15.123:0/3086991667 >> 172.21.15.123:0/3086991667 conn(0x7fd0100724e0 msgr2=0x7fd0100728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.278+0000 7fd015800640 1 -- 172.21.15.123:0/3086991667 shutdown_connections 2024-04-24T05:46:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.278+0000 7fd015800640 1 -- 172.21.15.123:0/3086991667 wait complete. 2024-04-24T05:46:58.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.280+0000 7fd015800640 1 Processor -- start 2024-04-24T05:46:58.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.280+0000 7fd015800640 1 -- start start 2024-04-24T05:46:58.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.281+0000 7fd015800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd01006d7e0 0x7fd0101587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:58.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.281+0000 7fd015800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd010158fe0 0x7fd010157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:58.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.281+0000 7fd015800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd01015ee30 0x7fd0101f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:58.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.281+0000 7fd015800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd01006f260 con 0x7fd01015ee30 2024-04-24T05:46:58.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.281+0000 7fd015800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd01006f0e0 con 0x7fd010158fe0 2024-04-24T05:46:58.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.281+0000 7fd015800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd01006f3e0 con 0x7fd01006d7e0 2024-04-24T05:46:58.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.281+0000 7fd00f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd01015ee30 0x7fd0101f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:58.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.281+0000 7fd00f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd01015ee30 0x7fd0101f54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56744/0 (socket says 172.21.15.123:56744) 2024-04-24T05:46:58.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.281+0000 7fd00f400640 1 -- 172.21.15.123:0/144380837 learned_addr learned my addr 172.21.15.123:0/144380837 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:58.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.281+0000 7fd00ea00640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd01006d7e0 0x7fd0101587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:58.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.281+0000 7fd00f400640 1 -- 172.21.15.123:0/144380837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd01006d7e0 msgr2=0x7fd0101587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:58.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.282+0000 7fd00f400640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd01006d7e0 0x7fd0101587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.282+0000 7fd00e000640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd010158fe0 0x7fd010157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:58.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.282+0000 7fd00f400640 1 -- 172.21.15.123:0/144380837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd010158fe0 msgr2=0x7fd010157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:58.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.282+0000 7fd00f400640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd010158fe0 0x7fd010157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.282+0000 7fd00f400640 1 -- 172.21.15.123:0/144380837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd010154340 con 0x7fd01015ee30 2024-04-24T05:46:58.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.282+0000 7fd00ea00640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd01006d7e0 0x7fd0101587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:58.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.282+0000 7fd00f400640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd01015ee30 0x7fd0101f54c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd00800c3f0 tx=0x7fd00800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:58.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.282+0000 7fd006a00640 1 -- 172.21.15.123:0/144380837 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0080052d0 con 0x7fd01015ee30 2024-04-24T05:46:58.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.282+0000 7fd006a00640 1 -- 172.21.15.123:0/144380837 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd008005490 con 0x7fd01015ee30 2024-04-24T05:46:58.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.282+0000 7fd015800640 1 -- 172.21.15.123:0/144380837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0101545e0 con 0x7fd01015ee30 2024-04-24T05:46:58.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.282+0000 7fd006a00640 1 -- 172.21.15.123:0/144380837 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008007260 con 0x7fd01015ee30 2024-04-24T05:46:58.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.282+0000 7fd015800640 1 -- 172.21.15.123:0/144380837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd010157e70 con 0x7fd01015ee30 2024-04-24T05:46:58.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.284+0000 7fd015800640 1 -- 172.21.15.123:0/144380837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd010068a20 con 0x7fd01015ee30 2024-04-24T05:46:58.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.285+0000 7fd006a00640 1 -- 172.21.15.123:0/144380837 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd008007400 con 0x7fd01015ee30 2024-04-24T05:46:58.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.286+0000 7fd006a00640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fcfec06c8f0 0x7fcfec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:58.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.286+0000 7fd006a00640 1 -- 172.21.15.123:0/144380837 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fd00808b3a0 con 0x7fd01015ee30 2024-04-24T05:46:58.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.287+0000 7fd00ea00640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fcfec06c8f0 0x7fcfec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:58.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.287+0000 7fd00ea00640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fcfec06c8f0 0x7fcfec06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd010155830 tx=0x7fd000009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:58.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.291+0000 7fd006a00640 1 -- 172.21.15.123:0/144380837 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0080593b0 con 0x7fd01015ee30 2024-04-24T05:46:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.494+0000 7fd015800640 1 -- 172.21.15.123:0/144380837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd01006eec0 con 0x7fd01015ee30 2024-04-24T05:46:58.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.495+0000 7fd006a00640 1 -- 172.21.15.123:0/144380837 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fd008058cb0 con 0x7fd01015ee30 2024-04-24T05:46:58.495 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:58.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.497+0000 7fd015800640 1 -- 172.21.15.123:0/144380837 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fcfec06c8f0 msgr2=0x7fcfec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:58.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.497+0000 7fd015800640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fcfec06c8f0 0x7fcfec06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd010155830 tx=0x7fd000009040 comp rx=0 tx=0).stop 2024-04-24T05:46:58.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.497+0000 7fd015800640 1 -- 172.21.15.123:0/144380837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd01015ee30 msgr2=0x7fd0101f54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:58.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.497+0000 7fd015800640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd01015ee30 0x7fd0101f54c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd00800c3f0 tx=0x7fd00800c8b0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.497+0000 7fd015800640 1 -- 172.21.15.123:0/144380837 shutdown_connections 2024-04-24T05:46:58.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.497+0000 7fd015800640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fcfec06c8f0 0x7fcfec06edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.497+0000 7fd015800640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd01015ee30 0x7fd0101f54c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.497+0000 7fd015800640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd010158fe0 0x7fd010157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.497+0000 7fd015800640 1 --2- 172.21.15.123:0/144380837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd01006d7e0 0x7fd0101587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.497+0000 7fd015800640 1 -- 172.21.15.123:0/144380837 >> 172.21.15.123:0/144380837 conn(0x7fd0100724e0 msgr2=0x7fd0100b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:58.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.498+0000 7fd015800640 1 -- 172.21.15.123:0/144380837 shutdown_connections 2024-04-24T05:46:58.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.498+0000 7fd015800640 1 -- 172.21.15.123:0/144380837 wait complete. 2024-04-24T05:46:58.545 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-04-24T05:46:58.545 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:46:58.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.738+0000 7ffb60800640 1 Processor -- start 2024-04-24T05:46:58.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.738+0000 7ffb60800640 1 -- start start 2024-04-24T05:46:58.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.740+0000 7ffb60800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffb5815e910 0x7ffb580d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:58.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.740+0000 7ffb60800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb580d5050 0x7ffb580cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:58.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.740+0000 7ffb60800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb580d3fb0 con 0x7ffb580d1cf0 2024-04-24T05:46:58.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.740+0000 7ffb60800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ffb580b6090 con 0x7ffb5815e910 2024-04-24T05:46:58.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.740+0000 7ffb60800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffb580b6fb0 con 0x7ffb580d5050 2024-04-24T05:46:58.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.740+0000 7ffb5de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffb5815e910 0x7ffb580d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:58.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.740+0000 7ffb5de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffb5815e910 0x7ffb580d17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56756/0 (socket says 172.21.15.123:56756) 2024-04-24T05:46:58.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.740+0000 7ffb5de00640 1 -- 172.21.15.123:0/3323190660 learned_addr learned my addr 172.21.15.123:0/3323190660 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:58.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.740+0000 7ffb5e800640 1 --2- 172.21.15.123:0/3323190660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb580d5050 0x7ffb580cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:58.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.741+0000 7ffb5de00640 1 -- 172.21.15.123:0/3323190660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb580d5050 msgr2=0x7ffb580cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:58.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.741+0000 7ffb5de00640 1 --2- 172.21.15.123:0/3323190660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb580d5050 0x7ffb580cfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.741+0000 7ffb5de00640 1 -- 172.21.15.123:0/3323190660 >> v1:172.21.15.155:6789/0 conn(0x7ffb580d1cf0 legacy=0x7ffb580d4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:58.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.741+0000 7ffb5de00640 1 -- 172.21.15.123:0/3323190660 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb580b6330 con 0x7ffb5815e910 2024-04-24T05:46:58.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.742+0000 7ffb5de00640 1 --2- 172.21.15.123:0/3323190660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb5815e910 0x7ffb580d17b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009e50 tx=0x7ffb5002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f41b081c65145f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:58.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.742+0000 7ffb5ca00640 1 -- 172.21.15.123:0/3323190660 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50036070 con 0x7ffb5815e910 2024-04-24T05:46:58.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.742+0000 7ffb5ca00640 1 -- 172.21.15.123:0/3323190660 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb50031600 con 0x7ffb5815e910 2024-04-24T05:46:58.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.742+0000 7ffb5ca00640 1 -- 172.21.15.123:0/3323190660 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb500319d0 con 0x7ffb5815e910 2024-04-24T05:46:58.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.743+0000 7ffb60800640 1 -- 172.21.15.123:0/3323190660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb5815e910 msgr2=0x7ffb580d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:58.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.743+0000 7ffb60800640 1 --2- 172.21.15.123:0/3323190660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb5815e910 0x7ffb580d17b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009e50 tx=0x7ffb5002a6c0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.743+0000 7ffb60800640 1 -- 172.21.15.123:0/3323190660 shutdown_connections 2024-04-24T05:46:58.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.743+0000 7ffb60800640 1 --2- 172.21.15.123:0/3323190660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb580d5050 0x7ffb580cfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.743+0000 7ffb60800640 1 --2- 172.21.15.123:0/3323190660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb5815e910 0x7ffb580d17b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.743+0000 7ffb60800640 1 -- 172.21.15.123:0/3323190660 >> 172.21.15.123:0/3323190660 conn(0x7ffb580b95d0 msgr2=0x7ffb580b99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:58.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.743+0000 7ffb60800640 1 -- 172.21.15.123:0/3323190660 shutdown_connections 2024-04-24T05:46:58.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.743+0000 7ffb60800640 1 -- 172.21.15.123:0/3323190660 wait complete. 2024-04-24T05:46:58.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.745+0000 7ffb60800640 1 Processor -- start 2024-04-24T05:46:58.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.745+0000 7ffb60800640 1 -- start start 2024-04-24T05:46:58.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.746+0000 7ffb60800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb580d1cf0 0x7ffb581580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:58.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.746+0000 7ffb60800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb580d5050 0x7ffb5806e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:58.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.746+0000 7ffb60800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb5815e910 0x7ffb580c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:58.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.746+0000 7ffb60800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb580b6bd0 con 0x7ffb5815e910 2024-04-24T05:46:58.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.746+0000 7ffb60800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffb580b68f0 con 0x7ffb580d1cf0 2024-04-24T05:46:58.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.746+0000 7ffb60800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffb580b11c0 con 0x7ffb580d5050 2024-04-24T05:46:58.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.747+0000 7ffb5e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb5815e910 0x7ffb580c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:58.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.747+0000 7ffb5e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb5815e910 0x7ffb580c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56758/0 (socket says 172.21.15.123:56758) 2024-04-24T05:46:58.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.747+0000 7ffb5e800640 1 -- 172.21.15.123:0/687157285 learned_addr learned my addr 172.21.15.123:0/687157285 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:58.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.747+0000 7ffb5de00640 1 --2- 172.21.15.123:0/687157285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb580d1cf0 0x7ffb581580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:58.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.747+0000 7ffb5e800640 1 -- 172.21.15.123:0/687157285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb580d5050 msgr2=0x7ffb5806e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:58.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.747+0000 7ffb5e800640 1 --2- 172.21.15.123:0/687157285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb580d5050 0x7ffb5806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.747+0000 7ffb5e800640 1 -- 172.21.15.123:0/687157285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb580d1cf0 msgr2=0x7ffb581580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:58.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.747+0000 7ffb5e800640 1 --2- 172.21.15.123:0/687157285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb580d1cf0 0x7ffb581580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.747+0000 7ffb5e800640 1 -- 172.21.15.123:0/687157285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5806d2f0 con 0x7ffb5815e910 2024-04-24T05:46:58.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.747+0000 7ffb5de00640 1 --2- 172.21.15.123:0/687157285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb580d1cf0 0x7ffb581580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:58.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.747+0000 7ffb5e800640 1 --2- 172.21.15.123:0/687157285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb5815e910 0x7ffb580c9220 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400d5d0 tx=0x7ffb5400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:58.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.747+0000 7ffb4e000640 1 -- 172.21.15.123:0/687157285 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb540125f0 con 0x7ffb5815e910 2024-04-24T05:46:58.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.748+0000 7ffb4e000640 1 -- 172.21.15.123:0/687157285 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb54019070 con 0x7ffb5815e910 2024-04-24T05:46:58.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.748+0000 7ffb4e000640 1 -- 172.21.15.123:0/687157285 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb540129d0 con 0x7ffb5815e910 2024-04-24T05:46:58.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.748+0000 7ffb60800640 1 -- 172.21.15.123:0/687157285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5806d5d0 con 0x7ffb5815e910 2024-04-24T05:46:58.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.748+0000 7ffb60800640 1 -- 172.21.15.123:0/687157285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb5806db90 con 0x7ffb5815e910 2024-04-24T05:46:58.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.749+0000 7ffb60800640 1 -- 172.21.15.123:0/687157285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb58068a20 con 0x7ffb5815e910 2024-04-24T05:46:58.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.750+0000 7ffb4e000640 1 -- 172.21.15.123:0/687157285 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb54012b70 con 0x7ffb5815e910 2024-04-24T05:46:58.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.750+0000 7ffb4e000640 1 --2- 172.21.15.123:0/687157285 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:58.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.751+0000 7ffb4e000640 1 -- 172.21.15.123:0/687157285 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7ffb5408c430 con 0x7ffb5815e910 2024-04-24T05:46:58.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.751+0000 7ffb5de00640 1 --2- 172.21.15.123:0/687157285 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:58.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.751+0000 7ffb5de00640 1 --2- 172.21.15.123:0/687157285 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffb50033000 tx=0x7ffb50009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:58.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.755+0000 7ffb4e000640 1 -- 172.21.15.123:0/687157285 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb5405a3b0 con 0x7ffb5815e910 2024-04-24T05:46:58.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.940+0000 7ffb60800640 1 -- 172.21.15.123:0/687157285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb58161f20 con 0x7ffb5815e910 2024-04-24T05:46:58.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.944+0000 7ffb4e000640 1 -- 172.21.15.123:0/687157285 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7ffb54059cb0 con 0x7ffb5815e910 2024-04-24T05:46:58.945 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:46:58.945 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:46:58.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.946+0000 7ffb60800640 1 -- 172.21.15.123:0/687157285 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ffb3c06c8f0 msgr2=0x7ffb3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:58.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.946+0000 7ffb60800640 1 --2- 172.21.15.123:0/687157285 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffb50033000 tx=0x7ffb50009000 comp rx=0 tx=0).stop 2024-04-24T05:46:58.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.946+0000 7ffb60800640 1 -- 172.21.15.123:0/687157285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb5815e910 msgr2=0x7ffb580c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:58.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.946+0000 7ffb60800640 1 --2- 172.21.15.123:0/687157285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb5815e910 0x7ffb580c9220 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400d5d0 tx=0x7ffb5400da90 comp rx=0 tx=0).stop 2024-04-24T05:46:58.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.946+0000 7ffb60800640 1 -- 172.21.15.123:0/687157285 shutdown_connections 2024-04-24T05:46:58.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.946+0000 7ffb60800640 1 --2- 172.21.15.123:0/687157285 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ffb3c06c8f0 0x7ffb3c06edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.946+0000 7ffb60800640 1 --2- 172.21.15.123:0/687157285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb5815e910 0x7ffb580c9220 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.946+0000 7ffb60800640 1 --2- 172.21.15.123:0/687157285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb580d5050 0x7ffb5806e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.946+0000 7ffb60800640 1 --2- 172.21.15.123:0/687157285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb580d1cf0 0x7ffb581580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:58.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.946+0000 7ffb60800640 1 -- 172.21.15.123:0/687157285 >> 172.21.15.123:0/687157285 conn(0x7ffb580b95d0 msgr2=0x7ffb580b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:58.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.947+0000 7ffb60800640 1 -- 172.21.15.123:0/687157285 shutdown_connections 2024-04-24T05:46:58.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:58.947+0000 7ffb60800640 1 -- 172.21.15.123:0/687157285 wait complete. 2024-04-24T05:46:58.966 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:46:58.966 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:46:59.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.163+0000 7f24e5c00640 1 Processor -- start 2024-04-24T05:46:59.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.164+0000 7f24e5c00640 1 -- start start 2024-04-24T05:46:59.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.165+0000 7f24e5c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f24e015e5f0 0x7f24e015f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:59.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.165+0000 7f24e5c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24e0162a60 con 0x7f24e0162250 2024-04-24T05:46:59.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.165+0000 7f24e5c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24e0162d40 con 0x7f24e00d15f0 2024-04-24T05:46:59.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.165+0000 7f24e5c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f24e00ceb50 con 0x7f24e015e5f0 2024-04-24T05:46:59.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.166+0000 7f24dfe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f24e015e5f0 0x7f24e015f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:59.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.166+0000 7f24dfe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f24e015e5f0 0x7f24e015f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42440/0 (socket says 172.21.15.123:42440) 2024-04-24T05:46:59.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.166+0000 7f24dfe00640 1 -- 172.21.15.123:0/1812096363 learned_addr learned my addr 172.21.15.123:0/1812096363 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:59.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.166+0000 7f24d7e00640 1 --1- 172.21.15.123:0/1812096363 >> v1:172.21.15.123:6789/0 conn(0x7f24e0162250 0x7f24e00d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49324/0 (socket says 172.21.15.123:49324) 2024-04-24T05:46:59.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.166+0000 7f24dfe00640 1 -- 172.21.15.123:0/1812096363 >> v1:172.21.15.155:6789/0 conn(0x7f24e00d15f0 legacy=0x7f24e00cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:59.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.166+0000 7f24dfe00640 1 -- 172.21.15.123:0/1812096363 >> v1:172.21.15.123:6789/0 conn(0x7f24e0162250 legacy=0x7f24e00d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:59.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.166+0000 7f24dfe00640 1 -- 172.21.15.123:0/1812096363 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e0163020 con 0x7f24e015e5f0 2024-04-24T05:46:59.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.166+0000 7f24df400640 1 -- 172.21.15.123:0/1812096363 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 447903480 0 0) 0x7f24e0162a60 con 0x7f24e0162250 2024-04-24T05:46:59.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.167+0000 7f24dfe00640 1 --2- 172.21.15.123:0/1812096363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e015e5f0 0x7f24e015f870 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009870 tx=0x7f24d002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1221c6ce86b6584 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:59.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.167+0000 7f24df400640 1 -- 172.21.15.123:0/1812096363 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0037070 con 0x7f24e015e5f0 2024-04-24T05:46:59.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.167+0000 7f24df400640 1 -- 172.21.15.123:0/1812096363 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24d0032440 con 0x7f24e015e5f0 2024-04-24T05:46:59.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.167+0000 7f24df400640 1 -- 172.21.15.123:0/1812096363 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0032810 con 0x7f24e015e5f0 2024-04-24T05:46:59.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.168+0000 7f24e5c00640 1 -- 172.21.15.123:0/1812096363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e015e5f0 msgr2=0x7f24e015f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:59.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.168+0000 7f24e5c00640 1 --2- 172.21.15.123:0/1812096363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e015e5f0 0x7f24e015f870 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009870 tx=0x7f24d002a2a0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.168+0000 7f24e5c00640 1 -- 172.21.15.123:0/1812096363 shutdown_connections 2024-04-24T05:46:59.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.168+0000 7f24e5c00640 1 --2- 172.21.15.123:0/1812096363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e015e5f0 0x7f24e015f870 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.168+0000 7f24e5c00640 1 -- 172.21.15.123:0/1812096363 >> 172.21.15.123:0/1812096363 conn(0x7f24e00b9db0 msgr2=0x7f24e009d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:59.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.168+0000 7f24e5c00640 1 -- 172.21.15.123:0/1812096363 shutdown_connections 2024-04-24T05:46:59.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.168+0000 7f24e5c00640 1 -- 172.21.15.123:0/1812096363 wait complete. 2024-04-24T05:46:59.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.170+0000 7f24e5c00640 1 Processor -- start 2024-04-24T05:46:59.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.170+0000 7f24e5c00640 1 -- start start 2024-04-24T05:46:59.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.171+0000 7f24e5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e00d15f0 0x7f24e01f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:59.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.171+0000 7f24e5c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24e015e5f0 0x7f24e00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:59.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.171+0000 7f24e5c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e0162250 0x7f24e00b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:59.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.171+0000 7f24e5c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24e015e9d0 con 0x7f24e00d15f0 2024-04-24T05:46:59.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.171+0000 7f24e5c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f24e00cee30 con 0x7f24e0162250 2024-04-24T05:46:59.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.171+0000 7f24e5c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f24e015eb50 con 0x7f24e015e5f0 2024-04-24T05:46:59.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.171+0000 7f24dfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e00d15f0 0x7f24e01f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:59.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.171+0000 7f24dfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e00d15f0 0x7f24e01f4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42454/0 (socket says 172.21.15.123:42454) 2024-04-24T05:46:59.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.171+0000 7f24dfe00640 1 -- 172.21.15.123:0/3510251820 learned_addr learned my addr 172.21.15.123:0/3510251820 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:59.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.171+0000 7f24dfe00640 1 -- 172.21.15.123:0/3510251820 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24e015e5f0 msgr2=0x7f24e00b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:59.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.171+0000 7f24dfe00640 1 --2- 172.21.15.123:0/3510251820 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24e015e5f0 0x7f24e00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.171+0000 7f24dfe00640 1 -- 172.21.15.123:0/3510251820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e0162250 msgr2=0x7f24e00b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:59.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.171+0000 7f24dfe00640 1 --2- 172.21.15.123:0/3510251820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e0162250 0x7f24e00b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.171+0000 7f24dfe00640 1 -- 172.21.15.123:0/3510251820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e0150910 con 0x7f24e00d15f0 2024-04-24T05:46:59.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.172+0000 7f24dfe00640 1 --2- 172.21.15.123:0/3510251820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e00d15f0 0x7f24e01f4820 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009870 tx=0x7f24d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:59.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.172+0000 7f24dcc00640 1 -- 172.21.15.123:0/3510251820 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0041070 con 0x7f24e00d15f0 2024-04-24T05:46:59.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.172+0000 7f24dcc00640 1 -- 172.21.15.123:0/3510251820 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24d003c870 con 0x7f24e00d15f0 2024-04-24T05:46:59.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.172+0000 7f24dcc00640 1 -- 172.21.15.123:0/3510251820 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0037040 con 0x7f24e00d15f0 2024-04-24T05:46:59.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.172+0000 7f24e5c00640 1 -- 172.21.15.123:0/3510251820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e0150bf0 con 0x7f24e00d15f0 2024-04-24T05:46:59.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.172+0000 7f24e5c00640 1 -- 172.21.15.123:0/3510251820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e01511b0 con 0x7f24e00d15f0 2024-04-24T05:46:59.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.173+0000 7f24e5c00640 1 -- 172.21.15.123:0/3510251820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e0068a20 con 0x7f24e00d15f0 2024-04-24T05:46:59.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.174+0000 7f24dcc00640 1 -- 172.21.15.123:0/3510251820 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24d0044070 con 0x7f24e00d15f0 2024-04-24T05:46:59.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.175+0000 7f24dcc00640 1 --2- 172.21.15.123:0/3510251820 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f24bc06c8f0 0x7f24bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:59.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.175+0000 7f24dcc00640 1 -- 172.21.15.123:0/3510251820 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f24d00aca20 con 0x7f24e00d15f0 2024-04-24T05:46:59.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.175+0000 7f24d7e00640 1 --2- 172.21.15.123:0/3510251820 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f24bc06c8f0 0x7f24bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:59.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.176+0000 7f24d7e00640 1 --2- 172.21.15.123:0/3510251820 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f24bc06c8f0 0x7f24bc06edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f24c8007fd0 tx=0x7f24c8007920 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:59.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.179+0000 7f24dcc00640 1 -- 172.21.15.123:0/3510251820 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24d007aa40 con 0x7f24e00d15f0 2024-04-24T05:46:59.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.378+0000 7f24e5c00640 1 -- 172.21.15.123:0/3510251820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f24e00c7de0 con 0x7f24e00d15f0 2024-04-24T05:46:59.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.379+0000 7f24dcc00640 1 -- 172.21.15.123:0/3510251820 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f24d007a340 con 0x7f24e00d15f0 2024-04-24T05:46:59.380 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:46:59.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.382+0000 7f24e5c00640 1 -- 172.21.15.123:0/3510251820 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f24bc06c8f0 msgr2=0x7f24bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:59.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.382+0000 7f24e5c00640 1 --2- 172.21.15.123:0/3510251820 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f24bc06c8f0 0x7f24bc06edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f24c8007fd0 tx=0x7f24c8007920 comp rx=0 tx=0).stop 2024-04-24T05:46:59.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.382+0000 7f24e5c00640 1 -- 172.21.15.123:0/3510251820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e00d15f0 msgr2=0x7f24e01f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:59.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.382+0000 7f24e5c00640 1 --2- 172.21.15.123:0/3510251820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e00d15f0 0x7f24e01f4820 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009870 tx=0x7f24d00046c0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.382+0000 7f24e5c00640 1 -- 172.21.15.123:0/3510251820 shutdown_connections 2024-04-24T05:46:59.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.382+0000 7f24e5c00640 1 --2- 172.21.15.123:0/3510251820 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f24bc06c8f0 0x7f24bc06edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.382+0000 7f24e5c00640 1 --2- 172.21.15.123:0/3510251820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24e0162250 0x7f24e00b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.382+0000 7f24e5c00640 1 --2- 172.21.15.123:0/3510251820 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24e015e5f0 0x7f24e00b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.382+0000 7f24e5c00640 1 --2- 172.21.15.123:0/3510251820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e00d15f0 0x7f24e01f4820 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.382+0000 7f24e5c00640 1 -- 172.21.15.123:0/3510251820 >> 172.21.15.123:0/3510251820 conn(0x7f24e00b9db0 msgr2=0x7f24e01607a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:59.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.382+0000 7f24e5c00640 1 -- 172.21.15.123:0/3510251820 shutdown_connections 2024-04-24T05:46:59.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.383+0000 7f24e5c00640 1 -- 172.21.15.123:0/3510251820 wait complete. 2024-04-24T05:46:59.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:46:59.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.597+0000 7f19f1000640 1 Processor -- start 2024-04-24T05:46:59.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.597+0000 7f19f1000640 1 -- start start 2024-04-24T05:46:59.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.599+0000 7f19f1000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f19ec15e5f0 0x7f19ec15f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:59.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.599+0000 7f19f1000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f19ec0d15f0 0x7f19ec0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:59.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.599+0000 7f19f1000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19ec162a60 con 0x7f19ec162250 2024-04-24T05:46:59.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.599+0000 7f19f1000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f19ec0ceb50 con 0x7f19ec0d15f0 2024-04-24T05:46:59.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.599+0000 7f19f1000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f19ec0cee30 con 0x7f19ec15e5f0 2024-04-24T05:46:59.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.599+0000 7f19ebe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f19ec0d15f0 0x7f19ec0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:59.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.599+0000 7f19ebe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f19ec0d15f0 0x7f19ec0cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42462/0 (socket says 172.21.15.123:42462) 2024-04-24T05:46:59.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.599+0000 7f19ebe00640 1 -- 172.21.15.123:0/218278183 learned_addr learned my addr 172.21.15.123:0/218278183 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:59.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.600+0000 7f19eb400640 1 --2- 172.21.15.123:0/218278183 >> v2:172.21.15.155:3300/0 conn(0x7f19ec15e5f0 0x7f19ec15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:59.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.600+0000 7f19ebe00640 1 -- 172.21.15.123:0/218278183 >> v2:172.21.15.155:3300/0 conn(0x7f19ec15e5f0 msgr2=0x7f19ec15f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:59.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.600+0000 7f19ebe00640 1 --2- 172.21.15.123:0/218278183 >> v2:172.21.15.155:3300/0 conn(0x7f19ec15e5f0 0x7f19ec15f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.600+0000 7f19ebe00640 1 -- 172.21.15.123:0/218278183 >> v1:172.21.15.155:6789/0 conn(0x7f19ec162250 legacy=0x7f19ec0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:46:59.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.600+0000 7f19ebe00640 1 -- 172.21.15.123:0/218278183 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec162d40 con 0x7f19ec0d15f0 2024-04-24T05:46:59.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.600+0000 7f19eb400640 1 --2- 172.21.15.123:0/218278183 >> v2:172.21.15.155:3300/0 conn(0x7f19ec15e5f0 0x7f19ec15f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:46:59.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.601+0000 7f19ebe00640 1 --2- 172.21.15.123:0/218278183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ec0d15f0 0x7f19ec0cf2c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0cecd0 tx=0x7f19e402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d029d5ba94799d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:59.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.601+0000 7f19eaa00640 1 -- 172.21.15.123:0/218278183 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e4036070 con 0x7f19ec0d15f0 2024-04-24T05:46:59.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.601+0000 7f19eaa00640 1 -- 172.21.15.123:0/218278183 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e4031600 con 0x7f19ec0d15f0 2024-04-24T05:46:59.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.601+0000 7f19eaa00640 1 -- 172.21.15.123:0/218278183 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e40319d0 con 0x7f19ec0d15f0 2024-04-24T05:46:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.602+0000 7f19f1000640 1 -- 172.21.15.123:0/218278183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ec0d15f0 msgr2=0x7f19ec0cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.602+0000 7f19f1000640 1 --2- 172.21.15.123:0/218278183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ec0d15f0 0x7f19ec0cf2c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0cecd0 tx=0x7f19e402a6d0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.602+0000 7f19f1000640 1 -- 172.21.15.123:0/218278183 shutdown_connections 2024-04-24T05:46:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.602+0000 7f19f1000640 1 --2- 172.21.15.123:0/218278183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ec0d15f0 0x7f19ec0cf2c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.602+0000 7f19f1000640 1 --2- 172.21.15.123:0/218278183 >> v2:172.21.15.155:3300/0 conn(0x7f19ec15e5f0 0x7f19ec15f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.602+0000 7f19f1000640 1 -- 172.21.15.123:0/218278183 >> 172.21.15.123:0/218278183 conn(0x7f19ec0b9db0 msgr2=0x7f19ec09d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.602+0000 7f19f1000640 1 -- 172.21.15.123:0/218278183 shutdown_connections 2024-04-24T05:46:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.602+0000 7f19f1000640 1 -- 172.21.15.123:0/218278183 wait complete. 2024-04-24T05:46:59.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.603+0000 7f19f1000640 1 Processor -- start 2024-04-24T05:46:59.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.604+0000 7f19f1000640 1 -- start start 2024-04-24T05:46:59.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.604+0000 7f19f1000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ec0d15f0 0x7f19ec1598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:59.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.604+0000 7f19f1000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19ec15e5f0 0x7f19ec0c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:59.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.604+0000 7f19f1000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19ec162250 0x7f19ec160e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:59.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.604+0000 7f19f1000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec15e9d0 con 0x7f19ec0d15f0 2024-04-24T05:46:59.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.604+0000 7f19f1000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec1626d0 con 0x7f19ec162250 2024-04-24T05:46:59.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.604+0000 7f19f1000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f19ec15eb50 con 0x7f19ec15e5f0 2024-04-24T05:46:59.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.605+0000 7f19eb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ec0d15f0 0x7f19ec1598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:59.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.605+0000 7f19eb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ec0d15f0 0x7f19ec1598c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42466/0 (socket says 172.21.15.123:42466) 2024-04-24T05:46:59.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.605+0000 7f19eb400640 1 -- 172.21.15.123:0/1983907299 learned_addr learned my addr 172.21.15.123:0/1983907299 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:46:59.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.605+0000 7f19e2a00640 1 --2- 172.21.15.123:0/1983907299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19ec15e5f0 0x7f19ec0c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:59.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.605+0000 7f19eb400640 1 -- 172.21.15.123:0/1983907299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19ec15e5f0 msgr2=0x7f19ec0c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:59.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.605+0000 7f19ebe00640 1 --2- 172.21.15.123:0/1983907299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19ec162250 0x7f19ec160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:59.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.605+0000 7f19eb400640 1 --2- 172.21.15.123:0/1983907299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19ec15e5f0 0x7f19ec0c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.605+0000 7f19eb400640 1 -- 172.21.15.123:0/1983907299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19ec162250 msgr2=0x7f19ec160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:59.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.605+0000 7f19eb400640 1 --2- 172.21.15.123:0/1983907299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19ec162250 0x7f19ec160e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.605+0000 7f19eb400640 1 -- 172.21.15.123:0/1983907299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec0c8ad0 con 0x7f19ec0d15f0 2024-04-24T05:46:59.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.605+0000 7f19eb400640 1 --2- 172.21.15.123:0/1983907299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ec0d15f0 0x7f19ec1598c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00c7f0 tx=0x7f19dc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:59.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.606+0000 7f19e3e00640 1 -- 172.21.15.123:0/1983907299 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc015070 con 0x7f19ec0d15f0 2024-04-24T05:46:59.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.606+0000 7f19e3e00640 1 -- 172.21.15.123:0/1983907299 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19dc007a30 con 0x7f19ec0d15f0 2024-04-24T05:46:59.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.606+0000 7f19f1000640 1 -- 172.21.15.123:0/1983907299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ec0c8db0 con 0x7f19ec0d15f0 2024-04-24T05:46:59.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.606+0000 7f19e3e00640 1 -- 172.21.15.123:0/1983907299 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc007e00 con 0x7f19ec0d15f0 2024-04-24T05:46:59.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.606+0000 7f19f1000640 1 -- 172.21.15.123:0/1983907299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ec0c9650 con 0x7f19ec0d15f0 2024-04-24T05:46:59.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.607+0000 7f19f1000640 1 -- 172.21.15.123:0/1983907299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19ec0c8f80 con 0x7f19ec0d15f0 2024-04-24T05:46:59.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.608+0000 7f19e3e00640 1 -- 172.21.15.123:0/1983907299 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19dc018070 con 0x7f19ec0d15f0 2024-04-24T05:46:59.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.609+0000 7f19e3e00640 1 --2- 172.21.15.123:0/1983907299 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f19c806c920 0x7f19c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:46:59.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.609+0000 7f19e3e00640 1 -- 172.21.15.123:0/1983907299 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f19dc08bfe0 con 0x7f19ec0d15f0 2024-04-24T05:46:59.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.609+0000 7f19e2a00640 1 --2- 172.21.15.123:0/1983907299 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f19c806c920 0x7f19c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:46:59.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.610+0000 7f19e2a00640 1 --2- 172.21.15.123:0/1983907299 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f19c806c920 0x7f19c806ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f19ec1f7d80 tx=0x7f19d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:46:59.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.613+0000 7f19e3e00640 1 -- 172.21.15.123:0/1983907299 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19dc059f60 con 0x7f19ec0d15f0 2024-04-24T05:46:59.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.796+0000 7f19f1000640 1 -- 172.21.15.123:0/1983907299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f19ec0c9820 con 0x7f19ec0d15f0 2024-04-24T05:46:59.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.797+0000 7f19e3e00640 1 -- 172.21.15.123:0/1983907299 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f19dc059860 con 0x7f19ec0d15f0 2024-04-24T05:46:59.799 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:46:59.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.800+0000 7f19f1000640 1 -- 172.21.15.123:0/1983907299 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f19c806c920 msgr2=0x7f19c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:59.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.800+0000 7f19f1000640 1 --2- 172.21.15.123:0/1983907299 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f19c806c920 0x7f19c806ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f19ec1f7d80 tx=0x7f19d4008040 comp rx=0 tx=0).stop 2024-04-24T05:46:59.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.800+0000 7f19f1000640 1 -- 172.21.15.123:0/1983907299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ec0d15f0 msgr2=0x7f19ec1598c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:46:59.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.800+0000 7f19f1000640 1 --2- 172.21.15.123:0/1983907299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ec0d15f0 0x7f19ec1598c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00c7f0 tx=0x7f19dc00cbf0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.800+0000 7f19f1000640 1 -- 172.21.15.123:0/1983907299 shutdown_connections 2024-04-24T05:46:59.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.800+0000 7f19f1000640 1 --2- 172.21.15.123:0/1983907299 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f19c806c920 0x7f19c806ede0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.800+0000 7f19f1000640 1 --2- 172.21.15.123:0/1983907299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19ec162250 0x7f19ec160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.800+0000 7f19f1000640 1 --2- 172.21.15.123:0/1983907299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19ec15e5f0 0x7f19ec0c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.800+0000 7f19f1000640 1 --2- 172.21.15.123:0/1983907299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ec0d15f0 0x7f19ec1598c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:46:59.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.800+0000 7f19f1000640 1 -- 172.21.15.123:0/1983907299 >> 172.21.15.123:0/1983907299 conn(0x7f19ec0b9db0 msgr2=0x7f19ec15ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:46:59.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.800+0000 7f19f1000640 1 -- 172.21.15.123:0/1983907299 shutdown_connections 2024-04-24T05:46:59.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:46:59.800+0000 7f19f1000640 1 -- 172.21.15.123:0/1983907299 wait complete. 2024-04-24T05:46:59.819 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:47:00.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.011+0000 7f4121200640 1 Processor -- start 2024-04-24T05:47:00.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.011+0000 7f4121200640 1 -- start start 2024-04-24T05:47:00.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.012+0000 7f4121200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f411c0ae990 0x7f411c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.012+0000 7f4121200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f411c0ce7e0 0x7f411c06e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.012+0000 7f4121200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f411c0d0740 con 0x7f411c06c410 2024-04-24T05:47:00.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.012+0000 7f4121200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f411c06dcb0 con 0x7f411c0ae990 2024-04-24T05:47:00.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.012+0000 7f4121200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f411c06e270 con 0x7f411c0ce7e0 2024-04-24T05:47:00.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.013+0000 7f4119e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f411c0ae990 0x7f411c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.013+0000 7f4119e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f411c0ae990 0x7f411c0d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42476/0 (socket says 172.21.15.123:42476) 2024-04-24T05:47:00.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.013+0000 7f4119e00640 1 -- 172.21.15.123:0/2484479953 learned_addr learned my addr 172.21.15.123:0/2484479953 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:00.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.013+0000 7f411a800640 1 --2- 172.21.15.123:0/2484479953 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f411c0ce7e0 0x7f411c06e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.013+0000 7f4119e00640 1 -- 172.21.15.123:0/2484479953 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f411c0ce7e0 msgr2=0x7f411c06e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:00.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.013+0000 7f4119e00640 1 --2- 172.21.15.123:0/2484479953 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f411c0ce7e0 0x7f411c06e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.013+0000 7f4119e00640 1 -- 172.21.15.123:0/2484479953 >> v1:172.21.15.123:6789/0 conn(0x7f411c06c410 legacy=0x7f411c0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.013+0000 7f4119e00640 1 -- 172.21.15.123:0/2484479953 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c0d0a20 con 0x7f411c0ae990 2024-04-24T05:47:00.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.013+0000 7f4119400640 1 -- 172.21.15.123:0/2484479953 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2455008639 0 0) 0x7f411c0d0740 con 0x7f411c06c410 2024-04-24T05:47:00.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.013+0000 7f411a800640 1 --2- 172.21.15.123:0/2484479953 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f411c0ce7e0 0x7f411c06e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:00.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.014+0000 7f4119e00640 1 --2- 172.21.15.123:0/2484479953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f411c0ae990 0x7f411c0d0160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f410c009870 tx=0x7f410c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a9d8bcdb02c762 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:00.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.014+0000 7f4119400640 1 -- 172.21.15.123:0/2484479953 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c037070 con 0x7f411c0ae990 2024-04-24T05:47:00.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.014+0000 7f4119400640 1 -- 172.21.15.123:0/2484479953 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f410c032440 con 0x7f411c0ae990 2024-04-24T05:47:00.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.014+0000 7f4119400640 1 -- 172.21.15.123:0/2484479953 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c032810 con 0x7f411c0ae990 2024-04-24T05:47:00.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.015+0000 7f4121200640 1 -- 172.21.15.123:0/2484479953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f411c0ae990 msgr2=0x7f411c0d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.015+0000 7f4121200640 1 --2- 172.21.15.123:0/2484479953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f411c0ae990 0x7f411c0d0160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f410c009870 tx=0x7f410c02a250 comp rx=0 tx=0).stop 2024-04-24T05:47:00.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.015+0000 7f4121200640 1 -- 172.21.15.123:0/2484479953 shutdown_connections 2024-04-24T05:47:00.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.015+0000 7f4121200640 1 --2- 172.21.15.123:0/2484479953 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f411c0ce7e0 0x7f411c06e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.015+0000 7f4121200640 1 --2- 172.21.15.123:0/2484479953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f411c0ae990 0x7f411c0d0160 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.015+0000 7f4121200640 1 -- 172.21.15.123:0/2484479953 >> 172.21.15.123:0/2484479953 conn(0x7f411c0b51f0 msgr2=0x7f411c0b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:00.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.015+0000 7f4121200640 1 -- 172.21.15.123:0/2484479953 shutdown_connections 2024-04-24T05:47:00.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.015+0000 7f4121200640 1 -- 172.21.15.123:0/2484479953 wait complete. 2024-04-24T05:47:00.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.017+0000 7f4121200640 1 Processor -- start 2024-04-24T05:47:00.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.017+0000 7f4121200640 1 -- start start 2024-04-24T05:47:00.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.018+0000 7f4121200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f411c06c410 0x7f411c0c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.018+0000 7f4121200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f411c0ae990 0x7f411c15e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.018+0000 7f4121200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f411c0ce7e0 0x7f411c0b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.018+0000 7f4121200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f411c06c870 con 0x7f411c0ae990 2024-04-24T05:47:00.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.018+0000 7f4121200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f411c0aed70 con 0x7f411c06c410 2024-04-24T05:47:00.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.018+0000 7f4121200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f411c0b0d40 con 0x7f411c0ce7e0 2024-04-24T05:47:00.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.018+0000 7f4111400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f411c0ae990 0x7f411c15e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.018+0000 7f4111400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f411c0ae990 0x7f411c15e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42488/0 (socket says 172.21.15.123:42488) 2024-04-24T05:47:00.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.018+0000 7f4111400640 1 -- 172.21.15.123:0/1651678218 learned_addr learned my addr 172.21.15.123:0/1651678218 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:00.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.019+0000 7f4111400640 1 -- 172.21.15.123:0/1651678218 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f411c0ce7e0 msgr2=0x7f411c0b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.019+0000 7f4111400640 1 --2- 172.21.15.123:0/1651678218 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f411c0ce7e0 0x7f411c0b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.019+0000 7f4111400640 1 -- 172.21.15.123:0/1651678218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f411c06c410 msgr2=0x7f411c0c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.019+0000 7f4111400640 1 --2- 172.21.15.123:0/1651678218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f411c06c410 0x7f411c0c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.019+0000 7f4111400640 1 -- 172.21.15.123:0/1651678218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c0b1020 con 0x7f411c0ae990 2024-04-24T05:47:00.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.019+0000 7f4111400640 1 --2- 172.21.15.123:0/1651678218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f411c0ae990 0x7f411c15e5d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f410400c4b0 tx=0x7f410400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:00.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.019+0000 7f4112a00640 1 -- 172.21.15.123:0/1651678218 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4104017070 con 0x7f411c0ae990 2024-04-24T05:47:00.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.019+0000 7f4112a00640 1 -- 172.21.15.123:0/1651678218 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4104005480 con 0x7f411c0ae990 2024-04-24T05:47:00.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.019+0000 7f4112a00640 1 -- 172.21.15.123:0/1651678218 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4104003830 con 0x7f411c0ae990 2024-04-24T05:47:00.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.019+0000 7f4121200640 1 -- 172.21.15.123:0/1651678218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411c0b1300 con 0x7f411c0ae990 2024-04-24T05:47:00.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.019+0000 7f4121200640 1 -- 172.21.15.123:0/1651678218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411c0b18c0 con 0x7f411c0ae990 2024-04-24T05:47:00.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.021+0000 7f4121200640 1 -- 172.21.15.123:0/1651678218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f411c0b1ac0 con 0x7f411c0ae990 2024-04-24T05:47:00.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.021+0000 7f4112a00640 1 -- 172.21.15.123:0/1651678218 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f410401a070 con 0x7f411c0ae990 2024-04-24T05:47:00.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.022+0000 7f4112a00640 1 --2- 172.21.15.123:0/1651678218 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f40f806c8f0 0x7f40f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.022+0000 7f4112a00640 1 -- 172.21.15.123:0/1651678218 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f410408b3c0 con 0x7f411c0ae990 2024-04-24T05:47:00.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.023+0000 7f4119e00640 1 --2- 172.21.15.123:0/1651678218 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f40f806c8f0 0x7f40f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.023+0000 7f4119e00640 1 --2- 172.21.15.123:0/1651678218 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f40f806c8f0 0x7f40f806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f410c034000 tx=0x7f410c02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:00.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.027+0000 7f4112a00640 1 -- 172.21.15.123:0/1651678218 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f411c0b1ac0 con 0x7f411c0ae990 2024-04-24T05:47:00.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.223+0000 7f4121200640 1 -- 172.21.15.123:0/1651678218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f411c0c7580 con 0x7f411c0ae990 2024-04-24T05:47:00.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.223+0000 7f4112a00640 1 -- 172.21.15.123:0/1651678218 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f4104059350 con 0x7f411c0ae990 2024-04-24T05:47:00.225 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:47:00.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.226+0000 7f4121200640 1 -- 172.21.15.123:0/1651678218 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f40f806c8f0 msgr2=0x7f40f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.226+0000 7f4121200640 1 --2- 172.21.15.123:0/1651678218 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f40f806c8f0 0x7f40f806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f410c034000 tx=0x7f410c02c000 comp rx=0 tx=0).stop 2024-04-24T05:47:00.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.226+0000 7f4121200640 1 -- 172.21.15.123:0/1651678218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f411c0ae990 msgr2=0x7f411c15e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.226+0000 7f4121200640 1 --2- 172.21.15.123:0/1651678218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f411c0ae990 0x7f411c15e5d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f410400c4b0 tx=0x7f410400c970 comp rx=0 tx=0).stop 2024-04-24T05:47:00.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.226+0000 7f4121200640 1 -- 172.21.15.123:0/1651678218 shutdown_connections 2024-04-24T05:47:00.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.226+0000 7f4121200640 1 --2- 172.21.15.123:0/1651678218 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f40f806c8f0 0x7f40f806edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.226+0000 7f4121200640 1 --2- 172.21.15.123:0/1651678218 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f411c0ce7e0 0x7f411c0b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.226+0000 7f4121200640 1 --2- 172.21.15.123:0/1651678218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f411c0ae990 0x7f411c15e5d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.226+0000 7f4121200640 1 --2- 172.21.15.123:0/1651678218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f411c06c410 0x7f411c0c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.226+0000 7f4121200640 1 -- 172.21.15.123:0/1651678218 >> 172.21.15.123:0/1651678218 conn(0x7f411c0b51f0 msgr2=0x7f411c0b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:00.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.226+0000 7f4121200640 1 -- 172.21.15.123:0/1651678218 shutdown_connections 2024-04-24T05:47:00.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.226+0000 7f4121200640 1 -- 172.21.15.123:0/1651678218 wait complete. 2024-04-24T05:47:00.245 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:47:00.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.447+0000 7fe28fa00640 1 Processor -- start 2024-04-24T05:47:00.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.447+0000 7fe28fa00640 1 -- start start 2024-04-24T05:47:00.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.448+0000 7fe28fa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2880cd2c0 0x7fe2880d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.448+0000 7fe28fa00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2880cf170 con 0x7fe28806c510 2024-04-24T05:47:00.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.448+0000 7fe28fa00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2880cf450 con 0x7fe288071c50 2024-04-24T05:47:00.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.448+0000 7fe28fa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe2880cf730 con 0x7fe2880cd2c0 2024-04-24T05:47:00.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.449+0000 7fe28d000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe28806c510 0x7fe28806c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49338/0 (socket says 172.21.15.123:49338) 2024-04-24T05:47:00.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.449+0000 7fe28d000640 1 -- 172.21.15.123:0/3064028706 learned_addr learned my addr 172.21.15.123:0/3064028706 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:00.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.449+0000 7fe28da00640 1 --2- 172.21.15.123:0/3064028706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2880cd2c0 0x7fe2880d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.449+0000 7fe287400640 1 -- 172.21.15.123:0/3064028706 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3849191238 0 0) 0x7fe2880cf170 con 0x7fe28806c510 2024-04-24T05:47:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.449+0000 7fe287400640 1 -- 172.21.15.123:0/3064028706 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe270003520 con 0x7fe28806c510 2024-04-24T05:47:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe287400640 1 -- 172.21.15.123:0/3064028706 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3112852966 0 0) 0x7fe270003520 con 0x7fe28806c510 2024-04-24T05:47:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe287400640 1 -- 172.21.15.123:0/3064028706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2880cd2c0 msgr2=0x7fe2880d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe287400640 1 --2- 172.21.15.123:0/3064028706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2880cd2c0 0x7fe2880d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe287400640 1 -- 172.21.15.123:0/3064028706 >> v1:172.21.15.155:6789/0 conn(0x7fe288071c50 legacy=0x7fe2880ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe287400640 1 -- 172.21.15.123:0/3064028706 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2880cfa10 con 0x7fe28806c510 2024-04-24T05:47:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe287400640 1 -- 172.21.15.123:0/3064028706 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7fe27c001670 con 0x7fe28806c510 2024-04-24T05:47:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe287400640 1 -- 172.21.15.123:0/3064028706 >> v1:172.21.15.123:6789/0 conn(0x7fe28806c510 legacy=0x7fe28806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe287400640 1 --2- 172.21.15.123:0/3064028706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2700039e0 0x7fe270003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe287400640 1 --2- 172.21.15.123:0/3064028706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe270004800 0x7fe270004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe287400640 1 --2- 172.21.15.123:0/3064028706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2700051c0 0x7fe27000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe287400640 1 -- 172.21.15.123:0/3064028706 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe2880cf730 con 0x7fe2700039e0 2024-04-24T05:47:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe287400640 1 -- 172.21.15.123:0/3064028706 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe27000bfe0 con 0x7fe2700051c0 2024-04-24T05:47:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe287400640 1 -- 172.21.15.123:0/3064028706 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe27000c1e0 con 0x7fe270004800 2024-04-24T05:47:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe28da00640 1 --2- 172.21.15.123:0/3064028706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2880cd2c0 0x7fe2880d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:47:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe28d000640 1 --2- 172.21.15.123:0/3064028706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2700039e0 0x7fe270003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.450+0000 7fe287e00640 1 --2- 172.21.15.123:0/3064028706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe270004800 0x7fe270004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.451+0000 7fe28da00640 1 --2- 172.21.15.123:0/3064028706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2700051c0 0x7fe27000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.451+0000 7fe28d000640 1 -- 172.21.15.123:0/3064028706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe270004800 msgr2=0x7fe270004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.451+0000 7fe28d000640 1 --2- 172.21.15.123:0/3064028706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe270004800 0x7fe270004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.451+0000 7fe28d000640 1 -- 172.21.15.123:0/3064028706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2700051c0 msgr2=0x7fe27000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.451+0000 7fe28d000640 1 --2- 172.21.15.123:0/3064028706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2700051c0 0x7fe27000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.451+0000 7fe28d000640 1 -- 172.21.15.123:0/3064028706 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2880cfa10 con 0x7fe2700039e0 2024-04-24T05:47:00.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.451+0000 7fe28da00640 1 --2- 172.21.15.123:0/3064028706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2700051c0 0x7fe27000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:00.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.452+0000 7fe28d000640 1 --2- 172.21.15.123:0/3064028706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2700039e0 0x7fe270003e00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe27c001eb0 tx=0x7fe270005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12bdfb6d2293c962 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:00.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.452+0000 7fe287400640 1 -- 172.21.15.123:0/3064028706 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c007d10 con 0x7fe2700039e0 2024-04-24T05:47:00.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.452+0000 7fe287400640 1 -- 172.21.15.123:0/3064028706 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe27c0058d0 con 0x7fe2700039e0 2024-04-24T05:47:00.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.452+0000 7fe287400640 1 -- 172.21.15.123:0/3064028706 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c005cc0 con 0x7fe2700039e0 2024-04-24T05:47:00.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.453+0000 7fe28fa00640 1 -- 172.21.15.123:0/3064028706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2700039e0 msgr2=0x7fe270003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.453+0000 7fe28fa00640 1 --2- 172.21.15.123:0/3064028706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2700039e0 0x7fe270003e00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe27c001eb0 tx=0x7fe270005830 comp rx=0 tx=0).stop 2024-04-24T05:47:00.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.453+0000 7fe28da00640 1 -- 172.21.15.123:0/3064028706 reap_dead start 2024-04-24T05:47:00.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.453+0000 7fe28fa00640 1 -- 172.21.15.123:0/3064028706 shutdown_connections 2024-04-24T05:47:00.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.453+0000 7fe28fa00640 1 -- 172.21.15.123:0/3064028706 >> 172.21.15.123:0/3064028706 conn(0x7fe2880b51f0 msgr2=0x7fe2880b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:00.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.453+0000 7fe28fa00640 1 -- 172.21.15.123:0/3064028706 shutdown_connections 2024-04-24T05:47:00.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.453+0000 7fe28fa00640 1 -- 172.21.15.123:0/3064028706 wait complete. 2024-04-24T05:47:00.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.455+0000 7fe28fa00640 1 Processor -- start 2024-04-24T05:47:00.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.455+0000 7fe28fa00640 1 -- start start 2024-04-24T05:47:00.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.456+0000 7fe28fa00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2700039e0 0x7fe2880c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.456+0000 7fe28fa00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe28815f9d0 0x7fe2881f8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.456+0000 7fe28fa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2881f1a90 0x7fe2881f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.456+0000 7fe28fa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe28806d180 con 0x7fe2881f1a90 2024-04-24T05:47:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.456+0000 7fe28fa00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe28806d000 con 0x7fe28815f9d0 2024-04-24T05:47:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.456+0000 7fe28fa00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe2880ced00 con 0x7fe2700039e0 2024-04-24T05:47:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.456+0000 7fe28da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2881f1a90 0x7fe2881f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.456+0000 7fe28da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2881f1a90 0x7fe2881f1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42506/0 (socket says 172.21.15.123:42506) 2024-04-24T05:47:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.456+0000 7fe28da00640 1 -- 172.21.15.123:0/682541149 learned_addr learned my addr 172.21.15.123:0/682541149 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.457+0000 7fe28da00640 1 -- 172.21.15.123:0/682541149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2700039e0 msgr2=0x7fe2880c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.457+0000 7fe28d000640 1 --2- 172.21.15.123:0/682541149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2700039e0 0x7fe2880c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.457+0000 7fe287e00640 1 --2- 172.21.15.123:0/682541149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe28815f9d0 0x7fe2881f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.457+0000 7fe28da00640 1 --2- 172.21.15.123:0/682541149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2700039e0 0x7fe2880c5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.457+0000 7fe28da00640 1 -- 172.21.15.123:0/682541149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe28815f9d0 msgr2=0x7fe2881f8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.457+0000 7fe28da00640 1 --2- 172.21.15.123:0/682541149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe28815f9d0 0x7fe2881f8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.457+0000 7fe28da00640 1 -- 172.21.15.123:0/682541149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28815fe30 con 0x7fe2881f1a90 2024-04-24T05:47:00.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.457+0000 7fe28da00640 1 --2- 172.21.15.123:0/682541149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2881f1a90 0x7fe2881f1f10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe288071c00 tx=0x7fe28000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:00.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.457+0000 7fe284c00640 1 -- 172.21.15.123:0/682541149 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28001c070 con 0x7fe2881f1a90 2024-04-24T05:47:00.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.457+0000 7fe284c00640 1 -- 172.21.15.123:0/682541149 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe280017600 con 0x7fe2881f1a90 2024-04-24T05:47:00.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.457+0000 7fe28fa00640 1 -- 172.21.15.123:0/682541149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2880c5b20 con 0x7fe2881f1a90 2024-04-24T05:47:00.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.457+0000 7fe284c00640 1 -- 172.21.15.123:0/682541149 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2800179f0 con 0x7fe2881f1a90 2024-04-24T05:47:00.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.458+0000 7fe28fa00640 1 -- 172.21.15.123:0/682541149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2880c60e0 con 0x7fe2881f1a90 2024-04-24T05:47:00.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.459+0000 7fe28fa00640 1 -- 172.21.15.123:0/682541149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe288068a20 con 0x7fe2881f1a90 2024-04-24T05:47:00.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.460+0000 7fe284c00640 1 -- 172.21.15.123:0/682541149 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe28001f070 con 0x7fe2881f1a90 2024-04-24T05:47:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.461+0000 7fe284c00640 1 --2- 172.21.15.123:0/682541149 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe26806c8f0 0x7fe26806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.461+0000 7fe284c00640 1 -- 172.21.15.123:0/682541149 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fe28008c070 con 0x7fe2881f1a90 2024-04-24T05:47:00.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.461+0000 7fe28d000640 1 --2- 172.21.15.123:0/682541149 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe26806c8f0 0x7fe26806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.462+0000 7fe28d000640 1 --2- 172.21.15.123:0/682541149 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe26806c8f0 0x7fe26806edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe27c033000 tx=0x7fe27c035040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:00.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.465+0000 7fe284c00640 1 -- 172.21.15.123:0/682541149 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe280012070 con 0x7fe2881f1a90 2024-04-24T05:47:00.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.654+0000 7fe28fa00640 1 -- 172.21.15.123:0/682541149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe2880caf10 con 0x7fe2881f1a90 2024-04-24T05:47:00.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.655+0000 7fe284c00640 1 -- 172.21.15.123:0/682541149 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fe28005a120 con 0x7fe2881f1a90 2024-04-24T05:47:00.657 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:00.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.659+0000 7fe28fa00640 1 -- 172.21.15.123:0/682541149 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe26806c8f0 msgr2=0x7fe26806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.659+0000 7fe28fa00640 1 --2- 172.21.15.123:0/682541149 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe26806c8f0 0x7fe26806edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe27c033000 tx=0x7fe27c035040 comp rx=0 tx=0).stop 2024-04-24T05:47:00.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.659+0000 7fe28fa00640 1 -- 172.21.15.123:0/682541149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2881f1a90 msgr2=0x7fe2881f1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.659+0000 7fe28fa00640 1 --2- 172.21.15.123:0/682541149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2881f1a90 0x7fe2881f1f10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe288071c00 tx=0x7fe28000f790 comp rx=0 tx=0).stop 2024-04-24T05:47:00.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.659+0000 7fe28fa00640 1 -- 172.21.15.123:0/682541149 shutdown_connections 2024-04-24T05:47:00.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.659+0000 7fe28fa00640 1 --2- 172.21.15.123:0/682541149 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe26806c8f0 0x7fe26806edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.659+0000 7fe28fa00640 1 --2- 172.21.15.123:0/682541149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2881f1a90 0x7fe2881f1f10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.659+0000 7fe28fa00640 1 --2- 172.21.15.123:0/682541149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe28815f9d0 0x7fe2881f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.659+0000 7fe28fa00640 1 --2- 172.21.15.123:0/682541149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2700039e0 0x7fe2880c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.659+0000 7fe28fa00640 1 -- 172.21.15.123:0/682541149 >> 172.21.15.123:0/682541149 conn(0x7fe2880b51f0 msgr2=0x7fe2880b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.660+0000 7fe28fa00640 1 -- 172.21.15.123:0/682541149 shutdown_connections 2024-04-24T05:47:00.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.660+0000 7fe28fa00640 1 -- 172.21.15.123:0/682541149 wait complete. 2024-04-24T05:47:00.679 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2024-04-24T05:47:00.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.864+0000 7f5394c00640 1 Processor -- start 2024-04-24T05:47:00.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.864+0000 7f5394c00640 1 -- start start 2024-04-24T05:47:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.865+0000 7f5394c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f539006f680 0x7f539015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.865+0000 7f5394c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f539006ba00 0x7f5390071c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.865+0000 7f5394c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5390072170 0x7f539015b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.865+0000 7f5394c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f539006be80 con 0x7f539006f680 2024-04-24T05:47:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.865+0000 7f5394c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f53900725a0 con 0x7f5390072170 2024-04-24T05:47:00.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.865+0000 7f5394c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f539006c0e0 con 0x7f539006ba00 2024-04-24T05:47:00.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.866+0000 7f538de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f539006f680 0x7f539015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.866+0000 7f538de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f539006f680 0x7f539015ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42508/0 (socket says 172.21.15.123:42508) 2024-04-24T05:47:00.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.866+0000 7f538de00640 1 -- 172.21.15.123:0/1027715132 learned_addr learned my addr 172.21.15.123:0/1027715132 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:00.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.866+0000 7f538d400640 1 --2- 172.21.15.123:0/1027715132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f539006ba00 0x7f5390071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.866+0000 7f538e800640 1 --2- 172.21.15.123:0/1027715132 >> v2:172.21.15.155:3300/0 conn(0x7f5390072170 0x7f539015b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.866+0000 7f538de00640 1 -- 172.21.15.123:0/1027715132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f539006ba00 msgr2=0x7f5390071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.866+0000 7f538de00640 1 --2- 172.21.15.123:0/1027715132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f539006ba00 0x7f5390071c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.866+0000 7f538de00640 1 -- 172.21.15.123:0/1027715132 >> v2:172.21.15.155:3300/0 conn(0x7f5390072170 msgr2=0x7f539015b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.866+0000 7f538de00640 1 --2- 172.21.15.123:0/1027715132 >> v2:172.21.15.155:3300/0 conn(0x7f5390072170 0x7f539015b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.866+0000 7f538de00640 1 -- 172.21.15.123:0/1027715132 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539006c3c0 con 0x7f539006f680 2024-04-24T05:47:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.868+0000 7f538de00640 1 --2- 172.21.15.123:0/1027715132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f539006f680 0x7f539015ed80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f53900b2e90 tx=0x7f538002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c5712eb62808 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.868+0000 7f538ca00640 1 -- 172.21.15.123:0/1027715132 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380037070 con 0x7f539006f680 2024-04-24T05:47:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.868+0000 7f538ca00640 1 -- 172.21.15.123:0/1027715132 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5380032440 con 0x7f539006f680 2024-04-24T05:47:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.868+0000 7f538ca00640 1 -- 172.21.15.123:0/1027715132 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380032810 con 0x7f539006f680 2024-04-24T05:47:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.869+0000 7f5394c00640 1 -- 172.21.15.123:0/1027715132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f539006f680 msgr2=0x7f539015ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.869+0000 7f5394c00640 1 --2- 172.21.15.123:0/1027715132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f539006f680 0x7f539015ed80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f53900b2e90 tx=0x7f538002a180 comp rx=0 tx=0).stop 2024-04-24T05:47:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.869+0000 7f5394c00640 1 -- 172.21.15.123:0/1027715132 shutdown_connections 2024-04-24T05:47:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.869+0000 7f5394c00640 1 --2- 172.21.15.123:0/1027715132 >> v2:172.21.15.155:3300/0 conn(0x7f5390072170 0x7f539015b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.869+0000 7f5394c00640 1 --2- 172.21.15.123:0/1027715132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f539006ba00 0x7f5390071c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.869+0000 7f5394c00640 1 --2- 172.21.15.123:0/1027715132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f539006f680 0x7f539015ed80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.869+0000 7f5394c00640 1 -- 172.21.15.123:0/1027715132 >> 172.21.15.123:0/1027715132 conn(0x7f53900b1e70 msgr2=0x7f53900b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.869+0000 7f5394c00640 1 -- 172.21.15.123:0/1027715132 shutdown_connections 2024-04-24T05:47:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.869+0000 7f5394c00640 1 -- 172.21.15.123:0/1027715132 wait complete. 2024-04-24T05:47:00.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.871+0000 7f5394c00640 1 Processor -- start 2024-04-24T05:47:00.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.871+0000 7f5394c00640 1 -- start start 2024-04-24T05:47:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.872+0000 7f5394c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f539006ba00 0x7f5390099930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.872+0000 7f5394c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f539006f680 0x7f53901619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.872+0000 7f5394c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5390072170 0x7f5390161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.872+0000 7f5394c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f539006cc60 con 0x7f539006ba00 2024-04-24T05:47:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.872+0000 7f5394c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f539006c980 con 0x7f539006f680 2024-04-24T05:47:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.872+0000 7f5394c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f539006fa60 con 0x7f5390072170 2024-04-24T05:47:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.873+0000 7f538de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f539006ba00 0x7f5390099930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.873+0000 7f538de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f539006ba00 0x7f5390099930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42514/0 (socket says 172.21.15.123:42514) 2024-04-24T05:47:00.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.873+0000 7f538de00640 1 -- 172.21.15.123:0/3998363774 learned_addr learned my addr 172.21.15.123:0/3998363774 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:00.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.873+0000 7f538de00640 1 -- 172.21.15.123:0/3998363774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5390072170 msgr2=0x7f5390161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.873+0000 7f538de00640 1 --2- 172.21.15.123:0/3998363774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5390072170 0x7f5390161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.873+0000 7f538de00640 1 -- 172.21.15.123:0/3998363774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f539006f680 msgr2=0x7f53901619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:00.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.873+0000 7f538de00640 1 --2- 172.21.15.123:0/3998363774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f539006f680 0x7f53901619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:00.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.873+0000 7f538de00640 1 -- 172.21.15.123:0/3998363774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5390099e70 con 0x7f539006ba00 2024-04-24T05:47:00.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.873+0000 7f538de00640 1 --2- 172.21.15.123:0/3998363774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f539006ba00 0x7f5390099930 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f538002a1c0 tx=0x7f53800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:00.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.874+0000 7f5386000640 1 -- 172.21.15.123:0/3998363774 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380041020 con 0x7f539006ba00 2024-04-24T05:47:00.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.874+0000 7f5386000640 1 -- 172.21.15.123:0/3998363774 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53800306c0 con 0x7f539006ba00 2024-04-24T05:47:00.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.874+0000 7f5386000640 1 -- 172.21.15.123:0/3998363774 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380037040 con 0x7f539006ba00 2024-04-24T05:47:00.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.874+0000 7f5394c00640 1 -- 172.21.15.123:0/3998363774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53901e8330 con 0x7f539006ba00 2024-04-24T05:47:00.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.874+0000 7f5394c00640 1 -- 172.21.15.123:0/3998363774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53901e88b0 con 0x7f539006ba00 2024-04-24T05:47:00.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.875+0000 7f5394c00640 1 -- 172.21.15.123:0/3998363774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5390068a20 con 0x7f539006ba00 2024-04-24T05:47:00.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.876+0000 7f5386000640 1 -- 172.21.15.123:0/3998363774 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5380044020 con 0x7f539006ba00 2024-04-24T05:47:00.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.877+0000 7f5386000640 1 --2- 172.21.15.123:0/3998363774 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f537006c8f0 0x7f537006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:00.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.877+0000 7f5386000640 1 -- 172.21.15.123:0/3998363774 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f53800ac800 con 0x7f539006ba00 2024-04-24T05:47:00.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.877+0000 7f538d400640 1 --2- 172.21.15.123:0/3998363774 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f537006c8f0 0x7f537006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.878+0000 7f538d400640 1 --2- 172.21.15.123:0/3998363774 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f537006c8f0 0x7f537006edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f53900b3130 tx=0x7f537c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:00.881+0000 7f5386000640 1 -- 172.21.15.123:0/3998363774 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f538004b030 con 0x7f539006ba00 2024-04-24T05:47:01.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.147+0000 7f5394c00640 1 -- 172.21.15.123:0/3998363774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) -- 0x7f53900afed0 con 0x7f539006ba00 2024-04-24T05:47:01.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.719+0000 7f5386000640 1 -- 172.21.15.123:0/3998363774 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v82) ==== 147+0+0 (secure 0 0 0) 0x7f5390099e70 con 0x7f539006ba00 2024-04-24T05:47:01.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.723+0000 7f5394c00640 1 -- 172.21.15.123:0/3998363774 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f537006c8f0 msgr2=0x7f537006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:01.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.723+0000 7f5394c00640 1 --2- 172.21.15.123:0/3998363774 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f537006c8f0 0x7f537006edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f53900b3130 tx=0x7f537c008040 comp rx=0 tx=0).stop 2024-04-24T05:47:01.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.723+0000 7f5394c00640 1 -- 172.21.15.123:0/3998363774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f539006ba00 msgr2=0x7f5390099930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:01.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.723+0000 7f5394c00640 1 --2- 172.21.15.123:0/3998363774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f539006ba00 0x7f5390099930 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f538002a1c0 tx=0x7f53800046c0 comp rx=0 tx=0).stop 2024-04-24T05:47:01.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.723+0000 7f5394c00640 1 -- 172.21.15.123:0/3998363774 shutdown_connections 2024-04-24T05:47:01.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.723+0000 7f5394c00640 1 --2- 172.21.15.123:0/3998363774 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f537006c8f0 0x7f537006edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:01.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.723+0000 7f5394c00640 1 --2- 172.21.15.123:0/3998363774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5390072170 0x7f5390161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:01.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.723+0000 7f5394c00640 1 --2- 172.21.15.123:0/3998363774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f539006f680 0x7f53901619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:01.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.723+0000 7f5394c00640 1 --2- 172.21.15.123:0/3998363774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f539006ba00 0x7f5390099930 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:01.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.723+0000 7f5394c00640 1 -- 172.21.15.123:0/3998363774 >> 172.21.15.123:0/3998363774 conn(0x7f53900b1e70 msgr2=0x7f539015eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:01.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.723+0000 7f5394c00640 1 -- 172.21.15.123:0/3998363774 shutdown_connections 2024-04-24T05:47:01.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.723+0000 7f5394c00640 1 -- 172.21.15.123:0/3998363774 wait complete. 2024-04-24T05:47:01.748 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-24T05:47:01.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.949+0000 7f6a0f200640 1 Processor -- start 2024-04-24T05:47:01.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.950+0000 7f6a0f200640 1 -- start start 2024-04-24T05:47:01.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.951+0000 7f6a0f200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6a0815f870 0x7f6a080d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:01.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.951+0000 7f6a0f200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6a080d1420 0x7f6a080cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:01.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.951+0000 7f6a0f200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a08162a60 con 0x7f6a0815e5f0 2024-04-24T05:47:01.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.951+0000 7f6a0f200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6a080ceb50 con 0x7f6a0815f870 2024-04-24T05:47:01.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.951+0000 7f6a0f200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6a080cee30 con 0x7f6a080d1420 2024-04-24T05:47:01.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.951+0000 7f6a0de00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6a0815e5f0 0x7f6a08162250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49340/0 (socket says 172.21.15.123:49340) 2024-04-24T05:47:01.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.951+0000 7f6a0de00640 1 -- 172.21.15.123:0/3946342718 learned_addr learned my addr 172.21.15.123:0/3946342718 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:01.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.951+0000 7f6a0d400640 1 --2- 172.21.15.123:0/3946342718 >> v2:172.21.15.123:3300/0 conn(0x7f6a0815f870 0x7f6a080d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:01.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.952+0000 7f6a0e800640 1 --2- 172.21.15.123:0/3946342718 >> v2:172.21.15.155:3300/0 conn(0x7f6a080d1420 0x7f6a080cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:01.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.952+0000 7f6a0ca00640 1 -- 172.21.15.123:0/3946342718 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3819084134 0 0) 0x7f6a08162a60 con 0x7f6a0815e5f0 2024-04-24T05:47:01.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.952+0000 7f6a0ca00640 1 -- 172.21.15.123:0/3946342718 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69f0003520 con 0x7f6a0815e5f0 2024-04-24T05:47:01.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.952+0000 7f6a0d400640 1 -- 172.21.15.123:0/3946342718 >> v2:172.21.15.155:3300/0 conn(0x7f6a080d1420 msgr2=0x7f6a080cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:01.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.952+0000 7f6a0d400640 1 --2- 172.21.15.123:0/3946342718 >> v2:172.21.15.155:3300/0 conn(0x7f6a080d1420 0x7f6a080cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:01.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.952+0000 7f6a0d400640 1 -- 172.21.15.123:0/3946342718 >> v1:172.21.15.123:6789/0 conn(0x7f6a0815e5f0 legacy=0x7f6a08162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:01.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.952+0000 7f6a0d400640 1 -- 172.21.15.123:0/3946342718 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a08162d40 con 0x7f6a0815f870 2024-04-24T05:47:01.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.952+0000 7f6a0e800640 1 --2- 172.21.15.123:0/3946342718 >> v2:172.21.15.155:3300/0 conn(0x7f6a080d1420 0x7f6a080cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:01.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.953+0000 7f6a0d400640 1 --2- 172.21.15.123:0/3946342718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a0815f870 0x7f6a080d0ee0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6a08162180 tx=0x7f69f802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34e3b993d1f39ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:01.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.953+0000 7f6a0ca00640 1 -- 172.21.15.123:0/3946342718 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8037070 con 0x7f6a0815f870 2024-04-24T05:47:01.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.953+0000 7f6a0ca00640 1 -- 172.21.15.123:0/3946342718 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69f8032440 con 0x7f6a0815f870 2024-04-24T05:47:01.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.953+0000 7f6a0ca00640 1 -- 172.21.15.123:0/3946342718 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8032810 con 0x7f6a0815f870 2024-04-24T05:47:01.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.954+0000 7f6a0f200640 1 -- 172.21.15.123:0/3946342718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a0815f870 msgr2=0x7f6a080d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:01.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.954+0000 7f6a0f200640 1 --2- 172.21.15.123:0/3946342718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a0815f870 0x7f6a080d0ee0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6a08162180 tx=0x7f69f802a160 comp rx=0 tx=0).stop 2024-04-24T05:47:01.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.954+0000 7f6a0f200640 1 -- 172.21.15.123:0/3946342718 shutdown_connections 2024-04-24T05:47:01.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.954+0000 7f6a0f200640 1 --2- 172.21.15.123:0/3946342718 >> v2:172.21.15.155:3300/0 conn(0x7f6a080d1420 0x7f6a080cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:01.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.954+0000 7f6a0f200640 1 --2- 172.21.15.123:0/3946342718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a0815f870 0x7f6a080d0ee0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:01.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.954+0000 7f6a0f200640 1 -- 172.21.15.123:0/3946342718 >> 172.21.15.123:0/3946342718 conn(0x7f6a080b9db0 msgr2=0x7f6a0809d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:01.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.954+0000 7f6a0f200640 1 -- 172.21.15.123:0/3946342718 shutdown_connections 2024-04-24T05:47:01.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.954+0000 7f6a0f200640 1 -- 172.21.15.123:0/3946342718 wait complete. 2024-04-24T05:47:01.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.956+0000 7f6a0f200640 1 Processor -- start 2024-04-24T05:47:01.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.956+0000 7f6a0f200640 1 -- start start 2024-04-24T05:47:01.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.957+0000 7f6a0f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a080d1420 0x7f6a080ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.957+0000 7f6a0f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a0815e5f0 0x7f6a081e9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.957+0000 7f6a0f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a0815f870 0x7f6a080cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.957+0000 7f6a0f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0815eb50 con 0x7f6a080d1420 2024-04-24T05:47:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.957+0000 7f6a0f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0815e9d0 con 0x7f6a0815f870 2024-04-24T05:47:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.957+0000 7f6a0f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6a080cf800 con 0x7f6a0815e5f0 2024-04-24T05:47:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.957+0000 7f6a0de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a080d1420 0x7f6a080ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.957+0000 7f6a0de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a080d1420 0x7f6a080ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42524/0 (socket says 172.21.15.123:42524) 2024-04-24T05:47:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.957+0000 7f6a0de00640 1 -- 172.21.15.123:0/1272026130 learned_addr learned my addr 172.21.15.123:0/1272026130 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.958+0000 7f6a0de00640 1 -- 172.21.15.123:0/1272026130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a0815e5f0 msgr2=0x7f6a081e9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.958+0000 7f6a0de00640 1 --2- 172.21.15.123:0/1272026130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a0815e5f0 0x7f6a081e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.958+0000 7f6a0de00640 1 -- 172.21.15.123:0/1272026130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a0815f870 msgr2=0x7f6a080cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.958+0000 7f6a0de00640 1 --2- 172.21.15.123:0/1272026130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a0815f870 0x7f6a080cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.958+0000 7f6a0de00640 1 -- 172.21.15.123:0/1272026130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0814b2d0 con 0x7f6a080d1420 2024-04-24T05:47:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.958+0000 7f6a0de00640 1 --2- 172.21.15.123:0/1272026130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a080d1420 0x7f6a080ca6d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00c550 tx=0x7f69fc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.958+0000 7f6a02000640 1 -- 172.21.15.123:0/1272026130 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc018070 con 0x7f6a080d1420 2024-04-24T05:47:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.958+0000 7f6a02000640 1 -- 172.21.15.123:0/1272026130 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69fc0074e0 con 0x7f6a080d1420 2024-04-24T05:47:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.958+0000 7f6a02000640 1 -- 172.21.15.123:0/1272026130 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc004a20 con 0x7f6a080d1420 2024-04-24T05:47:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.958+0000 7f6a0f200640 1 -- 172.21.15.123:0/1272026130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0814b5b0 con 0x7f6a080d1420 2024-04-24T05:47:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.958+0000 7f6a0f200640 1 -- 172.21.15.123:0/1272026130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0814bb70 con 0x7f6a080d1420 2024-04-24T05:47:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.959+0000 7f6a0f200640 1 -- 172.21.15.123:0/1272026130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a08068a20 con 0x7f6a080d1420 2024-04-24T05:47:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.960+0000 7f6a02000640 1 -- 172.21.15.123:0/1272026130 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69fc004bc0 con 0x7f6a080d1420 2024-04-24T05:47:01.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.962+0000 7f6a02000640 1 --2- 172.21.15.123:0/1272026130 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f69ec06c920 0x7f69ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:01.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.962+0000 7f6a02000640 1 -- 172.21.15.123:0/1272026130 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f69fc08b2d0 con 0x7f6a080d1420 2024-04-24T05:47:01.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.962+0000 7f6a0d400640 1 --2- 172.21.15.123:0/1272026130 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f69ec06c920 0x7f69ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:01.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.963+0000 7f6a0d400640 1 --2- 172.21.15.123:0/1272026130 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f69ec06c920 0x7f69ec06ede0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f69f8034000 tx=0x7f69f802c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:01.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:01.966+0000 7f6a02000640 1 -- 172.21.15.123:0/1272026130 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69fc0592e0 con 0x7f6a080d1420 2024-04-24T05:47:02.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.166+0000 7f6a0f200640 1 -- 172.21.15.123:0/1272026130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6a0806eff0 con 0x7f6a080d1420 2024-04-24T05:47:02.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.171+0000 7f6a02000640 1 -- 172.21.15.123:0/1272026130 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f69fc058be0 con 0x7f6a080d1420 2024-04-24T05:47:02.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.174+0000 7f6a0f200640 1 -- 172.21.15.123:0/1272026130 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f69ec06c920 msgr2=0x7f69ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:02.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.174+0000 7f6a0f200640 1 --2- 172.21.15.123:0/1272026130 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f69ec06c920 0x7f69ec06ede0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f69f8034000 tx=0x7f69f802c000 comp rx=0 tx=0).stop 2024-04-24T05:47:02.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.174+0000 7f6a0f200640 1 -- 172.21.15.123:0/1272026130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a080d1420 msgr2=0x7f6a080ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:02.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.174+0000 7f6a0f200640 1 --2- 172.21.15.123:0/1272026130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a080d1420 0x7f6a080ca6d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00c550 tx=0x7f69fc00ca10 comp rx=0 tx=0).stop 2024-04-24T05:47:02.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.174+0000 7f6a0f200640 1 -- 172.21.15.123:0/1272026130 shutdown_connections 2024-04-24T05:47:02.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.174+0000 7f6a0f200640 1 --2- 172.21.15.123:0/1272026130 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f69ec06c920 0x7f69ec06ede0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:02.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.174+0000 7f6a0f200640 1 --2- 172.21.15.123:0/1272026130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a0815f870 0x7f6a080cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:02.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.174+0000 7f6a0f200640 1 --2- 172.21.15.123:0/1272026130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a0815e5f0 0x7f6a081e9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:02.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.174+0000 7f6a0f200640 1 --2- 172.21.15.123:0/1272026130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a080d1420 0x7f6a080ca6d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:02.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.174+0000 7f6a0f200640 1 -- 172.21.15.123:0/1272026130 >> 172.21.15.123:0/1272026130 conn(0x7f6a080b9db0 msgr2=0x7f6a08161460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:02.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.174+0000 7f6a0f200640 1 -- 172.21.15.123:0/1272026130 shutdown_connections 2024-04-24T05:47:02.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.174+0000 7f6a0f200640 1 -- 172.21.15.123:0/1272026130 wait complete. 2024-04-24T05:47:02.195 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-24T05:47:02.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T05:47:02.298 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-24T05:47:02.436 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-24T05:47:02.436 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:47:02.436 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T05:47:02.492 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:47:02.492 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T05:47:02.541 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:02.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.779+0000 7f4e74c00640 1 Processor -- start 2024-04-24T05:47:02.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.779+0000 7f4e74c00640 1 -- start start 2024-04-24T05:47:02.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.781+0000 7f4e74c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4e700ad860 0x7f4e700af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:02.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.781+0000 7f4e74c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e700af780 0x7f4e7006bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:02.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.781+0000 7f4e74c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e700afbb0 con 0x7f4e7015a2a0 2024-04-24T05:47:02.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.781+0000 7f4e74c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4e7015a680 con 0x7f4e700ad860 2024-04-24T05:47:02.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.781+0000 7f4e74c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7015da00 con 0x7f4e700af780 2024-04-24T05:47:02.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.781+0000 7f4e6d400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4e700ad860 0x7f4e700af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:02.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.781+0000 7f4e6d400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4e700ad860 0x7f4e700af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42530/0 (socket says 172.21.15.123:42530) 2024-04-24T05:47:02.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.781+0000 7f4e6d400640 1 -- 172.21.15.123:0/1544084896 learned_addr learned my addr 172.21.15.123:0/1544084896 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:02.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.782+0000 7f4e6e800640 1 --2- 172.21.15.123:0/1544084896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e700af780 0x7f4e7006bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:02.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.782+0000 7f4e6d400640 1 -- 172.21.15.123:0/1544084896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e700af780 msgr2=0x7f4e7006bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:02.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.782+0000 7f4e6d400640 1 --2- 172.21.15.123:0/1544084896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e700af780 0x7f4e7006bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:02.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.782+0000 7f4e6d400640 1 -- 172.21.15.123:0/1544084896 >> v1:172.21.15.155:6789/0 conn(0x7f4e7015a2a0 legacy=0x7f4e700ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:02.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.782+0000 7f4e6d400640 1 -- 172.21.15.123:0/1544084896 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7015cdf0 con 0x7f4e700ad860 2024-04-24T05:47:02.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.783+0000 7f4e6d400640 1 --2- 172.21.15.123:0/1544084896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e700ad860 0x7f4e700af240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009870 tx=0x7f4e5c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d178fba975cacf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:02.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.783+0000 7f4e6ca00640 1 -- 172.21.15.123:0/1544084896 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c037070 con 0x7f4e700ad860 2024-04-24T05:47:02.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.783+0000 7f4e6ca00640 1 -- 172.21.15.123:0/1544084896 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e5c032440 con 0x7f4e700ad860 2024-04-24T05:47:02.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.783+0000 7f4e6ca00640 1 -- 172.21.15.123:0/1544084896 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c032810 con 0x7f4e700ad860 2024-04-24T05:47:02.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.784+0000 7f4e74c00640 1 -- 172.21.15.123:0/1544084896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e700ad860 msgr2=0x7f4e700af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:02.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.784+0000 7f4e74c00640 1 --2- 172.21.15.123:0/1544084896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e700ad860 0x7f4e700af240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009870 tx=0x7f4e5c02a180 comp rx=0 tx=0).stop 2024-04-24T05:47:02.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.784+0000 7f4e74c00640 1 -- 172.21.15.123:0/1544084896 shutdown_connections 2024-04-24T05:47:02.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.784+0000 7f4e74c00640 1 --2- 172.21.15.123:0/1544084896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e700af780 0x7f4e7006bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:02.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.784+0000 7f4e74c00640 1 --2- 172.21.15.123:0/1544084896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e700ad860 0x7f4e700af240 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:02.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.784+0000 7f4e74c00640 1 -- 172.21.15.123:0/1544084896 >> 172.21.15.123:0/1544084896 conn(0x7f4e700b1e70 msgr2=0x7f4e700b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:02.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.784+0000 7f4e74c00640 1 -- 172.21.15.123:0/1544084896 shutdown_connections 2024-04-24T05:47:02.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.784+0000 7f4e74c00640 1 -- 172.21.15.123:0/1544084896 wait complete. 2024-04-24T05:47:02.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.786+0000 7f4e74c00640 1 Processor -- start 2024-04-24T05:47:02.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.786+0000 7f4e74c00640 1 -- start start 2024-04-24T05:47:02.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.787+0000 7f4e74c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e700ad860 0x7f4e700ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:02.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.787+0000 7f4e74c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e700af780 0x7f4e7015edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:02.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.787+0000 7f4e74c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e7015a2a0 0x7f4e7015f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:02.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.787+0000 7f4e74c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7015d690 con 0x7f4e700ad860 2024-04-24T05:47:02.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.787+0000 7f4e74c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7015d3b0 con 0x7f4e7015a2a0 2024-04-24T05:47:02.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.787+0000 7f4e74c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7006c040 con 0x7f4e700af780 2024-04-24T05:47:02.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.787+0000 7f4e6de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e700ad860 0x7f4e700ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:02.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.787+0000 7f4e6de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e700ad860 0x7f4e700ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42534/0 (socket says 172.21.15.123:42534) 2024-04-24T05:47:02.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.787+0000 7f4e6de00640 1 -- 172.21.15.123:0/3924212260 learned_addr learned my addr 172.21.15.123:0/3924212260 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:02.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.788+0000 7f4e6de00640 1 -- 172.21.15.123:0/3924212260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e700af780 msgr2=0x7f4e7015edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:02.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.788+0000 7f4e6de00640 1 --2- 172.21.15.123:0/3924212260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e700af780 0x7f4e7015edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:02.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.788+0000 7f4e6de00640 1 -- 172.21.15.123:0/3924212260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e7015a2a0 msgr2=0x7f4e7015f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:02.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.788+0000 7f4e6e800640 1 --2- 172.21.15.123:0/3924212260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e7015a2a0 0x7f4e7015f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:02.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.788+0000 7f4e6de00640 1 --2- 172.21.15.123:0/3924212260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e7015a2a0 0x7f4e7015f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:02.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.788+0000 7f4e6de00640 1 -- 172.21.15.123:0/3924212260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e70151170 con 0x7f4e700ad860 2024-04-24T05:47:02.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.788+0000 7f4e6de00640 1 --2- 172.21.15.123:0/3924212260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e700ad860 0x7f4e700ce210 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000b3f0 tx=0x7f4e6000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:02.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.788+0000 7f4e66000640 1 -- 172.21.15.123:0/3924212260 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60015020 con 0x7f4e700ad860 2024-04-24T05:47:02.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.788+0000 7f4e66000640 1 -- 172.21.15.123:0/3924212260 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e60011830 con 0x7f4e700ad860 2024-04-24T05:47:02.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.788+0000 7f4e66000640 1 -- 172.21.15.123:0/3924212260 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60011c20 con 0x7f4e700ad860 2024-04-24T05:47:02.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.788+0000 7f4e74c00640 1 -- 172.21.15.123:0/3924212260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e70151450 con 0x7f4e700ad860 2024-04-24T05:47:02.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.789+0000 7f4e74c00640 1 -- 172.21.15.123:0/3924212260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e70151a10 con 0x7f4e700ad860 2024-04-24T05:47:02.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.790+0000 7f4e74c00640 1 -- 172.21.15.123:0/3924212260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e70068a20 con 0x7f4e700ad860 2024-04-24T05:47:02.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.791+0000 7f4e66000640 1 -- 172.21.15.123:0/3924212260 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e60018020 con 0x7f4e700ad860 2024-04-24T05:47:02.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.792+0000 7f4e66000640 1 --2- 172.21.15.123:0/3924212260 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4e5006c920 0x7f4e5006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:02.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.792+0000 7f4e66000640 1 -- 172.21.15.123:0/3924212260 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f4e6008bc30 con 0x7f4e700ad860 2024-04-24T05:47:02.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.792+0000 7f4e6d400640 1 --2- 172.21.15.123:0/3924212260 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4e5006c920 0x7f4e5006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:02.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.793+0000 7f4e6d400640 1 --2- 172.21.15.123:0/3924212260 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4e5006c920 0x7f4e5006ede0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c034000 tx=0x7f4e5c02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:02.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:02.797+0000 7f4e66000640 1 -- 172.21.15.123:0/3924212260 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e6001f030 con 0x7f4e700ad860 2024-04-24T05:47:03.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.002+0000 7f4e74c00640 1 -- 172.21.15.123:0/3924212260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e701551a0 con 0x7f4e700ad860 2024-04-24T05:47:03.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.005+0000 7f4e66000640 1 -- 172.21.15.123:0/3924212260 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7f4e600172c0 con 0x7f4e700ad860 2024-04-24T05:47:03.006 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:03.007 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:03.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.007+0000 7f4e74c00640 1 -- 172.21.15.123:0/3924212260 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4e5006c920 msgr2=0x7f4e5006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:03.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.007+0000 7f4e74c00640 1 --2- 172.21.15.123:0/3924212260 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4e5006c920 0x7f4e5006ede0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c034000 tx=0x7f4e5c02c000 comp rx=0 tx=0).stop 2024-04-24T05:47:03.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.008+0000 7f4e74c00640 1 -- 172.21.15.123:0/3924212260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e700ad860 msgr2=0x7f4e700ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:03.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.008+0000 7f4e74c00640 1 --2- 172.21.15.123:0/3924212260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e700ad860 0x7f4e700ce210 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000b3f0 tx=0x7f4e6000b8b0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.008+0000 7f4e74c00640 1 -- 172.21.15.123:0/3924212260 shutdown_connections 2024-04-24T05:47:03.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.008+0000 7f4e74c00640 1 --2- 172.21.15.123:0/3924212260 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4e5006c920 0x7f4e5006ede0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.008+0000 7f4e74c00640 1 --2- 172.21.15.123:0/3924212260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e7015a2a0 0x7f4e7015f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.008+0000 7f4e74c00640 1 --2- 172.21.15.123:0/3924212260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e700af780 0x7f4e7015edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.008+0000 7f4e74c00640 1 --2- 172.21.15.123:0/3924212260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e700ad860 0x7f4e700ce210 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.008+0000 7f4e74c00640 1 -- 172.21.15.123:0/3924212260 >> 172.21.15.123:0/3924212260 conn(0x7f4e700b1e70 msgr2=0x7f4e700b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:03.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.008+0000 7f4e74c00640 1 -- 172.21.15.123:0/3924212260 shutdown_connections 2024-04-24T05:47:03.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.008+0000 7f4e74c00640 1 -- 172.21.15.123:0/3924212260 wait complete. 2024-04-24T05:47:03.029 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:47:03.030 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:47:03.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.220+0000 7f6f09c00640 1 Processor -- start 2024-04-24T05:47:03.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.220+0000 7f6f09c00640 1 -- start start 2024-04-24T05:47:03.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.222+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f0415f870 0x7f6f040d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:03.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.222+0000 7f6f09c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f04162a60 con 0x7f6f0415e5f0 2024-04-24T05:47:03.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.222+0000 7f6f09c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f04162d40 con 0x7f6f040d1420 2024-04-24T05:47:03.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.222+0000 7f6f09c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6f040d18a0 con 0x7f6f0415f870 2024-04-24T05:47:03.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.222+0000 7f6f03e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6f0415e5f0 0x7f6f04162250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49348/0 (socket says 172.21.15.123:49348) 2024-04-24T05:47:03.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.222+0000 7f6f03e00640 1 -- 172.21.15.123:0/2588950005 learned_addr learned my addr 172.21.15.123:0/2588950005 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:03.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.223+0000 7f6efbe00640 1 --2- 172.21.15.123:0/2588950005 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f0415f870 0x7f6f040d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:03.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.223+0000 7f6f03400640 1 -- 172.21.15.123:0/2588950005 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4198922494 0 0) 0x7f6f04162a60 con 0x7f6f0415e5f0 2024-04-24T05:47:03.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.223+0000 7f6f03400640 1 -- 172.21.15.123:0/2588950005 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ee8003520 con 0x7f6f0415e5f0 2024-04-24T05:47:03.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.223+0000 7f6f03400640 1 -- 172.21.15.123:0/2588950005 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1564985255 0 0) 0x7f6ee8003520 con 0x7f6f0415e5f0 2024-04-24T05:47:03.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.223+0000 7f6f03400640 1 -- 172.21.15.123:0/2588950005 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f0415f870 msgr2=0x7f6f040d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:03.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.223+0000 7f6f03400640 1 --2- 172.21.15.123:0/2588950005 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f0415f870 0x7f6f040d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.223+0000 7f6f03400640 1 -- 172.21.15.123:0/2588950005 >> v1:172.21.15.155:6789/0 conn(0x7f6f040d1420 legacy=0x7f6f040cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.223+0000 7f6f03400640 1 -- 172.21.15.123:0/2588950005 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f04163020 con 0x7f6f0415e5f0 2024-04-24T05:47:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.223+0000 7f6f03400640 1 -- 172.21.15.123:0/2588950005 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f6ef4001670 con 0x7f6f0415e5f0 2024-04-24T05:47:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.223+0000 7f6f03400640 1 -- 172.21.15.123:0/2588950005 >> v1:172.21.15.123:6789/0 conn(0x7f6f0415e5f0 legacy=0x7f6f04162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.223+0000 7f6f03400640 1 --2- 172.21.15.123:0/2588950005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee80039e0 0x7f6ee8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.223+0000 7f6f03400640 1 --2- 172.21.15.123:0/2588950005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ee8004800 0x7f6ee8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.224+0000 7f6f03400640 1 --2- 172.21.15.123:0/2588950005 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f0415f870 0x7f6ee80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.224+0000 7f6f03400640 1 -- 172.21.15.123:0/2588950005 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6f040d18a0 con 0x7f6ee80039e0 2024-04-24T05:47:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.224+0000 7f6f03400640 1 -- 172.21.15.123:0/2588950005 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee800baa0 con 0x7f6ee8004800 2024-04-24T05:47:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.224+0000 7f6f03400640 1 -- 172.21.15.123:0/2588950005 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee800bca0 con 0x7f6f0415f870 2024-04-24T05:47:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.224+0000 7f6f03e00640 1 --2- 172.21.15.123:0/2588950005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee80039e0 0x7f6ee8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.224+0000 7f6f03e00640 1 -- 172.21.15.123:0/2588950005 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f0415f870 msgr2=0x7f6ee80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.224+0000 7f6f03e00640 1 --2- 172.21.15.123:0/2588950005 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f0415f870 0x7f6ee80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.224+0000 7f6f03e00640 1 -- 172.21.15.123:0/2588950005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ee8004800 msgr2=0x7f6ee8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:03.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.224+0000 7f6f03e00640 1 --2- 172.21.15.123:0/2588950005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ee8004800 0x7f6ee8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.224+0000 7f6f03e00640 1 -- 172.21.15.123:0/2588950005 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee8003520 con 0x7f6ee80039e0 2024-04-24T05:47:03.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.225+0000 7f6f03e00640 1 --2- 172.21.15.123:0/2588950005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee80039e0 0x7f6ee8003e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400c160 tx=0x7f6ee8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baddeebb1553c192 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:03.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.225+0000 7f6f03400640 1 -- 172.21.15.123:0/2588950005 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4007c60 con 0x7f6ee80039e0 2024-04-24T05:47:03.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.225+0000 7f6f03400640 1 -- 172.21.15.123:0/2588950005 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef4005af0 con 0x7f6ee80039e0 2024-04-24T05:47:03.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.225+0000 7f6f03400640 1 -- 172.21.15.123:0/2588950005 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4007070 con 0x7f6ee80039e0 2024-04-24T05:47:03.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.226+0000 7f6f09c00640 1 -- 172.21.15.123:0/2588950005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee80039e0 msgr2=0x7f6ee8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:03.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.226+0000 7f6f09c00640 1 --2- 172.21.15.123:0/2588950005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ee80039e0 0x7f6ee8003e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400c160 tx=0x7f6ee8005830 comp rx=0 tx=0).stop 2024-04-24T05:47:03.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.226+0000 7f6f09200640 1 -- 172.21.15.123:0/2588950005 reap_dead start 2024-04-24T05:47:03.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.226+0000 7f6f09c00640 1 -- 172.21.15.123:0/2588950005 shutdown_connections 2024-04-24T05:47:03.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.226+0000 7f6f09c00640 1 -- 172.21.15.123:0/2588950005 >> 172.21.15.123:0/2588950005 conn(0x7f6f040b9db0 msgr2=0x7f6f0409d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:03.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.226+0000 7f6f09c00640 1 -- 172.21.15.123:0/2588950005 shutdown_connections 2024-04-24T05:47:03.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.226+0000 7f6f09c00640 1 -- 172.21.15.123:0/2588950005 wait complete. 2024-04-24T05:47:03.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.228+0000 7f6f09c00640 1 Processor -- start 2024-04-24T05:47:03.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.228+0000 7f6f09c00640 1 -- start start 2024-04-24T05:47:03.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.229+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ee80039e0 0x7f6f040c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:03.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.229+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f040cb5d0 0x7f6f040b6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:03.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.229+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f040b46c0 0x7f6f040b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:03.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.229+0000 7f6f09c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6f040cee30 con 0x7f6f040cb5d0 2024-04-24T05:47:03.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.229+0000 7f6f09c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6f040ceb50 con 0x7f6f040b46c0 2024-04-24T05:47:03.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.229+0000 7f6f09c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6f040d0ee0 con 0x7f6ee80039e0 2024-04-24T05:47:03.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.229+0000 7f6efbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f040cb5d0 0x7f6f040b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:03.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.229+0000 7f6efbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f040cb5d0 0x7f6f040b6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42554/0 (socket says 172.21.15.123:42554) 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.229+0000 7f6efbe00640 1 -- 172.21.15.123:0/499721663 learned_addr learned my addr 172.21.15.123:0/499721663 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.230+0000 7f6f03e00640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ee80039e0 0x7f6f040c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.230+0000 7f6f09200640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f040b46c0 0x7f6f040b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.230+0000 7f6efbe00640 1 -- 172.21.15.123:0/499721663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ee80039e0 msgr2=0x7f6f040c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.230+0000 7f6efbe00640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ee80039e0 0x7f6f040c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.230+0000 7f6efbe00640 1 -- 172.21.15.123:0/499721663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f040b46c0 msgr2=0x7f6f040b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.230+0000 7f6efbe00640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f040b46c0 0x7f6f040b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.230+0000 7f6efbe00640 1 -- 172.21.15.123:0/499721663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f040cba30 con 0x7f6f040cb5d0 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.230+0000 7f6f03e00640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ee80039e0 0x7f6f040c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.230+0000 7f6f09200640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f040b46c0 0x7f6f040b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.230+0000 7f6efbe00640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f040cb5d0 0x7f6f040b6930 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00e870 tx=0x7f6eec00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.230+0000 7f6f00c00640 1 -- 172.21.15.123:0/499721663 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec017070 con 0x7f6f040cb5d0 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.230+0000 7f6f00c00640 1 -- 172.21.15.123:0/499721663 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eec00c940 con 0x7f6f040cb5d0 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.230+0000 7f6f00c00640 1 -- 172.21.15.123:0/499721663 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec00cd10 con 0x7f6f040cb5d0 2024-04-24T05:47:03.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.230+0000 7f6f09c00640 1 -- 172.21.15.123:0/499721663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f040b6e70 con 0x7f6f040cb5d0 2024-04-24T05:47:03.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.231+0000 7f6f09c00640 1 -- 172.21.15.123:0/499721663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f040cbfb0 con 0x7f6f040cb5d0 2024-04-24T05:47:03.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.232+0000 7f6f09c00640 1 -- 172.21.15.123:0/499721663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f04162a60 con 0x7f6f040cb5d0 2024-04-24T05:47:03.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.233+0000 7f6f00c00640 1 -- 172.21.15.123:0/499721663 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6eec01a070 con 0x7f6f040cb5d0 2024-04-24T05:47:03.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.234+0000 7f6f00c00640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6ee006c8f0 0x7f6ee006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:03.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.234+0000 7f6f00c00640 1 -- 172.21.15.123:0/499721663 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f6eec08c2f0 con 0x7f6f040cb5d0 2024-04-24T05:47:03.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.234+0000 7f6f03e00640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6ee006c8f0 0x7f6ee006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:03.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.235+0000 7f6f03e00640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6ee006c8f0 0x7f6ee006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80057e0 tx=0x7f6ef402d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:03.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.238+0000 7f6f00c00640 1 -- 172.21.15.123:0/499721663 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6f04162a60 con 0x7f6f040cb5d0 2024-04-24T05:47:03.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.440+0000 7f6f09c00640 1 -- 172.21.15.123:0/499721663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6f04068a20 con 0x7f6f040cb5d0 2024-04-24T05:47:03.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.441+0000 7f6f00c00640 1 -- 172.21.15.123:0/499721663 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f6eec05a1e0 con 0x7f6f040cb5d0 2024-04-24T05:47:03.443 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:47:03.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.444+0000 7f6f09c00640 1 -- 172.21.15.123:0/499721663 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6ee006c8f0 msgr2=0x7f6ee006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:03.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.444+0000 7f6f09c00640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6ee006c8f0 0x7f6ee006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80057e0 tx=0x7f6ef402d000 comp rx=0 tx=0).stop 2024-04-24T05:47:03.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.444+0000 7f6f09c00640 1 -- 172.21.15.123:0/499721663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f040cb5d0 msgr2=0x7f6f040b6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:03.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.444+0000 7f6f09c00640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f040cb5d0 0x7f6f040b6930 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00e870 tx=0x7f6eec00ed30 comp rx=0 tx=0).stop 2024-04-24T05:47:03.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.444+0000 7f6f09c00640 1 -- 172.21.15.123:0/499721663 shutdown_connections 2024-04-24T05:47:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.444+0000 7f6f09c00640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6ee006c8f0 0x7f6ee006edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.444+0000 7f6f09c00640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f040b46c0 0x7f6f040b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.444+0000 7f6f09c00640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f040cb5d0 0x7f6f040b6930 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.444+0000 7f6f09c00640 1 --2- 172.21.15.123:0/499721663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ee80039e0 0x7f6f040c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.444+0000 7f6f09c00640 1 -- 172.21.15.123:0/499721663 >> 172.21.15.123:0/499721663 conn(0x7f6f040b9db0 msgr2=0x7f6f0415fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.444+0000 7f6f09c00640 1 -- 172.21.15.123:0/499721663 shutdown_connections 2024-04-24T05:47:03.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.444+0000 7f6f09c00640 1 -- 172.21.15.123:0/499721663 wait complete. 2024-04-24T05:47:03.464 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:47:03.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.669+0000 7f85f3600640 1 Processor -- start 2024-04-24T05:47:03.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.669+0000 7f85f3600640 1 -- start start 2024-04-24T05:47:03.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.670+0000 7f85f3600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85ec06c510 0x7f85ec071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:03.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.670+0000 7f85f3600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f85ec0cd2c0 0x7f85ec06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:03.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.670+0000 7f85f3600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85ec0cf170 con 0x7f85ec06c8f0 2024-04-24T05:47:03.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.670+0000 7f85f3600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f85ec0cf450 con 0x7f85ec0cd2c0 2024-04-24T05:47:03.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.670+0000 7f85f3600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f85ec0cf730 con 0x7f85ec06c510 2024-04-24T05:47:03.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.671+0000 7f85f1600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f85ec0cd2c0 0x7f85ec06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:03.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.671+0000 7f85f1600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f85ec0cd2c0 0x7f85ec06cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42558/0 (socket says 172.21.15.123:42558) 2024-04-24T05:47:03.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.671+0000 7f85f1600640 1 -- 172.21.15.123:0/3589211389 learned_addr learned my addr 172.21.15.123:0/3589211389 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:03.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.671+0000 7f85f1600640 1 -- 172.21.15.123:0/3589211389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85ec06c510 msgr2=0x7f85ec071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:03.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.671+0000 7f85f0c00640 1 --2- 172.21.15.123:0/3589211389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85ec06c510 0x7f85ec071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:03.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.671+0000 7f85f1600640 1 --2- 172.21.15.123:0/3589211389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85ec06c510 0x7f85ec071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.671+0000 7f85f1600640 1 -- 172.21.15.123:0/3589211389 >> v1:172.21.15.155:6789/0 conn(0x7f85ec06c8f0 legacy=0x7f85ec0ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:03.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.671+0000 7f85f1600640 1 -- 172.21.15.123:0/3589211389 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec0cfa10 con 0x7f85ec0cd2c0 2024-04-24T05:47:03.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.671+0000 7f85f0c00640 1 --2- 172.21.15.123:0/3589211389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85ec06c510 0x7f85ec071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:03.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.673+0000 7f85f1600640 1 --2- 172.21.15.123:0/3589211389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85ec0cd2c0 0x7f85ec06cd50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009e50 tx=0x7f85e402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f359dca0d3e3f54b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:03.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.673+0000 7f85eb400640 1 -- 172.21.15.123:0/3589211389 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4036070 con 0x7f85ec0cd2c0 2024-04-24T05:47:03.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.673+0000 7f85eb400640 1 -- 172.21.15.123:0/3589211389 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e4031600 con 0x7f85ec0cd2c0 2024-04-24T05:47:03.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.673+0000 7f85eb400640 1 -- 172.21.15.123:0/3589211389 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e40319d0 con 0x7f85ec0cd2c0 2024-04-24T05:47:03.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.674+0000 7f85f3600640 1 -- 172.21.15.123:0/3589211389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85ec0cd2c0 msgr2=0x7f85ec06cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:03.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.674+0000 7f85f3600640 1 --2- 172.21.15.123:0/3589211389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85ec0cd2c0 0x7f85ec06cd50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009e50 tx=0x7f85e402a6c0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.674+0000 7f85f3600640 1 -- 172.21.15.123:0/3589211389 shutdown_connections 2024-04-24T05:47:03.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.674+0000 7f85f3600640 1 --2- 172.21.15.123:0/3589211389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85ec0cd2c0 0x7f85ec06cd50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.674+0000 7f85f3600640 1 --2- 172.21.15.123:0/3589211389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85ec06c510 0x7f85ec071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.674+0000 7f85f3600640 1 -- 172.21.15.123:0/3589211389 >> 172.21.15.123:0/3589211389 conn(0x7f85ec0b51f0 msgr2=0x7f85ec0b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:03.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.674+0000 7f85f3600640 1 -- 172.21.15.123:0/3589211389 shutdown_connections 2024-04-24T05:47:03.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.674+0000 7f85f3600640 1 -- 172.21.15.123:0/3589211389 wait complete. 2024-04-24T05:47:03.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.676+0000 7f85f3600640 1 Processor -- start 2024-04-24T05:47:03.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.676+0000 7f85f3600640 1 -- start start 2024-04-24T05:47:03.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.677+0000 7f85f3600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85ec06c510 0x7f85ec156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:03.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.677+0000 7f85f3600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85ec06c8f0 0x7f85ec15f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:03.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.677+0000 7f85f3600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec0cd2c0 0x7f85ec0c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:03.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.677+0000 7f85f3600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec0cee80 con 0x7f85ec06c8f0 2024-04-24T05:47:03.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.677+0000 7f85f3600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec0ced00 con 0x7f85ec0cd2c0 2024-04-24T05:47:03.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.677+0000 7f85f3600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f85ec0d0210 con 0x7f85ec06c510 2024-04-24T05:47:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.678+0000 7f85ebe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85ec06c8f0 0x7f85ec15f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.678+0000 7f85ebe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85ec06c8f0 0x7f85ec15f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42562/0 (socket says 172.21.15.123:42562) 2024-04-24T05:47:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.678+0000 7f85ebe00640 1 -- 172.21.15.123:0/2075725833 learned_addr learned my addr 172.21.15.123:0/2075725833 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.678+0000 7f85f0c00640 1 --2- 172.21.15.123:0/2075725833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85ec06c510 0x7f85ec156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.678+0000 7f85f1600640 1 --2- 172.21.15.123:0/2075725833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec0cd2c0 0x7f85ec0c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.678+0000 7f85ebe00640 1 -- 172.21.15.123:0/2075725833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85ec06c510 msgr2=0x7f85ec156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.678+0000 7f85ebe00640 1 --2- 172.21.15.123:0/2075725833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85ec06c510 0x7f85ec156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.678+0000 7f85ebe00640 1 -- 172.21.15.123:0/2075725833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec0cd2c0 msgr2=0x7f85ec0c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.678+0000 7f85ebe00640 1 --2- 172.21.15.123:0/2075725833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec0cd2c0 0x7f85ec0c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.678+0000 7f85ebe00640 1 -- 172.21.15.123:0/2075725833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec0c3f30 con 0x7f85ec06c8f0 2024-04-24T05:47:03.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.679+0000 7f85ebe00640 1 --2- 172.21.15.123:0/2075725833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85ec06c8f0 0x7f85ec15f110 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f85d800c460 tx=0x7f85d800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:03.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.679+0000 7f85e8c00640 1 -- 172.21.15.123:0/2075725833 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8016020 con 0x7f85ec06c8f0 2024-04-24T05:47:03.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.679+0000 7f85e8c00640 1 -- 172.21.15.123:0/2075725833 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d8012870 con 0x7f85ec06c8f0 2024-04-24T05:47:03.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.679+0000 7f85e8c00640 1 -- 172.21.15.123:0/2075725833 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8012c60 con 0x7f85ec06c8f0 2024-04-24T05:47:03.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.679+0000 7f85f3600640 1 -- 172.21.15.123:0/2075725833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ec0c8900 con 0x7f85ec06c8f0 2024-04-24T05:47:03.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.679+0000 7f85f3600640 1 -- 172.21.15.123:0/2075725833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ec0c8ec0 con 0x7f85ec06c8f0 2024-04-24T05:47:03.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.680+0000 7f85f3600640 1 -- 172.21.15.123:0/2075725833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85ec068a20 con 0x7f85ec06c8f0 2024-04-24T05:47:03.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.681+0000 7f85e8c00640 1 -- 172.21.15.123:0/2075725833 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85d8019020 con 0x7f85ec06c8f0 2024-04-24T05:47:03.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.682+0000 7f85e8c00640 1 --2- 172.21.15.123:0/2075725833 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f85cc06c8f0 0x7f85cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:03.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.682+0000 7f85e8c00640 1 -- 172.21.15.123:0/2075725833 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f85d808b9f0 con 0x7f85ec06c8f0 2024-04-24T05:47:03.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.683+0000 7f85f0c00640 1 --2- 172.21.15.123:0/2075725833 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f85cc06c8f0 0x7f85cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:03.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.683+0000 7f85f0c00640 1 --2- 172.21.15.123:0/2075725833 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f85cc06c8f0 0x7f85cc06edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0cda00 tx=0x7f85e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:03.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.687+0000 7f85e8c00640 1 -- 172.21.15.123:0/2075725833 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85d8020070 con 0x7f85ec06c8f0 2024-04-24T05:47:03.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.885+0000 7f85f3600640 1 -- 172.21.15.123:0/2075725833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f85ec06f620 con 0x7f85ec06c8f0 2024-04-24T05:47:03.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.886+0000 7f85e8c00640 1 -- 172.21.15.123:0/2075725833 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f85d8059a00 con 0x7f85ec06c8f0 2024-04-24T05:47:03.888 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:03.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.889+0000 7f85f3600640 1 -- 172.21.15.123:0/2075725833 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f85cc06c8f0 msgr2=0x7f85cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:03.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.889+0000 7f85f3600640 1 --2- 172.21.15.123:0/2075725833 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f85cc06c8f0 0x7f85cc06edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0cda00 tx=0x7f85e0008040 comp rx=0 tx=0).stop 2024-04-24T05:47:03.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.889+0000 7f85f3600640 1 -- 172.21.15.123:0/2075725833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85ec06c8f0 msgr2=0x7f85ec15f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:03.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.889+0000 7f85f3600640 1 --2- 172.21.15.123:0/2075725833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85ec06c8f0 0x7f85ec15f110 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f85d800c460 tx=0x7f85d800c920 comp rx=0 tx=0).stop 2024-04-24T05:47:03.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.889+0000 7f85f3600640 1 -- 172.21.15.123:0/2075725833 shutdown_connections 2024-04-24T05:47:03.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.889+0000 7f85f3600640 1 --2- 172.21.15.123:0/2075725833 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f85cc06c8f0 0x7f85cc06edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.889+0000 7f85f3600640 1 --2- 172.21.15.123:0/2075725833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec0cd2c0 0x7f85ec0c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.889+0000 7f85f3600640 1 --2- 172.21.15.123:0/2075725833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85ec06c8f0 0x7f85ec15f110 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.889+0000 7f85f3600640 1 --2- 172.21.15.123:0/2075725833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85ec06c510 0x7f85ec156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:03.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.889+0000 7f85f3600640 1 -- 172.21.15.123:0/2075725833 >> 172.21.15.123:0/2075725833 conn(0x7f85ec0b51f0 msgr2=0x7f85ec0b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:03.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.889+0000 7f85f3600640 1 -- 172.21.15.123:0/2075725833 shutdown_connections 2024-04-24T05:47:03.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:03.889+0000 7f85f3600640 1 -- 172.21.15.123:0/2075725833 wait complete. 2024-04-24T05:47:03.909 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:47:04.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.102+0000 7fce36a00640 1 Processor -- start 2024-04-24T05:47:04.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.102+0000 7fce36a00640 1 -- start start 2024-04-24T05:47:04.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.104+0000 7fce36a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce30071c50 0x7fce300ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.104+0000 7fce36a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce300cf170 con 0x7fce300cd0f0 2024-04-24T05:47:04.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.104+0000 7fce36a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce300cf450 con 0x7fce3006c510 2024-04-24T05:47:04.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.104+0000 7fce36a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fce300cf730 con 0x7fce30071c50 2024-04-24T05:47:04.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.104+0000 7fce34a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fce300cd0f0 0x7fce300d0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49362/0 (socket says 172.21.15.123:49362) 2024-04-24T05:47:04.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.104+0000 7fce34a00640 1 -- 172.21.15.123:0/1279397298 learned_addr learned my addr 172.21.15.123:0/1279397298 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:04.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.104+0000 7fce2f400640 1 --2- 172.21.15.123:0/1279397298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce30071c50 0x7fce300ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:04.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 -- 172.21.15.123:0/1279397298 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 639968082 0 0) 0x7fce300cf170 con 0x7fce300cd0f0 2024-04-24T05:47:04.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 -- 172.21.15.123:0/1279397298 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce14003520 con 0x7fce300cd0f0 2024-04-24T05:47:04.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 -- 172.21.15.123:0/1279397298 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2365738878 0 0) 0x7fce14003520 con 0x7fce300cd0f0 2024-04-24T05:47:04.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 -- 172.21.15.123:0/1279397298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce30071c50 msgr2=0x7fce300ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:04.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 --2- 172.21.15.123:0/1279397298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce30071c50 0x7fce300ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 -- 172.21.15.123:0/1279397298 >> v1:172.21.15.155:6789/0 conn(0x7fce3006c510 legacy=0x7fce3006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:04.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 -- 172.21.15.123:0/1279397298 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce300cfa10 con 0x7fce300cd0f0 2024-04-24T05:47:04.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 -- 172.21.15.123:0/1279397298 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7fce280019f0 con 0x7fce300cd0f0 2024-04-24T05:47:04.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 -- 172.21.15.123:0/1279397298 >> v1:172.21.15.123:6789/0 conn(0x7fce300cd0f0 legacy=0x7fce300d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:04.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 --2- 172.21.15.123:0/1279397298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce30071c50 0x7fce140039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 --2- 172.21.15.123:0/1279397298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce14003f60 0x7fce14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 --2- 172.21.15.123:0/1279397298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce14004d40 0x7fce140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 -- 172.21.15.123:0/1279397298 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce300cf730 con 0x7fce14003f60 2024-04-24T05:47:04.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 -- 172.21.15.123:0/1279397298 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce1400baa0 con 0x7fce14004d40 2024-04-24T05:47:04.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2ea00640 1 -- 172.21.15.123:0/1279397298 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fce1400bca0 con 0x7fce30071c50 2024-04-24T05:47:04.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.105+0000 7fce2f400640 1 --2- 172.21.15.123:0/1279397298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce14003f60 0x7fce14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:04.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.106+0000 7fce2f400640 1 -- 172.21.15.123:0/1279397298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce30071c50 msgr2=0x7fce140039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:04.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.106+0000 7fce2f400640 1 --2- 172.21.15.123:0/1279397298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce30071c50 0x7fce140039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.106+0000 7fce2f400640 1 -- 172.21.15.123:0/1279397298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce14004d40 msgr2=0x7fce140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:04.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.106+0000 7fce2f400640 1 --2- 172.21.15.123:0/1279397298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce14004d40 0x7fce140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.106+0000 7fce2f400640 1 -- 172.21.15.123:0/1279397298 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce14003520 con 0x7fce14003f60 2024-04-24T05:47:04.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.107+0000 7fce2f400640 1 --2- 172.21.15.123:0/1279397298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce14003f60 0x7fce14004800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009ac0 tx=0x7fce14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d339b3459571a48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:04.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.107+0000 7fce2ea00640 1 -- 172.21.15.123:0/1279397298 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c036070 con 0x7fce14003f60 2024-04-24T05:47:04.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.107+0000 7fce2ea00640 1 -- 172.21.15.123:0/1279397298 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce1c031400 con 0x7fce14003f60 2024-04-24T05:47:04.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.107+0000 7fce2ea00640 1 -- 172.21.15.123:0/1279397298 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c009690 con 0x7fce14003f60 2024-04-24T05:47:04.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.108+0000 7fce36a00640 1 -- 172.21.15.123:0/1279397298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce14003f60 msgr2=0x7fce14004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:04.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.108+0000 7fce36a00640 1 --2- 172.21.15.123:0/1279397298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce14003f60 0x7fce14004800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009ac0 tx=0x7fce14005830 comp rx=0 tx=0).stop 2024-04-24T05:47:04.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.108+0000 7fce34a00640 1 -- 172.21.15.123:0/1279397298 reap_dead start 2024-04-24T05:47:04.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.108+0000 7fce36a00640 1 -- 172.21.15.123:0/1279397298 shutdown_connections 2024-04-24T05:47:04.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.108+0000 7fce36a00640 1 -- 172.21.15.123:0/1279397298 >> 172.21.15.123:0/1279397298 conn(0x7fce300b51f0 msgr2=0x7fce300b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:04.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.108+0000 7fce36a00640 1 -- 172.21.15.123:0/1279397298 shutdown_connections 2024-04-24T05:47:04.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.108+0000 7fce36a00640 1 -- 172.21.15.123:0/1279397298 wait complete. 2024-04-24T05:47:04.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.109+0000 7fce36a00640 1 Processor -- start 2024-04-24T05:47:04.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.110+0000 7fce36a00640 1 -- start start 2024-04-24T05:47:04.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.111+0000 7fce36a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce14003f60 0x7fce3009da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.111+0000 7fce36a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce300cb9c0 0x7fce300cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.111+0000 7fce36a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce300b1870 0x7fce300b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.111+0000 7fce36a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce3006d180 con 0x7fce300cb9c0 2024-04-24T05:47:04.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.111+0000 7fce36a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce3006d000 con 0x7fce300b1870 2024-04-24T05:47:04.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.111+0000 7fce36a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fce300ced00 con 0x7fce14003f60 2024-04-24T05:47:04.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.111+0000 7fce2f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce300cb9c0 0x7fce300cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:04.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.111+0000 7fce2f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce300cb9c0 0x7fce300cbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42592/0 (socket says 172.21.15.123:42592) 2024-04-24T05:47:04.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.111+0000 7fce2f400640 1 -- 172.21.15.123:0/35530673 learned_addr learned my addr 172.21.15.123:0/35530673 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:04.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.111+0000 7fce2f400640 1 -- 172.21.15.123:0/35530673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce14003f60 msgr2=0x7fce3009da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:04.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.111+0000 7fce2f400640 1 --2- 172.21.15.123:0/35530673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce14003f60 0x7fce3009da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.111+0000 7fce2f400640 1 -- 172.21.15.123:0/35530673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce300b1870 msgr2=0x7fce300b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:04.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.111+0000 7fce2f400640 1 --2- 172.21.15.123:0/35530673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce300b1870 0x7fce300b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.111+0000 7fce2f400640 1 -- 172.21.15.123:0/35530673 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3009dfb0 con 0x7fce300cb9c0 2024-04-24T05:47:04.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.112+0000 7fce2f400640 1 --2- 172.21.15.123:0/35530673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce300cb9c0 0x7fce300cbe20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fce1c034040 tx=0x7fce1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:04.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.112+0000 7fce27e00640 1 -- 172.21.15.123:0/35530673 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c040020 con 0x7fce300cb9c0 2024-04-24T05:47:04.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.112+0000 7fce27e00640 1 -- 172.21.15.123:0/35530673 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce1c003c40 con 0x7fce300cb9c0 2024-04-24T05:47:04.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.112+0000 7fce36a00640 1 -- 172.21.15.123:0/35530673 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce300b2530 con 0x7fce300cb9c0 2024-04-24T05:47:04.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.112+0000 7fce27e00640 1 -- 172.21.15.123:0/35530673 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c036040 con 0x7fce300cb9c0 2024-04-24T05:47:04.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.112+0000 7fce36a00640 1 -- 172.21.15.123:0/35530673 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce300c4410 con 0x7fce300cb9c0 2024-04-24T05:47:04.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.113+0000 7fce36a00640 1 -- 172.21.15.123:0/35530673 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce3006bfe0 con 0x7fce300cb9c0 2024-04-24T05:47:04.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.114+0000 7fce27e00640 1 -- 172.21.15.123:0/35530673 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce1c043020 con 0x7fce300cb9c0 2024-04-24T05:47:04.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.115+0000 7fce27e00640 1 --2- 172.21.15.123:0/35530673 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fce1006c8f0 0x7fce1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.115+0000 7fce27e00640 1 -- 172.21.15.123:0/35530673 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fce1c0ac1b0 con 0x7fce300cb9c0 2024-04-24T05:47:04.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.115+0000 7fce2fe00640 1 --2- 172.21.15.123:0/35530673 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fce1006c8f0 0x7fce1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:04.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.116+0000 7fce2fe00640 1 --2- 172.21.15.123:0/35530673 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fce1006c8f0 0x7fce1006edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fce20001600 tx=0x7fce20007990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:04.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.119+0000 7fce27e00640 1 -- 172.21.15.123:0/35530673 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce3006bfe0 con 0x7fce300cb9c0 2024-04-24T05:47:04.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.324+0000 7fce36a00640 1 -- 172.21.15.123:0/35530673 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce30068a20 con 0x7fce300cb9c0 2024-04-24T05:47:04.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.325+0000 7fce27e00640 1 -- 172.21.15.123:0/35530673 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fce1c04a070 con 0x7fce300cb9c0 2024-04-24T05:47:04.326 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:47:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.327+0000 7fce36a00640 1 -- 172.21.15.123:0/35530673 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fce1006c8f0 msgr2=0x7fce1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.327+0000 7fce36a00640 1 --2- 172.21.15.123:0/35530673 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fce1006c8f0 0x7fce1006edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fce20001600 tx=0x7fce20007990 comp rx=0 tx=0).stop 2024-04-24T05:47:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.327+0000 7fce36a00640 1 -- 172.21.15.123:0/35530673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce300cb9c0 msgr2=0x7fce300cbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.327+0000 7fce36a00640 1 --2- 172.21.15.123:0/35530673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce300cb9c0 0x7fce300cbe20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fce1c034040 tx=0x7fce1c004730 comp rx=0 tx=0).stop 2024-04-24T05:47:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.327+0000 7fce36a00640 1 -- 172.21.15.123:0/35530673 shutdown_connections 2024-04-24T05:47:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.327+0000 7fce36a00640 1 --2- 172.21.15.123:0/35530673 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fce1006c8f0 0x7fce1006edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.327+0000 7fce36a00640 1 --2- 172.21.15.123:0/35530673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce300b1870 0x7fce300b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.327+0000 7fce36a00640 1 --2- 172.21.15.123:0/35530673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce300cb9c0 0x7fce300cbe20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.327+0000 7fce36a00640 1 --2- 172.21.15.123:0/35530673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce14003f60 0x7fce3009da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.327+0000 7fce36a00640 1 -- 172.21.15.123:0/35530673 >> 172.21.15.123:0/35530673 conn(0x7fce300b51f0 msgr2=0x7fce30072030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:04.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.327+0000 7fce36a00640 1 -- 172.21.15.123:0/35530673 shutdown_connections 2024-04-24T05:47:04.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.327+0000 7fce36a00640 1 -- 172.21.15.123:0/35530673 wait complete. 2024-04-24T05:47:04.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:47:04.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.551+0000 7f3967800640 1 Processor -- start 2024-04-24T05:47:04.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.551+0000 7f3967800640 1 -- start start 2024-04-24T05:47:04.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.552+0000 7f3967800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f39600ae990 0x7f39600d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.552+0000 7f3967800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f396006c410 0x7f39600ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.552+0000 7f3967800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39600d0740 con 0x7f39600ce610 2024-04-24T05:47:04.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.552+0000 7f3967800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f39600cea70 con 0x7f39600ae990 2024-04-24T05:47:04.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.552+0000 7f3967800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f396006dcb0 con 0x7f396006c410 2024-04-24T05:47:04.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.553+0000 7f3965800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f39600ce610 0x7f396015f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49378/0 (socket says 172.21.15.123:49378) 2024-04-24T05:47:04.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.553+0000 7f3965800640 1 -- 172.21.15.123:0/1306426783 learned_addr learned my addr 172.21.15.123:0/1306426783 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:04.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.553+0000 7f3964e00640 1 --2- 172.21.15.123:0/1306426783 >> v2:172.21.15.155:3300/0 conn(0x7f39600ae990 0x7f39600d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:04.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.553+0000 7f395fe00640 1 --2- 172.21.15.123:0/1306426783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f396006c410 0x7f39600ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:04.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.553+0000 7f395f400640 1 -- 172.21.15.123:0/1306426783 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4039603761 0 0) 0x7f39600d0740 con 0x7f39600ce610 2024-04-24T05:47:04.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.553+0000 7f395f400640 1 -- 172.21.15.123:0/1306426783 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3948003520 con 0x7f39600ce610 2024-04-24T05:47:04.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.553+0000 7f395f400640 1 -- 172.21.15.123:0/1306426783 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2300190712 0 0) 0x7f3948003520 con 0x7f39600ce610 2024-04-24T05:47:04.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.553+0000 7f395f400640 1 -- 172.21.15.123:0/1306426783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f396006c410 msgr2=0x7f39600ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:04.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.553+0000 7f395f400640 1 --2- 172.21.15.123:0/1306426783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f396006c410 0x7f39600ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.553+0000 7f395f400640 1 -- 172.21.15.123:0/1306426783 >> v2:172.21.15.155:3300/0 conn(0x7f39600ae990 msgr2=0x7f39600d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:04.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.553+0000 7f395f400640 1 --2- 172.21.15.123:0/1306426783 >> v2:172.21.15.155:3300/0 conn(0x7f39600ae990 0x7f39600d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.553+0000 7f395f400640 1 -- 172.21.15.123:0/1306426783 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39600d0a20 con 0x7f39600ce610 2024-04-24T05:47:04.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.553+0000 7f395f400640 1 -- 172.21.15.123:0/1306426783 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f39580019f0 con 0x7f39600ce610 2024-04-24T05:47:04.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.553+0000 7f395f400640 1 -- 172.21.15.123:0/1306426783 >> v1:172.21.15.123:6789/0 conn(0x7f39600ce610 legacy=0x7f396015f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:04.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.554+0000 7f395f400640 1 --2- 172.21.15.123:0/1306426783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39480039b0 0x7f3948003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.554+0000 7f395f400640 1 --2- 172.21.15.123:0/1306426783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f396006c410 0x7f3948004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.554+0000 7f395f400640 1 --2- 172.21.15.123:0/1306426783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3948004d40 0x7f39480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.554+0000 7f395f400640 1 -- 172.21.15.123:0/1306426783 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f396006dcb0 con 0x7f39480039b0 2024-04-24T05:47:04.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.554+0000 7f395f400640 1 -- 172.21.15.123:0/1306426783 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f394800dbd0 con 0x7f3948004d40 2024-04-24T05:47:04.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.554+0000 7f395f400640 1 -- 172.21.15.123:0/1306426783 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f394800dd50 con 0x7f396006c410 2024-04-24T05:47:04.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.554+0000 7f3964e00640 1 --2- 172.21.15.123:0/1306426783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39480039b0 0x7f3948003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:04.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.554+0000 7f3964e00640 1 -- 172.21.15.123:0/1306426783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f396006c410 msgr2=0x7f3948004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:04.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.554+0000 7f395fe00640 1 --2- 172.21.15.123:0/1306426783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f396006c410 0x7f3948004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:04.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.554+0000 7f3964e00640 1 --2- 172.21.15.123:0/1306426783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f396006c410 0x7f3948004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.554+0000 7f3964e00640 1 -- 172.21.15.123:0/1306426783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3948004d40 msgr2=0x7f39480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:04.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.554+0000 7f3964e00640 1 --2- 172.21.15.123:0/1306426783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3948004d40 0x7f39480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.554+0000 7f3964e00640 1 -- 172.21.15.123:0/1306426783 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3948003520 con 0x7f39480039b0 2024-04-24T05:47:04.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.555+0000 7f3964e00640 1 --2- 172.21.15.123:0/1306426783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39480039b0 0x7f3948003dd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3954009ac0 tx=0x7f3948005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cec2cd5175dc16 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:04.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.556+0000 7f395f400640 1 -- 172.21.15.123:0/1306426783 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954036070 con 0x7f39480039b0 2024-04-24T05:47:04.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.556+0000 7f395f400640 1 -- 172.21.15.123:0/1306426783 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3954031400 con 0x7f39480039b0 2024-04-24T05:47:04.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.556+0000 7f395f400640 1 -- 172.21.15.123:0/1306426783 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954009690 con 0x7f39480039b0 2024-04-24T05:47:04.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.556+0000 7f3967800640 1 -- 172.21.15.123:0/1306426783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39480039b0 msgr2=0x7f3948003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:04.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.556+0000 7f3967800640 1 --2- 172.21.15.123:0/1306426783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39480039b0 0x7f3948003dd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3954009ac0 tx=0x7f3948005830 comp rx=0 tx=0).stop 2024-04-24T05:47:04.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.556+0000 7f3965800640 1 -- 172.21.15.123:0/1306426783 reap_dead start 2024-04-24T05:47:04.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.556+0000 7f3967800640 1 -- 172.21.15.123:0/1306426783 shutdown_connections 2024-04-24T05:47:04.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.556+0000 7f3967800640 1 -- 172.21.15.123:0/1306426783 >> 172.21.15.123:0/1306426783 conn(0x7f39600b51f0 msgr2=0x7f39600b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:04.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.556+0000 7f3967800640 1 -- 172.21.15.123:0/1306426783 shutdown_connections 2024-04-24T05:47:04.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.556+0000 7f3967800640 1 -- 172.21.15.123:0/1306426783 wait complete. 2024-04-24T05:47:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.558+0000 7f3967800640 1 Processor -- start 2024-04-24T05:47:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.558+0000 7f3967800640 1 -- start start 2024-04-24T05:47:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.559+0000 7f3967800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39480039b0 0x7f396006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.559+0000 7f3967800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396015e570 0x7f396006f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.559+0000 7f3967800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39600ca240 0x7f39600ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.559+0000 7f3967800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f39600aed70 con 0x7f39600ca240 2024-04-24T05:47:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.559+0000 7f3967800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f396006e270 con 0x7f396015e570 2024-04-24T05:47:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.559+0000 7f3967800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f396006c870 con 0x7f39480039b0 2024-04-24T05:47:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.560+0000 7f3965800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39600ca240 0x7f39600ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.560+0000 7f3965800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39600ca240 0x7f39600ca800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42616/0 (socket says 172.21.15.123:42616) 2024-04-24T05:47:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.560+0000 7f3965800640 1 -- 172.21.15.123:0/2059127428 learned_addr learned my addr 172.21.15.123:0/2059127428 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.560+0000 7f395fe00640 1 --2- 172.21.15.123:0/2059127428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396015e570 0x7f396006f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:04.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.560+0000 7f3964e00640 1 --2- 172.21.15.123:0/2059127428 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39480039b0 0x7f396006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:04.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.560+0000 7f3965800640 1 -- 172.21.15.123:0/2059127428 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39480039b0 msgr2=0x7f396006ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:04.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.560+0000 7f3965800640 1 --2- 172.21.15.123:0/2059127428 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39480039b0 0x7f396006ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.560+0000 7f3965800640 1 -- 172.21.15.123:0/2059127428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396015e570 msgr2=0x7f396006f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:04.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.560+0000 7f3965800640 1 --2- 172.21.15.123:0/2059127428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396015e570 0x7f396006f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.560+0000 7f3965800640 1 -- 172.21.15.123:0/2059127428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39600c8200 con 0x7f39600ca240 2024-04-24T05:47:04.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.560+0000 7f3965800640 1 --2- 172.21.15.123:0/2059127428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39600ca240 0x7f39600ca800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f396015f370 tx=0x7f395800ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:04.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.561+0000 7f395cc00640 1 -- 172.21.15.123:0/2059127428 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395800a640 con 0x7f39600ca240 2024-04-24T05:47:04.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.561+0000 7f395cc00640 1 -- 172.21.15.123:0/2059127428 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f395800a7e0 con 0x7f39600ca240 2024-04-24T05:47:04.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.561+0000 7f3967800640 1 -- 172.21.15.123:0/2059127428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39600c7440 con 0x7f39600ca240 2024-04-24T05:47:04.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.561+0000 7f395cc00640 1 -- 172.21.15.123:0/2059127428 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395800abb0 con 0x7f39600ca240 2024-04-24T05:47:04.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.561+0000 7f3967800640 1 -- 172.21.15.123:0/2059127428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39600c7a00 con 0x7f39600ca240 2024-04-24T05:47:04.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.562+0000 7f3967800640 1 -- 172.21.15.123:0/2059127428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3960068a20 con 0x7f39600ca240 2024-04-24T05:47:04.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.563+0000 7f395cc00640 1 -- 172.21.15.123:0/2059127428 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f395800ad50 con 0x7f39600ca240 2024-04-24T05:47:04.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.564+0000 7f395cc00640 1 --2- 172.21.15.123:0/2059127428 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f394006c8f0 0x7f394006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:04.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.565+0000 7f395cc00640 1 -- 172.21.15.123:0/2059127428 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f395808ba40 con 0x7f39600ca240 2024-04-24T05:47:04.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.565+0000 7f3964e00640 1 --2- 172.21.15.123:0/2059127428 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f394006c8f0 0x7f394006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:04.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.565+0000 7f3964e00640 1 --2- 172.21.15.123:0/2059127428 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f394006c8f0 0x7f394006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3954033000 tx=0x7f3954008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:04.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.569+0000 7f395cc00640 1 -- 172.21.15.123:0/2059127428 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3958059a50 con 0x7f39600ca240 2024-04-24T05:47:04.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.777+0000 7f3967800640 1 -- 172.21.15.123:0/2059127428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f39600c6f80 con 0x7f39600ca240 2024-04-24T05:47:04.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.778+0000 7f395cc00640 1 -- 172.21.15.123:0/2059127428 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f3958059350 con 0x7f39600ca240 2024-04-24T05:47:04.780 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:04.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.781+0000 7f3967800640 1 -- 172.21.15.123:0/2059127428 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f394006c8f0 msgr2=0x7f394006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:04.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.781+0000 7f3967800640 1 --2- 172.21.15.123:0/2059127428 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f394006c8f0 0x7f394006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3954033000 tx=0x7f3954008030 comp rx=0 tx=0).stop 2024-04-24T05:47:04.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.781+0000 7f3967800640 1 -- 172.21.15.123:0/2059127428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39600ca240 msgr2=0x7f39600ca800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:04.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.781+0000 7f3967800640 1 --2- 172.21.15.123:0/2059127428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39600ca240 0x7f39600ca800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f396015f370 tx=0x7f395800ce40 comp rx=0 tx=0).stop 2024-04-24T05:47:04.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.781+0000 7f3967800640 1 -- 172.21.15.123:0/2059127428 shutdown_connections 2024-04-24T05:47:04.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.781+0000 7f3967800640 1 --2- 172.21.15.123:0/2059127428 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f394006c8f0 0x7f394006edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.781+0000 7f3967800640 1 --2- 172.21.15.123:0/2059127428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39600ca240 0x7f39600ca800 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.781+0000 7f3967800640 1 --2- 172.21.15.123:0/2059127428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f396015e570 0x7f396006f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.781+0000 7f3967800640 1 --2- 172.21.15.123:0/2059127428 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39480039b0 0x7f396006ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:04.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.781+0000 7f3967800640 1 -- 172.21.15.123:0/2059127428 >> 172.21.15.123:0/2059127428 conn(0x7f39600b51f0 msgr2=0x7f396015b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:04.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.782+0000 7f3967800640 1 -- 172.21.15.123:0/2059127428 shutdown_connections 2024-04-24T05:47:04.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.782+0000 7f3967800640 1 -- 172.21.15.123:0/2059127428 wait complete. 2024-04-24T05:47:04.801 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T05:47:05.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.998+0000 7f09f4000640 1 Processor -- start 2024-04-24T05:47:05.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:04.998+0000 7f09f4000640 1 -- start start 2024-04-24T05:47:05.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.000+0000 7f09f4000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09ec15b5d0 0x7f09ec0cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:05.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.000+0000 7f09f4000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f09ec15f030 0x7f09ec15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:05.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.000+0000 7f09f4000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09ec15fe50 con 0x7f09ec15fa00 2024-04-24T05:47:05.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.000+0000 7f09f4000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f09ec0af850 con 0x7f09ec15b5d0 2024-04-24T05:47:05.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.000+0000 7f09f4000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f09ec0aec40 con 0x7f09ec15f030 2024-04-24T05:47:05.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.000+0000 7f09f1600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09ec15b5d0 0x7f09ec0cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:05.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.000+0000 7f09f1600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09ec15b5d0 0x7f09ec0cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42630/0 (socket says 172.21.15.123:42630) 2024-04-24T05:47:05.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.000+0000 7f09f1600640 1 -- 172.21.15.123:0/2077320410 learned_addr learned my addr 172.21.15.123:0/2077320410 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:05.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.000+0000 7f09f0c00640 1 --2- 172.21.15.123:0/2077320410 >> v2:172.21.15.155:3300/0 conn(0x7f09ec15f030 0x7f09ec15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:05.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.000+0000 7f09f1600640 1 -- 172.21.15.123:0/2077320410 >> v2:172.21.15.155:3300/0 conn(0x7f09ec15f030 msgr2=0x7f09ec15f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:05.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.000+0000 7f09f1600640 1 --2- 172.21.15.123:0/2077320410 >> v2:172.21.15.155:3300/0 conn(0x7f09ec15f030 0x7f09ec15f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.000+0000 7f09f1600640 1 -- 172.21.15.123:0/2077320410 >> v1:172.21.15.123:6789/0 conn(0x7f09ec15fa00 legacy=0x7f09ec15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:05.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.000+0000 7f09f1600640 1 -- 172.21.15.123:0/2077320410 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec0aef20 con 0x7f09ec15b5d0 2024-04-24T05:47:05.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.000+0000 7f09f0c00640 1 --2- 172.21.15.123:0/2077320410 >> v2:172.21.15.155:3300/0 conn(0x7f09ec15f030 0x7f09ec15f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:05.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.001+0000 7f09f1600640 1 --2- 172.21.15.123:0/2077320410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09ec15b5d0 0x7f09ec0cc580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f09ec06d4e0 tx=0x7f09e002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5ce7a32963fc75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:05.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.002+0000 7f09ebe00640 1 -- 172.21.15.123:0/2077320410 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0036070 con 0x7f09ec15b5d0 2024-04-24T05:47:05.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.002+0000 7f09ebe00640 1 -- 172.21.15.123:0/2077320410 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e0031600 con 0x7f09ec15b5d0 2024-04-24T05:47:05.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.002+0000 7f09ebe00640 1 -- 172.21.15.123:0/2077320410 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e00319d0 con 0x7f09ec15b5d0 2024-04-24T05:47:05.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.002+0000 7f09f4000640 1 -- 172.21.15.123:0/2077320410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09ec15b5d0 msgr2=0x7f09ec0cc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:05.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.002+0000 7f09f4000640 1 --2- 172.21.15.123:0/2077320410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09ec15b5d0 0x7f09ec0cc580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f09ec06d4e0 tx=0x7f09e002a6a0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.003+0000 7f09f4000640 1 -- 172.21.15.123:0/2077320410 shutdown_connections 2024-04-24T05:47:05.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.003+0000 7f09f4000640 1 --2- 172.21.15.123:0/2077320410 >> v2:172.21.15.155:3300/0 conn(0x7f09ec15f030 0x7f09ec15f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.003+0000 7f09f4000640 1 --2- 172.21.15.123:0/2077320410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09ec15b5d0 0x7f09ec0cc580 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.003+0000 7f09f4000640 1 -- 172.21.15.123:0/2077320410 >> 172.21.15.123:0/2077320410 conn(0x7f09ec0b1e70 msgr2=0x7f09ec0b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:05.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.003+0000 7f09f4000640 1 -- 172.21.15.123:0/2077320410 shutdown_connections 2024-04-24T05:47:05.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.003+0000 7f09f4000640 1 -- 172.21.15.123:0/2077320410 wait complete. 2024-04-24T05:47:05.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.004+0000 7f09f4000640 1 Processor -- start 2024-04-24T05:47:05.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.004+0000 7f09f4000640 1 -- start start 2024-04-24T05:47:05.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.005+0000 7f09f4000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09ec15b5d0 0x7f09ec1f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:05.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.005+0000 7f09f4000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09ec15f030 0x7f09ec158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:05.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.005+0000 7f09f4000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09ec15fa00 0x7f09ec156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:05.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.005+0000 7f09f4000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec06ec80 con 0x7f09ec15b5d0 2024-04-24T05:47:05.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.005+0000 7f09f4000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec0af4e0 con 0x7f09ec15f030 2024-04-24T05:47:05.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.005+0000 7f09f4000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f09ec06ee00 con 0x7f09ec15fa00 2024-04-24T05:47:05.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.006+0000 7f09f1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09ec15b5d0 0x7f09ec1f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:05.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.006+0000 7f09f1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09ec15b5d0 0x7f09ec1f72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42632/0 (socket says 172.21.15.123:42632) 2024-04-24T05:47:05.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.006+0000 7f09f1600640 1 -- 172.21.15.123:0/834748327 learned_addr learned my addr 172.21.15.123:0/834748327 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:05.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.006+0000 7f09f0c00640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09ec15f030 0x7f09ec158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:05.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.006+0000 7f09f1600640 1 -- 172.21.15.123:0/834748327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09ec15fa00 msgr2=0x7f09ec156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:05.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.006+0000 7f09f2000640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09ec15fa00 0x7f09ec156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:05.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.006+0000 7f09f1600640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09ec15fa00 0x7f09ec156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.006+0000 7f09f1600640 1 -- 172.21.15.123:0/834748327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09ec15f030 msgr2=0x7f09ec158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:05.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.006+0000 7f09f1600640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09ec15f030 0x7f09ec158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.006+0000 7f09f1600640 1 -- 172.21.15.123:0/834748327 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec159780 con 0x7f09ec15b5d0 2024-04-24T05:47:05.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.006+0000 7f09f0c00640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09ec15f030 0x7f09ec158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:05.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.007+0000 7f09f1600640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09ec15b5d0 0x7f09ec1f72e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009f10 tx=0x7f09e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:05.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.007+0000 7f09e9600640 1 -- 172.21.15.123:0/834748327 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0041020 con 0x7f09ec15b5d0 2024-04-24T05:47:05.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.007+0000 7f09e9600640 1 -- 172.21.15.123:0/834748327 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e00087c0 con 0x7f09ec15b5d0 2024-04-24T05:47:05.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.007+0000 7f09e9600640 1 -- 172.21.15.123:0/834748327 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0036040 con 0x7f09ec15b5d0 2024-04-24T05:47:05.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.007+0000 7f09f4000640 1 -- 172.21.15.123:0/834748327 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ec159a60 con 0x7f09ec15b5d0 2024-04-24T05:47:05.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.007+0000 7f09f4000640 1 -- 172.21.15.123:0/834748327 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ec15a020 con 0x7f09ec15b5d0 2024-04-24T05:47:05.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.008+0000 7f09f4000640 1 -- 172.21.15.123:0/834748327 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ec068a20 con 0x7f09ec15b5d0 2024-04-24T05:47:05.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.009+0000 7f09e9600640 1 -- 172.21.15.123:0/834748327 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09e0044020 con 0x7f09ec15b5d0 2024-04-24T05:47:05.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.010+0000 7f09e9600640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f09cc06c8f0 0x7f09cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:05.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.010+0000 7f09e9600640 1 -- 172.21.15.123:0/834748327 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f09e00acda0 con 0x7f09ec15b5d0 2024-04-24T05:47:05.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.010+0000 7f09f0c00640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f09cc06c8f0 0x7f09cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:05.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.011+0000 7f09f0c00640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f09cc06c8f0 0x7f09cc06edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f09d8006fd0 tx=0x7f09d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:05.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.014+0000 7f09e9600640 1 -- 172.21.15.123:0/834748327 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f09e004b030 con 0x7f09ec15b5d0 2024-04-24T05:47:05.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.234+0000 7f09f4000640 1 -- 172.21.15.123:0/834748327 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f09ec15f6d0 con 0x7f09ec15b5d0 2024-04-24T05:47:05.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.235+0000 7f09e9600640 1 -- 172.21.15.123:0/834748327 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f09e007ad20 con 0x7f09ec15b5d0 2024-04-24T05:47:05.237 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-24T05:47:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.238+0000 7f09f4000640 1 -- 172.21.15.123:0/834748327 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f09cc06c8f0 msgr2=0x7f09cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.238+0000 7f09f4000640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f09cc06c8f0 0x7f09cc06edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f09d8006fd0 tx=0x7f09d8008040 comp rx=0 tx=0).stop 2024-04-24T05:47:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.238+0000 7f09f4000640 1 -- 172.21.15.123:0/834748327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09ec15b5d0 msgr2=0x7f09ec1f72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.238+0000 7f09f4000640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09ec15b5d0 0x7f09ec1f72e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009f10 tx=0x7f09e0004a40 comp rx=0 tx=0).stop 2024-04-24T05:47:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.238+0000 7f09f4000640 1 -- 172.21.15.123:0/834748327 shutdown_connections 2024-04-24T05:47:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.238+0000 7f09f4000640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f09cc06c8f0 0x7f09cc06edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.238+0000 7f09f4000640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09ec15fa00 0x7f09ec156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.238+0000 7f09f4000640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09ec15f030 0x7f09ec158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.238+0000 7f09f4000640 1 --2- 172.21.15.123:0/834748327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09ec15b5d0 0x7f09ec1f72e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.238+0000 7f09f4000640 1 -- 172.21.15.123:0/834748327 >> 172.21.15.123:0/834748327 conn(0x7f09ec0b1e70 msgr2=0x7f09ec15c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:05.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.239+0000 7f09f4000640 1 -- 172.21.15.123:0/834748327 shutdown_connections 2024-04-24T05:47:05.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.239+0000 7f09f4000640 1 -- 172.21.15.123:0/834748327 wait complete. 2024-04-24T05:47:05.259 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:47:05.259 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T05:47:05.306 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:05.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.544+0000 7f4328a00640 1 Processor -- start 2024-04-24T05:47:05.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.544+0000 7f4328a00640 1 -- start start 2024-04-24T05:47:05.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.546+0000 7f4328a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f43240d04b0 0x7f4324162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:05.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.546+0000 7f4328a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f432415ebd0 0x7f4324168860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.546+0000 7f4328a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43240b6cf0 con 0x7f432415e7f0 2024-04-24T05:47:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.546+0000 7f4328a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4324163480 con 0x7f43240d04b0 2024-04-24T05:47:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.546+0000 7f4328a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f43240cebc0 con 0x7f432415ebd0 2024-04-24T05:47:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.547+0000 7f431aa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f43240d04b0 0x7f4324162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.547+0000 7f431aa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f43240d04b0 0x7f4324162f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42642/0 (socket says 172.21.15.123:42642) 2024-04-24T05:47:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.547+0000 7f431aa00640 1 -- 172.21.15.123:0/1209869623 learned_addr learned my addr 172.21.15.123:0/1209869623 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.547+0000 7f4323e00640 1 --2- 172.21.15.123:0/1209869623 >> v2:172.21.15.155:3300/0 conn(0x7f432415ebd0 0x7f4324168860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.547+0000 7f431aa00640 1 -- 172.21.15.123:0/1209869623 >> v2:172.21.15.155:3300/0 conn(0x7f432415ebd0 msgr2=0x7f4324168860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.547+0000 7f431aa00640 1 --2- 172.21.15.123:0/1209869623 >> v2:172.21.15.155:3300/0 conn(0x7f432415ebd0 0x7f4324168860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.547+0000 7f431aa00640 1 -- 172.21.15.123:0/1209869623 >> v1:172.21.15.155:6789/0 conn(0x7f432415e7f0 legacy=0x7f43240cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.547+0000 7f431aa00640 1 -- 172.21.15.123:0/1209869623 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43240cee10 con 0x7f43240d04b0 2024-04-24T05:47:05.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.547+0000 7f4323e00640 1 --2- 172.21.15.123:0/1209869623 >> v2:172.21.15.155:3300/0 conn(0x7f432415ebd0 0x7f4324168860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:47:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.548+0000 7f431aa00640 1 --2- 172.21.15.123:0/1209869623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43240d04b0 0x7f4324162f40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f432408d780 tx=0x7f430c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3658ff7d40b517d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.549+0000 7f4322a00640 1 -- 172.21.15.123:0/1209869623 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c037070 con 0x7f43240d04b0 2024-04-24T05:47:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.549+0000 7f4322a00640 1 -- 172.21.15.123:0/1209869623 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430c032440 con 0x7f43240d04b0 2024-04-24T05:47:05.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.549+0000 7f4322a00640 1 -- 172.21.15.123:0/1209869623 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c032810 con 0x7f43240d04b0 2024-04-24T05:47:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.549+0000 7f4328a00640 1 -- 172.21.15.123:0/1209869623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43240d04b0 msgr2=0x7f4324162f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.549+0000 7f4328a00640 1 --2- 172.21.15.123:0/1209869623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43240d04b0 0x7f4324162f40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f432408d780 tx=0x7f430c02a1e0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.549+0000 7f4328a00640 1 -- 172.21.15.123:0/1209869623 shutdown_connections 2024-04-24T05:47:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.549+0000 7f4328a00640 1 --2- 172.21.15.123:0/1209869623 >> v2:172.21.15.155:3300/0 conn(0x7f432415ebd0 0x7f4324168860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.549+0000 7f4328a00640 1 --2- 172.21.15.123:0/1209869623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43240d04b0 0x7f4324162f40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.549+0000 7f4328a00640 1 -- 172.21.15.123:0/1209869623 >> 172.21.15.123:0/1209869623 conn(0x7f43240b9530 msgr2=0x7f43240b9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:05.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.550+0000 7f4328a00640 1 -- 172.21.15.123:0/1209869623 shutdown_connections 2024-04-24T05:47:05.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.550+0000 7f4328a00640 1 -- 172.21.15.123:0/1209869623 wait complete. 2024-04-24T05:47:05.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.551+0000 7f4328a00640 1 Processor -- start 2024-04-24T05:47:05.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.552+0000 7f4328a00640 1 -- start start 2024-04-24T05:47:05.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.552+0000 7f4328a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43240d04b0 0x7f432415da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:05.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.552+0000 7f4328a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f432415e7f0 0x7f4324160d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:05.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.553+0000 7f4328a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f432415ebd0 0x7f432406dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:05.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.553+0000 7f4328a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43240cf8f0 con 0x7f432415ebd0 2024-04-24T05:47:05.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.553+0000 7f4328a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f43240cf3d0 con 0x7f43240d04b0 2024-04-24T05:47:05.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.553+0000 7f4328a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f43240cfa70 con 0x7f432415e7f0 2024-04-24T05:47:05.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.553+0000 7f4323e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f432415ebd0 0x7f432406dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:05.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.553+0000 7f4323e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f432415ebd0 0x7f432406dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42644/0 (socket says 172.21.15.123:42644) 2024-04-24T05:47:05.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.553+0000 7f4323e00640 1 -- 172.21.15.123:0/3318491593 learned_addr learned my addr 172.21.15.123:0/3318491593 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:05.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.553+0000 7f4323400640 1 --2- 172.21.15.123:0/3318491593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43240d04b0 0x7f432415da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:05.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.553+0000 7f4323e00640 1 -- 172.21.15.123:0/3318491593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f432415e7f0 msgr2=0x7f4324160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:05.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.553+0000 7f4323e00640 1 --2- 172.21.15.123:0/3318491593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f432415e7f0 0x7f4324160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.553+0000 7f4323e00640 1 -- 172.21.15.123:0/3318491593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43240d04b0 msgr2=0x7f432415da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:05.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.553+0000 7f4323e00640 1 --2- 172.21.15.123:0/3318491593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43240d04b0 0x7f432415da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.553+0000 7f4323e00640 1 -- 172.21.15.123:0/3318491593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43241e86f0 con 0x7f432415ebd0 2024-04-24T05:47:05.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.554+0000 7f4323e00640 1 --2- 172.21.15.123:0/3318491593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f432415ebd0 0x7f432406dbd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f431c007ec0 tx=0x7f431c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:05.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.554+0000 7f431be00640 1 -- 172.21.15.123:0/3318491593 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c01a070 con 0x7f432415ebd0 2024-04-24T05:47:05.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.554+0000 7f431be00640 1 -- 172.21.15.123:0/3318491593 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f431c003b90 con 0x7f432415ebd0 2024-04-24T05:47:05.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.554+0000 7f431be00640 1 -- 172.21.15.123:0/3318491593 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c004030 con 0x7f432415ebd0 2024-04-24T05:47:05.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.554+0000 7f4328a00640 1 -- 172.21.15.123:0/3318491593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43241e89d0 con 0x7f432415ebd0 2024-04-24T05:47:05.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.554+0000 7f4328a00640 1 -- 172.21.15.123:0/3318491593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43241e8f90 con 0x7f432415ebd0 2024-04-24T05:47:05.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.555+0000 7f4328a00640 1 -- 172.21.15.123:0/3318491593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4324075b70 con 0x7f432415ebd0 2024-04-24T05:47:05.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.556+0000 7f431be00640 1 -- 172.21.15.123:0/3318491593 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f431c01d070 con 0x7f432415ebd0 2024-04-24T05:47:05.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.557+0000 7f431be00640 1 --2- 172.21.15.123:0/3318491593 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f430006c8f0 0x7f430006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:05.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.557+0000 7f431be00640 1 -- 172.21.15.123:0/3318491593 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f431c08c340 con 0x7f432415ebd0 2024-04-24T05:47:05.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.558+0000 7f4323400640 1 --2- 172.21.15.123:0/3318491593 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f430006c8f0 0x7f430006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:05.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.558+0000 7f4323400640 1 --2- 172.21.15.123:0/3318491593 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f430006c8f0 0x7f430006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4314007fd0 tx=0x7f4314009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:05.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.562+0000 7f431be00640 1 -- 172.21.15.123:0/3318491593 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f431c010070 con 0x7f432415ebd0 2024-04-24T05:47:05.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.765+0000 7f4328a00640 1 -- 172.21.15.123:0/3318491593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43240d5120 con 0x7f432415ebd0 2024-04-24T05:47:05.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.768+0000 7f431be00640 1 -- 172.21.15.123:0/3318491593 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7f431c05a2c0 con 0x7f432415ebd0 2024-04-24T05:47:05.770 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:05.771 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:05.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.771+0000 7f4328a00640 1 -- 172.21.15.123:0/3318491593 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f430006c8f0 msgr2=0x7f430006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:05.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.771+0000 7f4328a00640 1 --2- 172.21.15.123:0/3318491593 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f430006c8f0 0x7f430006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4314007fd0 tx=0x7f4314009040 comp rx=0 tx=0).stop 2024-04-24T05:47:05.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.771+0000 7f4328a00640 1 -- 172.21.15.123:0/3318491593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f432415ebd0 msgr2=0x7f432406dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:05.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.771+0000 7f4328a00640 1 --2- 172.21.15.123:0/3318491593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f432415ebd0 0x7f432406dbd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f431c007ec0 tx=0x7f431c00d4a0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.771+0000 7f4328a00640 1 -- 172.21.15.123:0/3318491593 shutdown_connections 2024-04-24T05:47:05.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.771+0000 7f4328a00640 1 --2- 172.21.15.123:0/3318491593 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f430006c8f0 0x7f430006edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.771+0000 7f4328a00640 1 --2- 172.21.15.123:0/3318491593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f432415ebd0 0x7f432406dbd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.771+0000 7f4328a00640 1 --2- 172.21.15.123:0/3318491593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f432415e7f0 0x7f4324160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.771+0000 7f4328a00640 1 --2- 172.21.15.123:0/3318491593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43240d04b0 0x7f432415da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.771+0000 7f4328a00640 1 -- 172.21.15.123:0/3318491593 >> 172.21.15.123:0/3318491593 conn(0x7f43240b9530 msgr2=0x7f43240b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:05.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.771+0000 7f4328a00640 1 -- 172.21.15.123:0/3318491593 shutdown_connections 2024-04-24T05:47:05.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.772+0000 7f4328a00640 1 -- 172.21.15.123:0/3318491593 wait complete. 2024-04-24T05:47:05.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:47:05.791 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:47:05.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.989+0000 7f64aca00640 1 Processor -- start 2024-04-24T05:47:05.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.989+0000 7f64aca00640 1 -- start start 2024-04-24T05:47:05.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.990+0000 7f64aca00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64a815c2f0 0x7f64a8141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:05.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.990+0000 7f64aca00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f64a80cc5d0 0x7f64a80acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:05.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.991+0000 7f64aca00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64a806ba00 con 0x7f64a80ad510 2024-04-24T05:47:05.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.991+0000 7f64aca00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f64a806bd30 con 0x7f64a815c2f0 2024-04-24T05:47:05.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.991+0000 7f64aca00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f64a80ad940 con 0x7f64a80cc5d0 2024-04-24T05:47:05.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.991+0000 7f64a7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64a815c2f0 0x7f64a8141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:05.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.991+0000 7f64a7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64a815c2f0 0x7f64a8141ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42658/0 (socket says 172.21.15.123:42658) 2024-04-24T05:47:05.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.991+0000 7f64a7400640 1 -- 172.21.15.123:0/3257142204 learned_addr learned my addr 172.21.15.123:0/3257142204 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:05.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.991+0000 7f64a7400640 1 -- 172.21.15.123:0/3257142204 >> v2:172.21.15.155:3300/0 conn(0x7f64a80cc5d0 msgr2=0x7f64a80acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:05.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.991+0000 7f64a7400640 1 --2- 172.21.15.123:0/3257142204 >> v2:172.21.15.155:3300/0 conn(0x7f64a80cc5d0 0x7f64a80acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.991+0000 7f64a7400640 1 -- 172.21.15.123:0/3257142204 >> v1:172.21.15.155:6789/0 conn(0x7f64a80ad510 legacy=0x7f64a815ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:05.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.991+0000 7f64a7400640 1 -- 172.21.15.123:0/3257142204 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a8142530 con 0x7f64a815c2f0 2024-04-24T05:47:05.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.992+0000 7f64a7400640 1 --2- 172.21.15.123:0/3257142204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a815c2f0 0x7f64a8141ce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f64a80afb30 tx=0x7f649802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d311d9472942a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:05.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.993+0000 7f64a6a00640 1 -- 172.21.15.123:0/3257142204 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498037070 con 0x7f64a815c2f0 2024-04-24T05:47:05.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.993+0000 7f64a6a00640 1 -- 172.21.15.123:0/3257142204 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6498032440 con 0x7f64a815c2f0 2024-04-24T05:47:05.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.993+0000 7f64a6a00640 1 -- 172.21.15.123:0/3257142204 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498032810 con 0x7f64a815c2f0 2024-04-24T05:47:05.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.993+0000 7f64aca00640 1 -- 172.21.15.123:0/3257142204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a815c2f0 msgr2=0x7f64a8141ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:05.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.993+0000 7f64aca00640 1 --2- 172.21.15.123:0/3257142204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a815c2f0 0x7f64a8141ce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f64a80afb30 tx=0x7f649802a180 comp rx=0 tx=0).stop 2024-04-24T05:47:05.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.993+0000 7f64aca00640 1 -- 172.21.15.123:0/3257142204 shutdown_connections 2024-04-24T05:47:05.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.993+0000 7f64aca00640 1 --2- 172.21.15.123:0/3257142204 >> v2:172.21.15.155:3300/0 conn(0x7f64a80cc5d0 0x7f64a80acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.993+0000 7f64aca00640 1 --2- 172.21.15.123:0/3257142204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a815c2f0 0x7f64a8141ce0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:05.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.993+0000 7f64aca00640 1 -- 172.21.15.123:0/3257142204 >> 172.21.15.123:0/3257142204 conn(0x7f64a80b1cf0 msgr2=0x7f64a80b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:05.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.994+0000 7f64aca00640 1 -- 172.21.15.123:0/3257142204 shutdown_connections 2024-04-24T05:47:05.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.994+0000 7f64aca00640 1 -- 172.21.15.123:0/3257142204 wait complete. 2024-04-24T05:47:05.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.996+0000 7f64aca00640 1 Processor -- start 2024-04-24T05:47:05.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.996+0000 7f64aca00640 1 -- start start 2024-04-24T05:47:06.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.997+0000 7f64aca00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64a80ad510 0x7f64a815ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.997+0000 7f64aca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a80cc5d0 0x7f64a80733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.997+0000 7f64aca00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64a815c2f0 0x7f64a8146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.997+0000 7f64aca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64a8142220 con 0x7f64a80cc5d0 2024-04-24T05:47:06.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.997+0000 7f64aca00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f64a8142af0 con 0x7f64a815c2f0 2024-04-24T05:47:06.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.997+0000 7f64aca00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f64a806d1d0 con 0x7f64a80ad510 2024-04-24T05:47:06.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.997+0000 7f649fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a80cc5d0 0x7f64a80733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.997+0000 7f649fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a80cc5d0 0x7f64a80733e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42668/0 (socket says 172.21.15.123:42668) 2024-04-24T05:47:06.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.997+0000 7f649fe00640 1 -- 172.21.15.123:0/2527163278 learned_addr learned my addr 172.21.15.123:0/2527163278 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:06.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.998+0000 7f64a7400640 1 --2- 172.21.15.123:0/2527163278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64a80ad510 0x7f64a815ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.998+0000 7f649fe00640 1 -- 172.21.15.123:0/2527163278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64a80ad510 msgr2=0x7f64a815ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:06.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.998+0000 7f649fe00640 1 --2- 172.21.15.123:0/2527163278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64a80ad510 0x7f64a815ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.998+0000 7f649fe00640 1 -- 172.21.15.123:0/2527163278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64a815c2f0 msgr2=0x7f64a8146310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:06.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.998+0000 7f649fe00640 1 --2- 172.21.15.123:0/2527163278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64a815c2f0 0x7f64a8146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.998+0000 7f649fe00640 1 -- 172.21.15.123:0/2527163278 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a80cdf40 con 0x7f64a80cc5d0 2024-04-24T05:47:06.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.998+0000 7f649fe00640 1 --2- 172.21.15.123:0/2527163278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a80cc5d0 0x7f64a80733e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f649000b4a0 tx=0x7f649000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:06.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.998+0000 7f649f400640 1 -- 172.21.15.123:0/2527163278 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490014070 con 0x7f64a80cc5d0 2024-04-24T05:47:06.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.999+0000 7f649f400640 1 -- 172.21.15.123:0/2527163278 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6490010610 con 0x7f64a80cc5d0 2024-04-24T05:47:06.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.999+0000 7f649f400640 1 -- 172.21.15.123:0/2527163278 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490010a00 con 0x7f64a80cc5d0 2024-04-24T05:47:06.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.999+0000 7f64aca00640 1 -- 172.21.15.123:0/2527163278 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a80ce220 con 0x7f64a80cc5d0 2024-04-24T05:47:06.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:05.999+0000 7f64aca00640 1 -- 172.21.15.123:0/2527163278 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a80ce7e0 con 0x7f64a80cc5d0 2024-04-24T05:47:06.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.000+0000 7f64aca00640 1 -- 172.21.15.123:0/2527163278 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a8068a20 con 0x7f64a80cc5d0 2024-04-24T05:47:06.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.001+0000 7f649f400640 1 -- 172.21.15.123:0/2527163278 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6490017070 con 0x7f64a80cc5d0 2024-04-24T05:47:06.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.002+0000 7f649f400640 1 --2- 172.21.15.123:0/2527163278 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f648406c8f0 0x7f648406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.002+0000 7f649f400640 1 -- 172.21.15.123:0/2527163278 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f649008b9a0 con 0x7f64a80cc5d0 2024-04-24T05:47:06.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.003+0000 7f64a7400640 1 --2- 172.21.15.123:0/2527163278 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f648406c8f0 0x7f648406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.003+0000 7f64a7400640 1 --2- 172.21.15.123:0/2527163278 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f648406c8f0 0x7f648406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6498034000 tx=0x7f649802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:06.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.007+0000 7f649f400640 1 -- 172.21.15.123:0/2527163278 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64900599c0 con 0x7f64a80cc5d0 2024-04-24T05:47:06.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.208+0000 7f64aca00640 1 -- 172.21.15.123:0/2527163278 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f64a806ef90 con 0x7f64a80cc5d0 2024-04-24T05:47:06.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.209+0000 7f649f400640 1 -- 172.21.15.123:0/2527163278 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f64900592c0 con 0x7f64a80cc5d0 2024-04-24T05:47:06.210 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:47:06.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.211+0000 7f64aca00640 1 -- 172.21.15.123:0/2527163278 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f648406c8f0 msgr2=0x7f648406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:06.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.211+0000 7f64aca00640 1 --2- 172.21.15.123:0/2527163278 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f648406c8f0 0x7f648406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6498034000 tx=0x7f649802e000 comp rx=0 tx=0).stop 2024-04-24T05:47:06.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.211+0000 7f64aca00640 1 -- 172.21.15.123:0/2527163278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a80cc5d0 msgr2=0x7f64a80733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:06.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.211+0000 7f64aca00640 1 --2- 172.21.15.123:0/2527163278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a80cc5d0 0x7f64a80733e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f649000b4a0 tx=0x7f649000b960 comp rx=0 tx=0).stop 2024-04-24T05:47:06.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.212+0000 7f64aca00640 1 -- 172.21.15.123:0/2527163278 shutdown_connections 2024-04-24T05:47:06.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.212+0000 7f64aca00640 1 --2- 172.21.15.123:0/2527163278 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f648406c8f0 0x7f648406edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.212+0000 7f64aca00640 1 --2- 172.21.15.123:0/2527163278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64a815c2f0 0x7f64a8146310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.212+0000 7f64aca00640 1 --2- 172.21.15.123:0/2527163278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64a80cc5d0 0x7f64a80733e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.212+0000 7f64aca00640 1 --2- 172.21.15.123:0/2527163278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64a80ad510 0x7f64a815ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.212+0000 7f64aca00640 1 -- 172.21.15.123:0/2527163278 >> 172.21.15.123:0/2527163278 conn(0x7f64a80b1cf0 msgr2=0x7f64a80d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:06.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.212+0000 7f64aca00640 1 -- 172.21.15.123:0/2527163278 shutdown_connections 2024-04-24T05:47:06.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.212+0000 7f64aca00640 1 -- 172.21.15.123:0/2527163278 wait complete. 2024-04-24T05:47:06.232 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:47:06.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.432+0000 7f9cba400640 1 Processor -- start 2024-04-24T05:47:06.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.432+0000 7f9cba400640 1 -- start start 2024-04-24T05:47:06.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.434+0000 7f9cba400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9cb4071c50 0x7f9cb40ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.434+0000 7f9cba400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9cb40cd0f0 0x7f9cb40d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.434+0000 7f9cba400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cb40cf170 con 0x7f9cb406c510 2024-04-24T05:47:06.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.434+0000 7f9cba400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb40cd520 con 0x7f9cb4071c50 2024-04-24T05:47:06.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.434+0000 7f9cba400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb40cf450 con 0x7f9cb40cd0f0 2024-04-24T05:47:06.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.434+0000 7f9caaa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9cb4071c50 0x7f9cb40ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.434+0000 7f9caaa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9cb4071c50 0x7f9cb40ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42680/0 (socket says 172.21.15.123:42680) 2024-04-24T05:47:06.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.434+0000 7f9caaa00640 1 -- 172.21.15.123:0/3763188654 learned_addr learned my addr 172.21.15.123:0/3763188654 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:06.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.434+0000 7f9cb3e00640 1 --2- 172.21.15.123:0/3763188654 >> v2:172.21.15.155:3300/0 conn(0x7f9cb40cd0f0 0x7f9cb40d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.435+0000 7f9caaa00640 1 -- 172.21.15.123:0/3763188654 >> v2:172.21.15.155:3300/0 conn(0x7f9cb40cd0f0 msgr2=0x7f9cb40d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:06.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.435+0000 7f9caaa00640 1 --2- 172.21.15.123:0/3763188654 >> v2:172.21.15.155:3300/0 conn(0x7f9cb40cd0f0 0x7f9cb40d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.435+0000 7f9caaa00640 1 -- 172.21.15.123:0/3763188654 >> v1:172.21.15.123:6789/0 conn(0x7f9cb406c510 legacy=0x7f9cb406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:06.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.435+0000 7f9caaa00640 1 -- 172.21.15.123:0/3763188654 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb40cf730 con 0x7f9cb4071c50 2024-04-24T05:47:06.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.436+0000 7f9caaa00640 1 --2- 172.21.15.123:0/3763188654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb4071c50 0x7f9cb40ccbb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9cb406da40 tx=0x7f9c9c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3916f95c423f2819 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:06.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.436+0000 7f9cb2a00640 1 -- 172.21.15.123:0/3763188654 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c037070 con 0x7f9cb4071c50 2024-04-24T05:47:06.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.436+0000 7f9cb2a00640 1 -- 172.21.15.123:0/3763188654 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9c032440 con 0x7f9cb4071c50 2024-04-24T05:47:06.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.436+0000 7f9cb2a00640 1 -- 172.21.15.123:0/3763188654 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c032810 con 0x7f9cb4071c50 2024-04-24T05:47:06.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.437+0000 7f9cba400640 1 -- 172.21.15.123:0/3763188654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb4071c50 msgr2=0x7f9cb40ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:06.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.437+0000 7f9cba400640 1 --2- 172.21.15.123:0/3763188654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb4071c50 0x7f9cb40ccbb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9cb406da40 tx=0x7f9c9c02a270 comp rx=0 tx=0).stop 2024-04-24T05:47:06.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.437+0000 7f9cba400640 1 -- 172.21.15.123:0/3763188654 shutdown_connections 2024-04-24T05:47:06.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.437+0000 7f9cba400640 1 --2- 172.21.15.123:0/3763188654 >> v2:172.21.15.155:3300/0 conn(0x7f9cb40cd0f0 0x7f9cb40d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.437+0000 7f9cba400640 1 --2- 172.21.15.123:0/3763188654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb4071c50 0x7f9cb40ccbb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.437+0000 7f9cba400640 1 -- 172.21.15.123:0/3763188654 >> 172.21.15.123:0/3763188654 conn(0x7f9cb40b51f0 msgr2=0x7f9cb40b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:06.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.437+0000 7f9cba400640 1 -- 172.21.15.123:0/3763188654 shutdown_connections 2024-04-24T05:47:06.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.437+0000 7f9cba400640 1 -- 172.21.15.123:0/3763188654 wait complete. 2024-04-24T05:47:06.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.439+0000 7f9cba400640 1 Processor -- start 2024-04-24T05:47:06.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.439+0000 7f9cba400640 1 -- start start 2024-04-24T05:47:06.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cba400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb406c510 0x7f9cb4159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cba400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cb4071c50 0x7f9cb40c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cba400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb40cd0f0 0x7f9cb406e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cba400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb406d000 con 0x7f9cb406c510 2024-04-24T05:47:06.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cba400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb40cfcf0 con 0x7f9cb40cd0f0 2024-04-24T05:47:06.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cba400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb406d180 con 0x7f9cb4071c50 2024-04-24T05:47:06.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cb3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb406c510 0x7f9cb4159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cb3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb406c510 0x7f9cb4159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42692/0 (socket says 172.21.15.123:42692) 2024-04-24T05:47:06.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cb3400640 1 -- 172.21.15.123:0/1783077578 learned_addr learned my addr 172.21.15.123:0/1783077578 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:06.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cb3400640 1 -- 172.21.15.123:0/1783077578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cb4071c50 msgr2=0x7f9cb40c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:06.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cb3400640 1 --2- 172.21.15.123:0/1783077578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cb4071c50 0x7f9cb40c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cb3e00640 1 --2- 172.21.15.123:0/1783077578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb40cd0f0 0x7f9cb406e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cb3400640 1 -- 172.21.15.123:0/1783077578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb40cd0f0 msgr2=0x7f9cb406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:06.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cb3400640 1 --2- 172.21.15.123:0/1783077578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb40cd0f0 0x7f9cb406e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.440+0000 7f9cb3400640 1 -- 172.21.15.123:0/1783077578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb4159a80 con 0x7f9cb406c510 2024-04-24T05:47:06.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.441+0000 7f9cb3400640 1 --2- 172.21.15.123:0/1783077578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb406c510 0x7f9cb4159540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400c3c0 tx=0x7f9ca400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:06.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.441+0000 7f9cabe00640 1 -- 172.21.15.123:0/1783077578 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca40052d0 con 0x7f9cb406c510 2024-04-24T05:47:06.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.441+0000 7f9cabe00640 1 -- 172.21.15.123:0/1783077578 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca4005490 con 0x7f9cb406c510 2024-04-24T05:47:06.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.441+0000 7f9cabe00640 1 -- 172.21.15.123:0/1783077578 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4007260 con 0x7f9cb406c510 2024-04-24T05:47:06.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.441+0000 7f9cba400640 1 -- 172.21.15.123:0/1783077578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb40c44b0 con 0x7f9cb406c510 2024-04-24T05:47:06.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.441+0000 7f9cba400640 1 -- 172.21.15.123:0/1783077578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb40cad50 con 0x7f9cb406c510 2024-04-24T05:47:06.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.442+0000 7f9cba400640 1 -- 172.21.15.123:0/1783077578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb4068a20 con 0x7f9cb406c510 2024-04-24T05:47:06.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.443+0000 7f9cabe00640 1 -- 172.21.15.123:0/1783077578 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ca4007400 con 0x7f9cb406c510 2024-04-24T05:47:06.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.444+0000 7f9cabe00640 1 --2- 172.21.15.123:0/1783077578 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f9c9006c8f0 0x7f9c9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.444+0000 7f9cabe00640 1 -- 172.21.15.123:0/1783077578 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f9ca408b500 con 0x7f9cb406c510 2024-04-24T05:47:06.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.445+0000 7f9caaa00640 1 --2- 172.21.15.123:0/1783077578 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f9c9006c8f0 0x7f9c9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.445+0000 7f9caaa00640 1 --2- 172.21.15.123:0/1783077578 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f9c9006c8f0 0x7f9c9006edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c034000 tx=0x7f9c9c02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:06.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.449+0000 7f9cabe00640 1 -- 172.21.15.123:0/1783077578 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ca4059480 con 0x7f9cb406c510 2024-04-24T05:47:06.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.648+0000 7f9cba400640 1 -- 172.21.15.123:0/1783077578 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9cb40aff90 con 0x7f9cb406c510 2024-04-24T05:47:06.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.648+0000 7f9cabe00640 1 -- 172.21.15.123:0/1783077578 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f9ca4058d80 con 0x7f9cb406c510 2024-04-24T05:47:06.650 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:06.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.651+0000 7f9cba400640 1 -- 172.21.15.123:0/1783077578 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f9c9006c8f0 msgr2=0x7f9c9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:06.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.651+0000 7f9cba400640 1 --2- 172.21.15.123:0/1783077578 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f9c9006c8f0 0x7f9c9006edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c034000 tx=0x7f9c9c02c000 comp rx=0 tx=0).stop 2024-04-24T05:47:06.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.651+0000 7f9cba400640 1 -- 172.21.15.123:0/1783077578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb406c510 msgr2=0x7f9cb4159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:06.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.651+0000 7f9cba400640 1 --2- 172.21.15.123:0/1783077578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb406c510 0x7f9cb4159540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400c3c0 tx=0x7f9ca400c880 comp rx=0 tx=0).stop 2024-04-24T05:47:06.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.651+0000 7f9cba400640 1 -- 172.21.15.123:0/1783077578 shutdown_connections 2024-04-24T05:47:06.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.651+0000 7f9cba400640 1 --2- 172.21.15.123:0/1783077578 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f9c9006c8f0 0x7f9c9006edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.651+0000 7f9cba400640 1 --2- 172.21.15.123:0/1783077578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb40cd0f0 0x7f9cb406e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.651+0000 7f9cba400640 1 --2- 172.21.15.123:0/1783077578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cb4071c50 0x7f9cb40c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.651+0000 7f9cba400640 1 --2- 172.21.15.123:0/1783077578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cb406c510 0x7f9cb4159540 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.651+0000 7f9cba400640 1 -- 172.21.15.123:0/1783077578 >> 172.21.15.123:0/1783077578 conn(0x7f9cb40b51f0 msgr2=0x7f9cb40b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:06.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.652+0000 7f9cba400640 1 -- 172.21.15.123:0/1783077578 shutdown_connections 2024-04-24T05:47:06.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.652+0000 7f9cba400640 1 -- 172.21.15.123:0/1783077578 wait complete. 2024-04-24T05:47:06.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:47:06.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.870+0000 7effd1e00640 1 Processor -- start 2024-04-24T05:47:06.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.870+0000 7effd1e00640 1 -- start start 2024-04-24T05:47:06.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.871+0000 7effd1e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effcc071c50 0x7effcc0ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.871+0000 7effd1e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7effcc0cd0f0 0x7effcc0d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.871+0000 7effd1e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effcc0cf170 con 0x7effcc06c510 2024-04-24T05:47:06.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.871+0000 7effd1e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7effcc0cf450 con 0x7effcc0cd0f0 2024-04-24T05:47:06.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.871+0000 7effd1e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7effcc0cf730 con 0x7effcc071c50 2024-04-24T05:47:06.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.872+0000 7effcaa00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7effcc06c510 0x7effcc06c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49398/0 (socket says 172.21.15.123:49398) 2024-04-24T05:47:06.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.872+0000 7effcaa00640 1 -- 172.21.15.123:0/1092479491 learned_addr learned my addr 172.21.15.123:0/1092479491 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:06.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.872+0000 7effcb400640 1 --2- 172.21.15.123:0/1092479491 >> v2:172.21.15.155:3300/0 conn(0x7effcc0cd0f0 0x7effcc0d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.872+0000 7effca000640 1 --2- 172.21.15.123:0/1092479491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effcc071c50 0x7effcc0ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 -- 172.21.15.123:0/1092479491 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2306387347 0 0) 0x7effcc0cf170 con 0x7effcc06c510 2024-04-24T05:47:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 -- 172.21.15.123:0/1092479491 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effb0003520 con 0x7effcc06c510 2024-04-24T05:47:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 -- 172.21.15.123:0/1092479491 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1424257925 0 0) 0x7effb0003520 con 0x7effcc06c510 2024-04-24T05:47:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 -- 172.21.15.123:0/1092479491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effcc071c50 msgr2=0x7effcc0ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 --2- 172.21.15.123:0/1092479491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effcc071c50 0x7effcc0ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 -- 172.21.15.123:0/1092479491 >> v2:172.21.15.155:3300/0 conn(0x7effcc0cd0f0 msgr2=0x7effcc0d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 --2- 172.21.15.123:0/1092479491 >> v2:172.21.15.155:3300/0 conn(0x7effcc0cd0f0 0x7effcc0d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 -- 172.21.15.123:0/1092479491 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effcc0cfa10 con 0x7effcc06c510 2024-04-24T05:47:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 -- 172.21.15.123:0/1092479491 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7effbc001670 con 0x7effcc06c510 2024-04-24T05:47:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 -- 172.21.15.123:0/1092479491 >> v1:172.21.15.123:6789/0 conn(0x7effcc06c510 legacy=0x7effcc06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 --2- 172.21.15.123:0/1092479491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effcc071c50 0x7effb00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 --2- 172.21.15.123:0/1092479491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effb0003f30 0x7effb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 --2- 172.21.15.123:0/1092479491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0004d40 0x7effb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 -- 172.21.15.123:0/1092479491 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7effcc0cf450 con 0x7effb0004d40 2024-04-24T05:47:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 -- 172.21.15.123:0/1092479491 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7effcc0cf730 con 0x7effb0003f30 2024-04-24T05:47:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.873+0000 7effc9600640 1 -- 172.21.15.123:0/1092479491 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7effb000fdd0 con 0x7effcc071c50 2024-04-24T05:47:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.874+0000 7effcb400640 1 --2- 172.21.15.123:0/1092479491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0004d40 0x7effb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.874+0000 7effcaa00640 1 --2- 172.21.15.123:0/1092479491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effcc071c50 0x7effb00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.874+0000 7effca000640 1 --2- 172.21.15.123:0/1092479491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effb0003f30 0x7effb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.874+0000 7effcb400640 1 -- 172.21.15.123:0/1092479491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effcc071c50 msgr2=0x7effb00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.874+0000 7effcb400640 1 --2- 172.21.15.123:0/1092479491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effcc071c50 0x7effb00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.874+0000 7effcb400640 1 -- 172.21.15.123:0/1092479491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effb0003f30 msgr2=0x7effb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:06.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.874+0000 7effcb400640 1 --2- 172.21.15.123:0/1092479491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effb0003f30 0x7effb0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.874+0000 7effcb400640 1 -- 172.21.15.123:0/1092479491 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb0003520 con 0x7effb0004d40 2024-04-24T05:47:06.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.875+0000 7effcb400640 1 --2- 172.21.15.123:0/1092479491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0004d40 0x7effb00051c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7effc4009fc0 tx=0x7effb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca94610a7cabc08 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:06.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.875+0000 7effc9600640 1 -- 172.21.15.123:0/1092479491 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4009c10 con 0x7effb0004d40 2024-04-24T05:47:06.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.875+0000 7effc9600640 1 -- 172.21.15.123:0/1092479491 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc4036070 con 0x7effb0004d40 2024-04-24T05:47:06.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.875+0000 7effc9600640 1 -- 172.21.15.123:0/1092479491 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc402d070 con 0x7effb0004d40 2024-04-24T05:47:06.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.876+0000 7effd1e00640 1 -- 172.21.15.123:0/1092479491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0004d40 msgr2=0x7effb00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:06.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.876+0000 7effd1e00640 1 --2- 172.21.15.123:0/1092479491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0004d40 0x7effb00051c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7effc4009fc0 tx=0x7effb0005830 comp rx=0 tx=0).stop 2024-04-24T05:47:06.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.876+0000 7effcb400640 1 -- 172.21.15.123:0/1092479491 reap_dead start 2024-04-24T05:47:06.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.876+0000 7effd1e00640 1 -- 172.21.15.123:0/1092479491 shutdown_connections 2024-04-24T05:47:06.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.876+0000 7effd1e00640 1 -- 172.21.15.123:0/1092479491 >> 172.21.15.123:0/1092479491 conn(0x7effcc0b51f0 msgr2=0x7effcc0b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:06.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.876+0000 7effd1e00640 1 -- 172.21.15.123:0/1092479491 shutdown_connections 2024-04-24T05:47:06.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.876+0000 7effd1e00640 1 -- 172.21.15.123:0/1092479491 wait complete. 2024-04-24T05:47:06.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.878+0000 7effd1e00640 1 Processor -- start 2024-04-24T05:47:06.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.878+0000 7effd1e00640 1 -- start start 2024-04-24T05:47:06.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.879+0000 7effd1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0004d40 0x7effcc0c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.879+0000 7effd1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effcc156360 0x7effcc1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.879+0000 7effd1e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effcc0afcd0 0x7effcc0b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.879+0000 7effd1e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7effcc06d180 con 0x7effb0004d40 2024-04-24T05:47:06.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.879+0000 7effd1e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7effcc06d000 con 0x7effcc156360 2024-04-24T05:47:06.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.879+0000 7effd1e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7effcc0ced00 con 0x7effcc0afcd0 2024-04-24T05:47:06.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.879+0000 7effcaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0004d40 0x7effcc0c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.879+0000 7effcaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0004d40 0x7effcc0c66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42708/0 (socket says 172.21.15.123:42708) 2024-04-24T05:47:06.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.879+0000 7effcaa00640 1 -- 172.21.15.123:0/547369900 learned_addr learned my addr 172.21.15.123:0/547369900 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:06.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.880+0000 7effcaa00640 1 -- 172.21.15.123:0/547369900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effcc0afcd0 msgr2=0x7effcc0b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:06.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.880+0000 7effcaa00640 1 --2- 172.21.15.123:0/547369900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effcc0afcd0 0x7effcc0b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.880+0000 7effcaa00640 1 -- 172.21.15.123:0/547369900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effcc156360 msgr2=0x7effcc1567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:06.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.880+0000 7effcaa00640 1 --2- 172.21.15.123:0/547369900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effcc156360 0x7effcc1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:06.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.880+0000 7effcaa00640 1 -- 172.21.15.123:0/547369900 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effcc0c3c40 con 0x7effb0004d40 2024-04-24T05:47:06.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.880+0000 7effcaa00640 1 --2- 172.21.15.123:0/547369900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0004d40 0x7effcc0c66c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7effbc00e8f0 tx=0x7effbc00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:06.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.880+0000 7effc3400640 1 -- 172.21.15.123:0/547369900 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc00c7f0 con 0x7effb0004d40 2024-04-24T05:47:06.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.880+0000 7effc3400640 1 -- 172.21.15.123:0/547369900 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effbc018070 con 0x7effb0004d40 2024-04-24T05:47:06.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.880+0000 7effc3400640 1 -- 172.21.15.123:0/547369900 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc00cbd0 con 0x7effb0004d40 2024-04-24T05:47:06.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.880+0000 7effd1e00640 1 -- 172.21.15.123:0/547369900 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effcc0c3f20 con 0x7effb0004d40 2024-04-24T05:47:06.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.880+0000 7effd1e00640 1 -- 172.21.15.123:0/547369900 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effcc0c44e0 con 0x7effb0004d40 2024-04-24T05:47:06.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.882+0000 7effd1e00640 1 -- 172.21.15.123:0/547369900 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effcc068a20 con 0x7effb0004d40 2024-04-24T05:47:06.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.883+0000 7effc3400640 1 -- 172.21.15.123:0/547369900 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effbc01d460 con 0x7effb0004d40 2024-04-24T05:47:06.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.884+0000 7effc3400640 1 --2- 172.21.15.123:0/547369900 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7effac06c8f0 0x7effac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:06.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.884+0000 7effc3400640 1 -- 172.21.15.123:0/547369900 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7effbc08bfa0 con 0x7effb0004d40 2024-04-24T05:47:06.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.884+0000 7effca000640 1 --2- 172.21.15.123:0/547369900 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7effac06c8f0 0x7effac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:06.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.885+0000 7effca000640 1 --2- 172.21.15.123:0/547369900 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7effac06c8f0 0x7effac06edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7effcc1f3740 tx=0x7effb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:06.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:06.888+0000 7effc3400640 1 -- 172.21.15.123:0/547369900 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effbc059f20 con 0x7effb0004d40 2024-04-24T05:47:07.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.088+0000 7effd1e00640 1 -- 172.21.15.123:0/547369900 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7effcc0c3990 con 0x7effb0004d40 2024-04-24T05:47:07.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.089+0000 7effc3400640 1 -- 172.21.15.123:0/547369900 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7effbc059820 con 0x7effb0004d40 2024-04-24T05:47:07.091 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:47:07.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.091+0000 7effd1e00640 1 -- 172.21.15.123:0/547369900 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7effac06c8f0 msgr2=0x7effac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:07.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.091+0000 7effd1e00640 1 --2- 172.21.15.123:0/547369900 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7effac06c8f0 0x7effac06edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7effcc1f3740 tx=0x7effb8008040 comp rx=0 tx=0).stop 2024-04-24T05:47:07.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.091+0000 7effd1e00640 1 -- 172.21.15.123:0/547369900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0004d40 msgr2=0x7effcc0c66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:07.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.091+0000 7effd1e00640 1 --2- 172.21.15.123:0/547369900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0004d40 0x7effcc0c66c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7effbc00e8f0 tx=0x7effbc00edb0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.091+0000 7effd1e00640 1 -- 172.21.15.123:0/547369900 shutdown_connections 2024-04-24T05:47:07.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.091+0000 7effd1e00640 1 --2- 172.21.15.123:0/547369900 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7effac06c8f0 0x7effac06edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.091+0000 7effd1e00640 1 --2- 172.21.15.123:0/547369900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effcc0afcd0 0x7effcc0b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.091+0000 7effd1e00640 1 --2- 172.21.15.123:0/547369900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effcc156360 0x7effcc1567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.091+0000 7effd1e00640 1 --2- 172.21.15.123:0/547369900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0004d40 0x7effcc0c66c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.091+0000 7effd1e00640 1 -- 172.21.15.123:0/547369900 >> 172.21.15.123:0/547369900 conn(0x7effcc0b51f0 msgr2=0x7effcc15a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:07.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.091+0000 7effd1e00640 1 -- 172.21.15.123:0/547369900 shutdown_connections 2024-04-24T05:47:07.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.092+0000 7effd1e00640 1 -- 172.21.15.123:0/547369900 wait complete. 2024-04-24T05:47:07.111 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:47:07.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.310+0000 7f7e82800640 1 Processor -- start 2024-04-24T05:47:07.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.311+0000 7f7e82800640 1 -- start start 2024-04-24T05:47:07.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.312+0000 7f7e82800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7e7c15e5f0 0x7f7e7c15f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:07.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.312+0000 7f7e82800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7e7c162250 0x7f7e7c0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:07.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.312+0000 7f7e82800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e7c162a60 con 0x7f7e7c0d1420 2024-04-24T05:47:07.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.312+0000 7f7e82800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7e7c0ceb50 con 0x7f7e7c15e5f0 2024-04-24T05:47:07.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.312+0000 7f7e82800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7e7c0cee30 con 0x7f7e7c162250 2024-04-24T05:47:07.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.312+0000 7f7e81400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7e7c15e5f0 0x7f7e7c15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:07.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.312+0000 7f7e81400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7e7c15e5f0 0x7f7e7c15f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42712/0 (socket says 172.21.15.123:42712) 2024-04-24T05:47:07.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.312+0000 7f7e81400640 1 -- 172.21.15.123:0/3990765387 learned_addr learned my addr 172.21.15.123:0/3990765387 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:07.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.313+0000 7f7e80a00640 1 --2- 172.21.15.123:0/3990765387 >> v2:172.21.15.155:3300/0 conn(0x7f7e7c162250 0x7f7e7c0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:07.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.313+0000 7f7e81400640 1 -- 172.21.15.123:0/3990765387 >> v2:172.21.15.155:3300/0 conn(0x7f7e7c162250 msgr2=0x7f7e7c0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:07.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.313+0000 7f7e81400640 1 --2- 172.21.15.123:0/3990765387 >> v2:172.21.15.155:3300/0 conn(0x7f7e7c162250 0x7f7e7c0d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.313+0000 7f7e81400640 1 -- 172.21.15.123:0/3990765387 >> v1:172.21.15.123:6789/0 conn(0x7f7e7c0d1420 legacy=0x7f7e7c0b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:07.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.313+0000 7f7e81400640 1 -- 172.21.15.123:0/3990765387 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7c162d40 con 0x7f7e7c15e5f0 2024-04-24T05:47:07.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.314+0000 7f7e81400640 1 --2- 172.21.15.123:0/3990765387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c15e5f0 0x7f7e7c15f870 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c08c860 tx=0x7f7e7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a9a16d7aa45d1c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:07.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.315+0000 7f7e7be00640 1 -- 172.21.15.123:0/3990765387 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70037070 con 0x7f7e7c15e5f0 2024-04-24T05:47:07.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.315+0000 7f7e7be00640 1 -- 172.21.15.123:0/3990765387 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e70032440 con 0x7f7e7c15e5f0 2024-04-24T05:47:07.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.315+0000 7f7e7be00640 1 -- 172.21.15.123:0/3990765387 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70032810 con 0x7f7e7c15e5f0 2024-04-24T05:47:07.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.315+0000 7f7e82800640 1 -- 172.21.15.123:0/3990765387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c15e5f0 msgr2=0x7f7e7c15f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:07.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.315+0000 7f7e82800640 1 --2- 172.21.15.123:0/3990765387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c15e5f0 0x7f7e7c15f870 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c08c860 tx=0x7f7e7002a270 comp rx=0 tx=0).stop 2024-04-24T05:47:07.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.315+0000 7f7e82800640 1 -- 172.21.15.123:0/3990765387 shutdown_connections 2024-04-24T05:47:07.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.315+0000 7f7e82800640 1 --2- 172.21.15.123:0/3990765387 >> v2:172.21.15.155:3300/0 conn(0x7f7e7c162250 0x7f7e7c0d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.315+0000 7f7e82800640 1 --2- 172.21.15.123:0/3990765387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c15e5f0 0x7f7e7c15f870 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.315+0000 7f7e82800640 1 -- 172.21.15.123:0/3990765387 >> 172.21.15.123:0/3990765387 conn(0x7f7e7c0b9db0 msgr2=0x7f7e7c09d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:07.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.316+0000 7f7e82800640 1 -- 172.21.15.123:0/3990765387 shutdown_connections 2024-04-24T05:47:07.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.316+0000 7f7e82800640 1 -- 172.21.15.123:0/3990765387 wait complete. 2024-04-24T05:47:07.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.317+0000 7f7e82800640 1 Processor -- start 2024-04-24T05:47:07.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.317+0000 7f7e82800640 1 -- start start 2024-04-24T05:47:07.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.318+0000 7f7e82800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e7c0d1420 0x7f7e7c153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:07.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.318+0000 7f7e82800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e7c15e5f0 0x7f7e7c06c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:07.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.318+0000 7f7e82800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c162250 0x7f7e7c0d4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:07.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.318+0000 7f7e82800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c15e9d0 con 0x7f7e7c162250 2024-04-24T05:47:07.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.318+0000 7f7e82800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c1626d0 con 0x7f7e7c0d1420 2024-04-24T05:47:07.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.318+0000 7f7e82800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7e7c15eb50 con 0x7f7e7c15e5f0 2024-04-24T05:47:07.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.319+0000 7f7e81e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c162250 0x7f7e7c0d4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:07.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.319+0000 7f7e81e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c162250 0x7f7e7c0d4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42718/0 (socket says 172.21.15.123:42718) 2024-04-24T05:47:07.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.319+0000 7f7e81e00640 1 -- 172.21.15.123:0/3682594377 learned_addr learned my addr 172.21.15.123:0/3682594377 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:07.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.319+0000 7f7e81400640 1 --2- 172.21.15.123:0/3682594377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e7c0d1420 0x7f7e7c153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:07.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.319+0000 7f7e80a00640 1 --2- 172.21.15.123:0/3682594377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e7c15e5f0 0x7f7e7c06c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:07.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.319+0000 7f7e81e00640 1 -- 172.21.15.123:0/3682594377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e7c15e5f0 msgr2=0x7f7e7c06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:07.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.319+0000 7f7e81e00640 1 --2- 172.21.15.123:0/3682594377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e7c15e5f0 0x7f7e7c06c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.319+0000 7f7e81e00640 1 -- 172.21.15.123:0/3682594377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e7c0d1420 msgr2=0x7f7e7c153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.319+0000 7f7e81e00640 1 --2- 172.21.15.123:0/3682594377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e7c0d1420 0x7f7e7c153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.319+0000 7f7e81e00640 1 -- 172.21.15.123:0/3682594377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7c06d890 con 0x7f7e7c162250 2024-04-24T05:47:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.319+0000 7f7e81e00640 1 --2- 172.21.15.123:0/3682594377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c162250 0x7f7e7c0d4120 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400c7d0 tx=0x7f7e7400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.320+0000 7f7e79600640 1 -- 172.21.15.123:0/3682594377 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74003b70 con 0x7f7e7c162250 2024-04-24T05:47:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.320+0000 7f7e79600640 1 -- 172.21.15.123:0/3682594377 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e74003d30 con 0x7f7e7c162250 2024-04-24T05:47:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.320+0000 7f7e79600640 1 -- 172.21.15.123:0/3682594377 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7400aa90 con 0x7f7e7c162250 2024-04-24T05:47:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.320+0000 7f7e82800640 1 -- 172.21.15.123:0/3682594377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e7c153e60 con 0x7f7e7c162250 2024-04-24T05:47:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.320+0000 7f7e82800640 1 -- 172.21.15.123:0/3682594377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e7c06d240 con 0x7f7e7c162250 2024-04-24T05:47:07.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.321+0000 7f7e82800640 1 -- 172.21.15.123:0/3682594377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e7c06cce0 con 0x7f7e7c162250 2024-04-24T05:47:07.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.322+0000 7f7e79600640 1 -- 172.21.15.123:0/3682594377 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e7400ac30 con 0x7f7e7c162250 2024-04-24T05:47:07.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.323+0000 7f7e79600640 1 --2- 172.21.15.123:0/3682594377 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7e5c06c8f0 0x7f7e5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:07.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.323+0000 7f7e79600640 1 -- 172.21.15.123:0/3682594377 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f7e7408be30 con 0x7f7e7c162250 2024-04-24T05:47:07.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.323+0000 7f7e81400640 1 --2- 172.21.15.123:0/3682594377 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7e5c06c8f0 0x7f7e5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:07.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.324+0000 7f7e81400640 1 --2- 172.21.15.123:0/3682594377 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7e5c06c8f0 0x7f7e5c06edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7e70034000 tx=0x7f7e7002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:07.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.328+0000 7f7e79600640 1 -- 172.21.15.123:0/3682594377 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e7c06cce0 con 0x7f7e7c162250 2024-04-24T05:47:07.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.527+0000 7f7e82800640 1 -- 172.21.15.123:0/3682594377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7e7c0c7de0 con 0x7f7e7c162250 2024-04-24T05:47:07.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.528+0000 7f7e79600640 1 -- 172.21.15.123:0/3682594377 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f7e74059ee0 con 0x7f7e7c162250 2024-04-24T05:47:07.530 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:07.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.530+0000 7f7e82800640 1 -- 172.21.15.123:0/3682594377 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7e5c06c8f0 msgr2=0x7f7e5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:07.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.530+0000 7f7e82800640 1 --2- 172.21.15.123:0/3682594377 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7e5c06c8f0 0x7f7e5c06edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7e70034000 tx=0x7f7e7002e000 comp rx=0 tx=0).stop 2024-04-24T05:47:07.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.530+0000 7f7e82800640 1 -- 172.21.15.123:0/3682594377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c162250 msgr2=0x7f7e7c0d4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:07.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.530+0000 7f7e82800640 1 --2- 172.21.15.123:0/3682594377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c162250 0x7f7e7c0d4120 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400c7d0 tx=0x7f7e7400cc90 comp rx=0 tx=0).stop 2024-04-24T05:47:07.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.530+0000 7f7e82800640 1 -- 172.21.15.123:0/3682594377 shutdown_connections 2024-04-24T05:47:07.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.530+0000 7f7e82800640 1 --2- 172.21.15.123:0/3682594377 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f7e5c06c8f0 0x7f7e5c06edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.531+0000 7f7e82800640 1 --2- 172.21.15.123:0/3682594377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c162250 0x7f7e7c0d4120 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.531+0000 7f7e82800640 1 --2- 172.21.15.123:0/3682594377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e7c15e5f0 0x7f7e7c06c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.531+0000 7f7e82800640 1 --2- 172.21.15.123:0/3682594377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e7c0d1420 0x7f7e7c153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.531+0000 7f7e82800640 1 -- 172.21.15.123:0/3682594377 >> 172.21.15.123:0/3682594377 conn(0x7f7e7c0b9db0 msgr2=0x7f7e7c0d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:07.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.531+0000 7f7e82800640 1 -- 172.21.15.123:0/3682594377 shutdown_connections 2024-04-24T05:47:07.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.531+0000 7f7e82800640 1 -- 172.21.15.123:0/3682594377 wait complete. 2024-04-24T05:47:07.552 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T05:47:07.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.756+0000 7f3aebe00640 1 Processor -- start 2024-04-24T05:47:07.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.757+0000 7f3aebe00640 1 -- start start 2024-04-24T05:47:07.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.758+0000 7f3aebe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3aec15e4f0 0x7f3aec0cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:07.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.758+0000 7f3aebe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3aec0d3440 0x7f3aec0d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:07.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.758+0000 7f3aebe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3aec0d4390 0x7f3aec0d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:07.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.758+0000 7f3aebe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3aec0d0cd0 con 0x7f3aec0d4390 2024-04-24T05:47:07.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.758+0000 7f3aebe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3aec0d4cf0 con 0x7f3aec15e4f0 2024-04-24T05:47:07.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.758+0000 7f3aebe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3aec0cfe30 con 0x7f3aec0d3440 2024-04-24T05:47:07.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.758+0000 7f3aeb400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3aec0d4390 0x7f3aec0d47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:07.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.758+0000 7f3aeb400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3aec0d4390 0x7f3aec0d47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42730/0 (socket says 172.21.15.123:42730) 2024-04-24T05:47:07.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.758+0000 7f3aeb400640 1 -- 172.21.15.123:0/3710800357 learned_addr learned my addr 172.21.15.123:0/3710800357 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:07.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.759+0000 7f3aeb400640 1 -- 172.21.15.123:0/3710800357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3aec0d3440 msgr2=0x7f3aec0d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:07.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.759+0000 7f3aeb400640 1 --2- 172.21.15.123:0/3710800357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3aec0d3440 0x7f3aec0d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.759+0000 7f3aeb400640 1 -- 172.21.15.123:0/3710800357 >> v2:172.21.15.155:3300/0 conn(0x7f3aec15e4f0 msgr2=0x7f3aec0cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:07.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.759+0000 7f3aeb400640 1 --2- 172.21.15.123:0/3710800357 >> v2:172.21.15.155:3300/0 conn(0x7f3aec15e4f0 0x7f3aec0cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.759+0000 7f3aeb400640 1 -- 172.21.15.123:0/3710800357 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aec0d1fd0 con 0x7f3aec0d4390 2024-04-24T05:47:07.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.760+0000 7f3aeb400640 1 --2- 172.21.15.123:0/3710800357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec0d4390 0x7f3aec0d47b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3aec08dd50 tx=0x7f3ae402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ff88899ebf403e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:07.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.760+0000 7f3ae9600640 1 -- 172.21.15.123:0/3710800357 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4036070 con 0x7f3aec0d4390 2024-04-24T05:47:07.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.760+0000 7f3ae9600640 1 -- 172.21.15.123:0/3710800357 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae4031600 con 0x7f3aec0d4390 2024-04-24T05:47:07.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.760+0000 7f3ae9600640 1 -- 172.21.15.123:0/3710800357 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae40319d0 con 0x7f3aec0d4390 2024-04-24T05:47:07.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.761+0000 7f3aebe00640 1 -- 172.21.15.123:0/3710800357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec0d4390 msgr2=0x7f3aec0d47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:07.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.761+0000 7f3aebe00640 1 --2- 172.21.15.123:0/3710800357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec0d4390 0x7f3aec0d47b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3aec08dd50 tx=0x7f3ae402a6a0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.761+0000 7f3aebe00640 1 -- 172.21.15.123:0/3710800357 shutdown_connections 2024-04-24T05:47:07.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.761+0000 7f3aebe00640 1 --2- 172.21.15.123:0/3710800357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec0d4390 0x7f3aec0d47b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.761+0000 7f3aebe00640 1 --2- 172.21.15.123:0/3710800357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3aec0d3440 0x7f3aec0d3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.761+0000 7f3aebe00640 1 --2- 172.21.15.123:0/3710800357 >> v2:172.21.15.155:3300/0 conn(0x7f3aec15e4f0 0x7f3aec0cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.761+0000 7f3aebe00640 1 -- 172.21.15.123:0/3710800357 >> 172.21.15.123:0/3710800357 conn(0x7f3aec09d780 msgr2=0x7f3aec0b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:07.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.761+0000 7f3aebe00640 1 -- 172.21.15.123:0/3710800357 shutdown_connections 2024-04-24T05:47:07.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.761+0000 7f3aebe00640 1 -- 172.21.15.123:0/3710800357 wait complete. 2024-04-24T05:47:07.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.762+0000 7f3aebe00640 1 Processor -- start 2024-04-24T05:47:07.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.763+0000 7f3aebe00640 1 -- start start 2024-04-24T05:47:07.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.763+0000 7f3aebe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aec0d3440 0x7f3aec15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:07.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.763+0000 7f3aebe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec0d4390 0x7f3aec1e9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:07.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.763+0000 7f3aebe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3aec15e4f0 0x7f3aec1f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:07.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.764+0000 7f3aebe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec0d2850 con 0x7f3aec0d4390 2024-04-24T05:47:07.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.764+0000 7f3aebe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec0d2570 con 0x7f3aec0d3440 2024-04-24T05:47:07.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.764+0000 7f3aebe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3aec0d2b30 con 0x7f3aec15e4f0 2024-04-24T05:47:07.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.764+0000 7f3aea000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec0d4390 0x7f3aec1e9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:07.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.764+0000 7f3aea000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec0d4390 0x7f3aec1e9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42742/0 (socket says 172.21.15.123:42742) 2024-04-24T05:47:07.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.764+0000 7f3aea000640 1 -- 172.21.15.123:0/183853285 learned_addr learned my addr 172.21.15.123:0/183853285 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:07.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.764+0000 7f3aeaa00640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aec0d3440 0x7f3aec15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:07.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.764+0000 7f3aeb400640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3aec15e4f0 0x7f3aec1f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:07.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.764+0000 7f3aea000640 1 -- 172.21.15.123:0/183853285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3aec15e4f0 msgr2=0x7f3aec1f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:07.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.764+0000 7f3aea000640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3aec15e4f0 0x7f3aec1f4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.764+0000 7f3aea000640 1 -- 172.21.15.123:0/183853285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aec0d3440 msgr2=0x7f3aec15dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:07.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.764+0000 7f3aea000640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aec0d3440 0x7f3aec15dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.764+0000 7f3aea000640 1 -- 172.21.15.123:0/183853285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aec0c83c0 con 0x7f3aec0d4390 2024-04-24T05:47:07.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.764+0000 7f3aeaa00640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aec0d3440 0x7f3aec15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:07.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.765+0000 7f3aea000640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec0d4390 0x7f3aec1e9eb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800b4a0 tx=0x7f3ad800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:07.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.765+0000 7f3ae2a00640 1 -- 172.21.15.123:0/183853285 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8014070 con 0x7f3aec0d4390 2024-04-24T05:47:07.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.765+0000 7f3ae2a00640 1 -- 172.21.15.123:0/183853285 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad8010610 con 0x7f3aec0d4390 2024-04-24T05:47:07.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.765+0000 7f3ae2a00640 1 -- 172.21.15.123:0/183853285 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8010a00 con 0x7f3aec0d4390 2024-04-24T05:47:07.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.765+0000 7f3aebe00640 1 -- 172.21.15.123:0/183853285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aec0c86a0 con 0x7f3aec0d4390 2024-04-24T05:47:07.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.765+0000 7f3aebe00640 1 -- 172.21.15.123:0/183853285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aec0c8c60 con 0x7f3aec0d4390 2024-04-24T05:47:07.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.766+0000 7f3aebe00640 1 -- 172.21.15.123:0/183853285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aec075b70 con 0x7f3aec0d4390 2024-04-24T05:47:07.768 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.767+0000 7f3ae2a00640 1 -- 172.21.15.123:0/183853285 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ad8017070 con 0x7f3aec0d4390 2024-04-24T05:47:07.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.768+0000 7f3ae2a00640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3acc06c8f0 0x7f3acc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:07.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.769+0000 7f3ae2a00640 1 -- 172.21.15.123:0/183853285 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f3ad808ba90 con 0x7f3aec0d4390 2024-04-24T05:47:07.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.769+0000 7f3aeaa00640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3acc06c8f0 0x7f3acc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:07.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.769+0000 7f3aeaa00640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3acc06c8f0 0x7f3acc06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00a870 tx=0x7f3adc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:07.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.773+0000 7f3ae2a00640 1 -- 172.21.15.123:0/183853285 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ad8059a10 con 0x7f3aec0d4390 2024-04-24T05:47:07.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.980+0000 7f3aebe00640 1 -- 172.21.15.123:0/183853285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3aec161b20 con 0x7f3aec0d4390 2024-04-24T05:47:07.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.981+0000 7f3ae2a00640 1 -- 172.21.15.123:0/183853285 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f3ad8059310 con 0x7f3aec0d4390 2024-04-24T05:47:07.982 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-24T05:47:07.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.983+0000 7f3aebe00640 1 -- 172.21.15.123:0/183853285 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3acc06c8f0 msgr2=0x7f3acc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:07.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.983+0000 7f3aebe00640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3acc06c8f0 0x7f3acc06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00a870 tx=0x7f3adc008040 comp rx=0 tx=0).stop 2024-04-24T05:47:07.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.983+0000 7f3aebe00640 1 -- 172.21.15.123:0/183853285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec0d4390 msgr2=0x7f3aec1e9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:07.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.983+0000 7f3aebe00640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec0d4390 0x7f3aec1e9eb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800b4a0 tx=0x7f3ad800b960 comp rx=0 tx=0).stop 2024-04-24T05:47:07.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.983+0000 7f3aebe00640 1 -- 172.21.15.123:0/183853285 shutdown_connections 2024-04-24T05:47:07.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.983+0000 7f3aebe00640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3acc06c8f0 0x7f3acc06edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.983+0000 7f3aebe00640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3aec15e4f0 0x7f3aec1f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.983+0000 7f3aebe00640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec0d4390 0x7f3aec1e9eb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.983+0000 7f3aebe00640 1 --2- 172.21.15.123:0/183853285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aec0d3440 0x7f3aec15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:07.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.983+0000 7f3aebe00640 1 -- 172.21.15.123:0/183853285 >> 172.21.15.123:0/183853285 conn(0x7f3aec09d780 msgr2=0x7f3aec06ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:07.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.984+0000 7f3aebe00640 1 -- 172.21.15.123:0/183853285 shutdown_connections 2024-04-24T05:47:07.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:07.984+0000 7f3aebe00640 1 -- 172.21.15.123:0/183853285 wait complete. 2024-04-24T05:47:08.002 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2024-04-24T05:47:08.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:08.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T05:47:08.082 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T05:47:08.083 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-04-24T05:47:08.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:08.083 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T05:47:08.131 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:47:08.131 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T05:47:08.190 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:47:08.190 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T05:47:08.243 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:08.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.439+0000 7f3cf2c00640 1 Processor -- start 2024-04-24T05:47:08.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.439+0000 7f3cf2c00640 1 -- start start 2024-04-24T05:47:08.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.441+0000 7f3cf2c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cec15e7f0 0x7f3cec162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:08.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.441+0000 7f3cf2c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cec0b6cf0 con 0x7f3cec0d04b0 2024-04-24T05:47:08.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.441+0000 7f3cf2c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cec0cee10 con 0x7f3cec15ebd0 2024-04-24T05:47:08.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.441+0000 7f3cf2c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3cec163480 con 0x7f3cec15e7f0 2024-04-24T05:47:08.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.441+0000 7f3cf2200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3cec0d04b0 0x7f3cec168860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:49410/0 (socket says 172.21.15.123:49410) 2024-04-24T05:47:08.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.441+0000 7f3cf2200640 1 -- 172.21.15.123:0/1664629548 learned_addr learned my addr 172.21.15.123:0/1664629548 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.441+0000 7f3cf1800640 1 --2- 172.21.15.123:0/1664629548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cec15e7f0 0x7f3cec162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.441+0000 7f3cebe00640 1 -- 172.21.15.123:0/1664629548 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4064633034 0 0) 0x7f3cec0b6cf0 con 0x7f3cec0d04b0 2024-04-24T05:47:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cebe00640 1 -- 172.21.15.123:0/1664629548 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cd0003520 con 0x7f3cec0d04b0 2024-04-24T05:47:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cebe00640 1 -- 172.21.15.123:0/1664629548 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 924142458 0 0) 0x7f3cd0003520 con 0x7f3cec0d04b0 2024-04-24T05:47:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cebe00640 1 -- 172.21.15.123:0/1664629548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cec15e7f0 msgr2=0x7f3cec162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cebe00640 1 --2- 172.21.15.123:0/1664629548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cec15e7f0 0x7f3cec162f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cebe00640 1 -- 172.21.15.123:0/1664629548 >> v1:172.21.15.155:6789/0 conn(0x7f3cec15ebd0 legacy=0x7f3cec0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cebe00640 1 -- 172.21.15.123:0/1664629548 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec0cf0f0 con 0x7f3cec0d04b0 2024-04-24T05:47:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cebe00640 1 -- 172.21.15.123:0/1664629548 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f3cdc001670 con 0x7f3cec0d04b0 2024-04-24T05:47:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cebe00640 1 -- 172.21.15.123:0/1664629548 >> v1:172.21.15.123:6789/0 conn(0x7f3cec0d04b0 legacy=0x7f3cec168860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cebe00640 1 --2- 172.21.15.123:0/1664629548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cec15e7f0 0x7f3cd00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cebe00640 1 --2- 172.21.15.123:0/1664629548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd0003f60 0x7f3cd0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cebe00640 1 --2- 172.21.15.123:0/1664629548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cd0004d40 0x7f3cd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cebe00640 1 -- 172.21.15.123:0/1664629548 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec163480 con 0x7f3cd0004d40 2024-04-24T05:47:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cebe00640 1 -- 172.21.15.123:0/1664629548 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd000bb50 con 0x7f3cd0003f60 2024-04-24T05:47:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cebe00640 1 -- 172.21.15.123:0/1664629548 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd000bd50 con 0x7f3cec15e7f0 2024-04-24T05:47:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.442+0000 7f3cf2200640 1 --2- 172.21.15.123:0/1664629548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cd0004d40 0x7f3cd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.443+0000 7f3cf1800640 1 --2- 172.21.15.123:0/1664629548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cec15e7f0 0x7f3cd00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.443+0000 7f3cf0e00640 1 --2- 172.21.15.123:0/1664629548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd0003f60 0x7f3cd0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.443+0000 7f3cf2200640 1 -- 172.21.15.123:0/1664629548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cec15e7f0 msgr2=0x7f3cd00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.443+0000 7f3cf2200640 1 --2- 172.21.15.123:0/1664629548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cec15e7f0 0x7f3cd00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:08.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.443+0000 7f3cf2200640 1 -- 172.21.15.123:0/1664629548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd0003f60 msgr2=0x7f3cd0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:08.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.443+0000 7f3cf2200640 1 --2- 172.21.15.123:0/1664629548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd0003f60 0x7f3cd0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:08.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.443+0000 7f3cf2200640 1 -- 172.21.15.123:0/1664629548 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd0003520 con 0x7f3cd0004d40 2024-04-24T05:47:08.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.444+0000 7f3cf2200640 1 --2- 172.21.15.123:0/1664629548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cd0004d40 0x7f3cd00051c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c210 tx=0x7f3cd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec13234a089ad5ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:08.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.444+0000 7f3cebe00640 1 -- 172.21.15.123:0/1664629548 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc007d30 con 0x7f3cd0004d40 2024-04-24T05:47:08.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.444+0000 7f3cebe00640 1 -- 172.21.15.123:0/1664629548 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cdc005af0 con 0x7f3cd0004d40 2024-04-24T05:47:08.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.444+0000 7f3cebe00640 1 -- 172.21.15.123:0/1664629548 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc007070 con 0x7f3cd0004d40 2024-04-24T05:47:08.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.445+0000 7f3cf2c00640 1 -- 172.21.15.123:0/1664629548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cd0004d40 msgr2=0x7f3cd00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:08.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.445+0000 7f3cf2c00640 1 --2- 172.21.15.123:0/1664629548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cd0004d40 0x7f3cd00051c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c210 tx=0x7f3cd0005830 comp rx=0 tx=0).stop 2024-04-24T05:47:08.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.445+0000 7f3cf2200640 1 -- 172.21.15.123:0/1664629548 reap_dead start 2024-04-24T05:47:08.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.445+0000 7f3cf2c00640 1 -- 172.21.15.123:0/1664629548 shutdown_connections 2024-04-24T05:47:08.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.445+0000 7f3cf2c00640 1 -- 172.21.15.123:0/1664629548 >> 172.21.15.123:0/1664629548 conn(0x7f3cec0b9530 msgr2=0x7f3cec0b9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:08.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.445+0000 7f3cf2c00640 1 -- 172.21.15.123:0/1664629548 shutdown_connections 2024-04-24T05:47:08.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.445+0000 7f3cf2c00640 1 -- 172.21.15.123:0/1664629548 wait complete. 2024-04-24T05:47:08.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.447+0000 7f3cf2c00640 1 Processor -- start 2024-04-24T05:47:08.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.447+0000 7f3cf2c00640 1 -- start start 2024-04-24T05:47:08.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.448+0000 7f3cf2c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd0004d40 0x7f3cec0ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:08.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.448+0000 7f3cf2c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cec0cd610 0x7f3cec152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:08.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.448+0000 7f3cf2c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cec0b3190 0x7f3cec0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:08.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.448+0000 7f3cf2c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec0cf8f0 con 0x7f3cec0cd610 2024-04-24T05:47:08.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.448+0000 7f3cf2c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec0cebc0 con 0x7f3cd0004d40 2024-04-24T05:47:08.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.448+0000 7f3cf2c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3cec0cfa70 con 0x7f3cec0b3190 2024-04-24T05:47:08.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.448+0000 7f3cf0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cec0cd610 0x7f3cec152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:08.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.448+0000 7f3cf0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cec0cd610 0x7f3cec152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42766/0 (socket says 172.21.15.123:42766) 2024-04-24T05:47:08.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.448+0000 7f3cf0e00640 1 -- 172.21.15.123:0/2997585041 learned_addr learned my addr 172.21.15.123:0/2997585041 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:08.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.449+0000 7f3cf0e00640 1 -- 172.21.15.123:0/2997585041 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cec0b3190 msgr2=0x7f3cec0b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:08.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.449+0000 7f3cf0e00640 1 --2- 172.21.15.123:0/2997585041 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cec0b3190 0x7f3cec0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:08.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.449+0000 7f3cf0e00640 1 -- 172.21.15.123:0/2997585041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd0004d40 msgr2=0x7f3cec0ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:08.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.449+0000 7f3cf0e00640 1 --2- 172.21.15.123:0/2997585041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd0004d40 0x7f3cec0ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:08.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.449+0000 7f3cf0e00640 1 -- 172.21.15.123:0/2997585041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec0cb770 con 0x7f3cec0cd610 2024-04-24T05:47:08.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.449+0000 7f3cf0e00640 1 --2- 172.21.15.123:0/2997585041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cec0cd610 0x7f3cec152100 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800e580 tx=0x7f3cd800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:08.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.449+0000 7f3ce9600640 1 -- 172.21.15.123:0/2997585041 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd80052e0 con 0x7f3cec0cd610 2024-04-24T05:47:08.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.449+0000 7f3ce9600640 1 -- 172.21.15.123:0/2997585041 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd8018070 con 0x7f3cec0cd610 2024-04-24T05:47:08.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.449+0000 7f3cf2c00640 1 -- 172.21.15.123:0/2997585041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cec0cba50 con 0x7f3cec0cd610 2024-04-24T05:47:08.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.449+0000 7f3ce9600640 1 -- 172.21.15.123:0/2997585041 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8003810 con 0x7f3cec0cd610 2024-04-24T05:47:08.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.450+0000 7f3cf2c00640 1 -- 172.21.15.123:0/2997585041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cec152640 con 0x7f3cec0cd610 2024-04-24T05:47:08.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.451+0000 7f3cf2c00640 1 -- 172.21.15.123:0/2997585041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cec152c00 con 0x7f3cec0cd610 2024-04-24T05:47:08.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.452+0000 7f3ce9600640 1 -- 172.21.15.123:0/2997585041 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3cd80039b0 con 0x7f3cec0cd610 2024-04-24T05:47:08.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.452+0000 7f3ce9600640 1 --2- 172.21.15.123:0/2997585041 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3ccc06c920 0x7f3ccc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:08.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.453+0000 7f3ce9600640 1 -- 172.21.15.123:0/2997585041 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f3cd808bbb0 con 0x7f3cec0cd610 2024-04-24T05:47:08.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.453+0000 7f3cf1800640 1 --2- 172.21.15.123:0/2997585041 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3ccc06c920 0x7f3ccc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:08.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.453+0000 7f3cf1800640 1 --2- 172.21.15.123:0/2997585041 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3ccc06c920 0x7f3ccc06ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009fd0 tx=0x7f3ce4009750 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:08.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.457+0000 7f3ce9600640 1 -- 172.21.15.123:0/2997585041 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3cd8059b40 con 0x7f3cec0cd610 2024-04-24T05:47:08.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.665+0000 7f3cf2c00640 1 -- 172.21.15.123:0/2997585041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cec075b70 con 0x7f3cec0cd610 2024-04-24T05:47:08.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.667+0000 7f3ce9600640 1 -- 172.21.15.123:0/2997585041 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7f3cd8059440 con 0x7f3cec0cd610 2024-04-24T05:47:08.669 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:08.669 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:08.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.669+0000 7f3cf2c00640 1 -- 172.21.15.123:0/2997585041 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3ccc06c920 msgr2=0x7f3ccc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:08.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.669+0000 7f3cf2c00640 1 --2- 172.21.15.123:0/2997585041 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3ccc06c920 0x7f3ccc06ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009fd0 tx=0x7f3ce4009750 comp rx=0 tx=0).stop 2024-04-24T05:47:08.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.669+0000 7f3cf2c00640 1 -- 172.21.15.123:0/2997585041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cec0cd610 msgr2=0x7f3cec152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:08.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.669+0000 7f3cf2c00640 1 --2- 172.21.15.123:0/2997585041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cec0cd610 0x7f3cec152100 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800e580 tx=0x7f3cd800ea40 comp rx=0 tx=0).stop 2024-04-24T05:47:08.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.670+0000 7f3cf2c00640 1 -- 172.21.15.123:0/2997585041 shutdown_connections 2024-04-24T05:47:08.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.670+0000 7f3cf2c00640 1 --2- 172.21.15.123:0/2997585041 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3ccc06c920 0x7f3ccc06ede0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:08.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.670+0000 7f3cf2c00640 1 --2- 172.21.15.123:0/2997585041 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cec0b3190 0x7f3cec0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:08.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.670+0000 7f3cf2c00640 1 --2- 172.21.15.123:0/2997585041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cec0cd610 0x7f3cec152100 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:08.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.670+0000 7f3cf2c00640 1 --2- 172.21.15.123:0/2997585041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd0004d40 0x7f3cec0ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:08.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.670+0000 7f3cf2c00640 1 -- 172.21.15.123:0/2997585041 >> 172.21.15.123:0/2997585041 conn(0x7f3cec0b9530 msgr2=0x7f3cec0b7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:08.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.670+0000 7f3cf2c00640 1 -- 172.21.15.123:0/2997585041 shutdown_connections 2024-04-24T05:47:08.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.670+0000 7f3cf2c00640 1 -- 172.21.15.123:0/2997585041 wait complete. 2024-04-24T05:47:08.690 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:47:08.691 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:47:08.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.886+0000 7f973fe00640 1 Processor -- start 2024-04-24T05:47:08.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.886+0000 7f973fe00640 1 -- start start 2024-04-24T05:47:08.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.888+0000 7f973fe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f974015c2f0 0x7f9740141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:08.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.888+0000 7f973fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97400cc5d0 0x7f97400acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:08.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.888+0000 7f973fe00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f974006ba00 con 0x7f97400ad510 2024-04-24T05:47:08.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.888+0000 7f973fe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f974006bd30 con 0x7f97400cc5d0 2024-04-24T05:47:08.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.888+0000 7f973fe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f97400ad940 con 0x7f974015c2f0 2024-04-24T05:47:08.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.888+0000 7f9736000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97400cc5d0 0x7f97400acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:08.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.888+0000 7f9736000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97400cc5d0 0x7f97400acfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59480/0 (socket says 172.21.15.123:59480) 2024-04-24T05:47:08.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.888+0000 7f9736000640 1 -- 172.21.15.123:0/864609349 learned_addr learned my addr 172.21.15.123:0/864609349 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:08.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.888+0000 7f9736000640 1 -- 172.21.15.123:0/864609349 >> v2:172.21.15.155:3300/0 conn(0x7f974015c2f0 msgr2=0x7f9740141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:08.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.888+0000 7f9736000640 1 --2- 172.21.15.123:0/864609349 >> v2:172.21.15.155:3300/0 conn(0x7f974015c2f0 0x7f9740141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:08.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.888+0000 7f9736000640 1 -- 172.21.15.123:0/864609349 >> v1:172.21.15.155:6789/0 conn(0x7f97400ad510 legacy=0x7f974015ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:08.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.889+0000 7f9736000640 1 -- 172.21.15.123:0/864609349 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9740142530 con 0x7f97400cc5d0 2024-04-24T05:47:08.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.890+0000 7f9736000640 1 --2- 172.21.15.123:0/864609349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97400cc5d0 0x7f97400acfd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9740073fd0 tx=0x7f972802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d261728cae0932 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:08.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.890+0000 7f973e000640 1 -- 172.21.15.123:0/864609349 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728037070 con 0x7f97400cc5d0 2024-04-24T05:47:08.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.890+0000 7f973e000640 1 -- 172.21.15.123:0/864609349 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9728032440 con 0x7f97400cc5d0 2024-04-24T05:47:08.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.890+0000 7f973e000640 1 -- 172.21.15.123:0/864609349 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728032810 con 0x7f97400cc5d0 2024-04-24T05:47:08.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.890+0000 7f973fe00640 1 -- 172.21.15.123:0/864609349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97400cc5d0 msgr2=0x7f97400acfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:08.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.890+0000 7f973fe00640 1 --2- 172.21.15.123:0/864609349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97400cc5d0 0x7f97400acfd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9740073fd0 tx=0x7f972802a180 comp rx=0 tx=0).stop 2024-04-24T05:47:08.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.891+0000 7f973fe00640 1 -- 172.21.15.123:0/864609349 shutdown_connections 2024-04-24T05:47:08.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.891+0000 7f973fe00640 1 --2- 172.21.15.123:0/864609349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97400cc5d0 0x7f97400acfd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:08.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.891+0000 7f973fe00640 1 --2- 172.21.15.123:0/864609349 >> v2:172.21.15.155:3300/0 conn(0x7f974015c2f0 0x7f9740141ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:08.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.891+0000 7f973fe00640 1 -- 172.21.15.123:0/864609349 >> 172.21.15.123:0/864609349 conn(0x7f97400b1cf0 msgr2=0x7f97400b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:08.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.891+0000 7f973fe00640 1 -- 172.21.15.123:0/864609349 shutdown_connections 2024-04-24T05:47:08.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.891+0000 7f973fe00640 1 -- 172.21.15.123:0/864609349 wait complete. 2024-04-24T05:47:08.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.892+0000 7f973fe00640 1 Processor -- start 2024-04-24T05:47:08.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.892+0000 7f973fe00640 1 -- start start 2024-04-24T05:47:08.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.893+0000 7f973fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97400ad510 0x7f974015afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:08.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.893+0000 7f973fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97400cc5d0 0x7f97401f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:08.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.893+0000 7f973fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f974015c2f0 0x7f97400d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:08.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.893+0000 7f973fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9740142220 con 0x7f974015c2f0 2024-04-24T05:47:08.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.893+0000 7f973fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9740142af0 con 0x7f97400cc5d0 2024-04-24T05:47:08.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.893+0000 7f973fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f974006d1d0 con 0x7f97400ad510 2024-04-24T05:47:08.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.894+0000 7f973f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f974015c2f0 0x7f97400d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:08.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.894+0000 7f973f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f974015c2f0 0x7f97400d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59492/0 (socket says 172.21.15.123:59492) 2024-04-24T05:47:08.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.894+0000 7f973f400640 1 -- 172.21.15.123:0/933609228 learned_addr learned my addr 172.21.15.123:0/933609228 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.894+0000 7f9736000640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97400cc5d0 0x7f97401f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.894+0000 7f973ea00640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97400ad510 0x7f974015afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.894+0000 7f973f400640 1 -- 172.21.15.123:0/933609228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97400ad510 msgr2=0x7f974015afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.894+0000 7f973f400640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97400ad510 0x7f974015afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.894+0000 7f973f400640 1 -- 172.21.15.123:0/933609228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97400cc5d0 msgr2=0x7f97401f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.894+0000 7f973f400640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97400cc5d0 0x7f97401f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.894+0000 7f973f400640 1 -- 172.21.15.123:0/933609228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97400d0d30 con 0x7f974015c2f0 2024-04-24T05:47:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.894+0000 7f973ea00640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97400ad510 0x7f974015afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:08.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.895+0000 7f973f400640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f974015c2f0 0x7f97400d07f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f973800b3e0 tx=0x7f973800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:08.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.895+0000 7f9737400640 1 -- 172.21.15.123:0/933609228 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738015020 con 0x7f974015c2f0 2024-04-24T05:47:08.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.895+0000 7f9737400640 1 -- 172.21.15.123:0/933609228 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9738011830 con 0x7f974015c2f0 2024-04-24T05:47:08.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.895+0000 7f973fe00640 1 -- 172.21.15.123:0/933609228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97401583d0 con 0x7f974015c2f0 2024-04-24T05:47:08.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.895+0000 7f9737400640 1 -- 172.21.15.123:0/933609228 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738011c20 con 0x7f974015c2f0 2024-04-24T05:47:08.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.895+0000 7f973fe00640 1 -- 172.21.15.123:0/933609228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9740151d20 con 0x7f974015c2f0 2024-04-24T05:47:08.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.896+0000 7f973fe00640 1 -- 172.21.15.123:0/933609228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97401588f0 con 0x7f974015c2f0 2024-04-24T05:47:08.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.897+0000 7f9737400640 1 -- 172.21.15.123:0/933609228 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9738018020 con 0x7f974015c2f0 2024-04-24T05:47:08.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.898+0000 7f9737400640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f971c06c8f0 0x7f971c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:08.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.899+0000 7f9737400640 1 -- 172.21.15.123:0/933609228 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f973808be00 con 0x7f974015c2f0 2024-04-24T05:47:08.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.899+0000 7f973ea00640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f971c06c8f0 0x7f971c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:08.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.899+0000 7f973ea00640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f971c06c8f0 0x7f971c06edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f97400b01d0 tx=0x7f97300073e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:08.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:08.903+0000 7f9737400640 1 -- 172.21.15.123:0/933609228 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97401588f0 con 0x7f974015c2f0 2024-04-24T05:47:09.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.104+0000 7f973fe00640 1 -- 172.21.15.123:0/933609228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9740068a20 con 0x7f974015c2f0 2024-04-24T05:47:09.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.105+0000 7f9737400640 1 -- 172.21.15.123:0/933609228 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f973801f020 con 0x7f974015c2f0 2024-04-24T05:47:09.107 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:47:09.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.108+0000 7f973fe00640 1 -- 172.21.15.123:0/933609228 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f971c06c8f0 msgr2=0x7f971c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:09.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.108+0000 7f973fe00640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f971c06c8f0 0x7f971c06edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f97400b01d0 tx=0x7f97300073e0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.108+0000 7f973fe00640 1 -- 172.21.15.123:0/933609228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f974015c2f0 msgr2=0x7f97400d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:09.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.108+0000 7f973fe00640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f974015c2f0 0x7f97400d07f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f973800b3e0 tx=0x7f973800b8a0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.108+0000 7f973fe00640 1 -- 172.21.15.123:0/933609228 shutdown_connections 2024-04-24T05:47:09.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.108+0000 7f973fe00640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f971c06c8f0 0x7f971c06edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.108+0000 7f973fe00640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f974015c2f0 0x7f97400d07f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.108+0000 7f973fe00640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97400cc5d0 0x7f97401f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.108+0000 7f973fe00640 1 --2- 172.21.15.123:0/933609228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f97400ad510 0x7f974015afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.108+0000 7f973fe00640 1 -- 172.21.15.123:0/933609228 >> 172.21.15.123:0/933609228 conn(0x7f97400b1cf0 msgr2=0x7f97400d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:09.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.108+0000 7f973fe00640 1 -- 172.21.15.123:0/933609228 shutdown_connections 2024-04-24T05:47:09.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.108+0000 7f973fe00640 1 -- 172.21.15.123:0/933609228 wait complete. 2024-04-24T05:47:09.130 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:47:09.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.327+0000 7fdbfd200640 1 Processor -- start 2024-04-24T05:47:09.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.327+0000 7fdbfd200640 1 -- start start 2024-04-24T05:47:09.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.329+0000 7fdbfd200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80d0160 0x7fdbf80ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:09.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.329+0000 7fdbfd200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdbf80ce610 0x7fdbf806e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:09.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.329+0000 7fdbfd200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbf80d0740 con 0x7fdbf80ae990 2024-04-24T05:47:09.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.329+0000 7fdbfd200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf80cea70 con 0x7fdbf80ce610 2024-04-24T05:47:09.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.329+0000 7fdbfd200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf806dcb0 con 0x7fdbf80d0160 2024-04-24T05:47:09.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.329+0000 7fdbf6800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdbf80ce610 0x7fdbf806e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:09.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.329+0000 7fdbf6800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdbf80ce610 0x7fdbf806e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59508/0 (socket says 172.21.15.123:59508) 2024-04-24T05:47:09.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.329+0000 7fdbf6800640 1 -- 172.21.15.123:0/861370393 learned_addr learned my addr 172.21.15.123:0/861370393 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:09.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.329+0000 7fdbed400640 1 --2- 172.21.15.123:0/861370393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80d0160 0x7fdbf80ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:09.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.330+0000 7fdbf6800640 1 -- 172.21.15.123:0/861370393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80d0160 msgr2=0x7fdbf80ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:09.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.330+0000 7fdbf6800640 1 --2- 172.21.15.123:0/861370393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80d0160 0x7fdbf80ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.330+0000 7fdbf6800640 1 -- 172.21.15.123:0/861370393 >> v1:172.21.15.155:6789/0 conn(0x7fdbf80ae990 legacy=0x7fdbf806c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:09.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.330+0000 7fdbf6800640 1 -- 172.21.15.123:0/861370393 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf80d0a20 con 0x7fdbf80ce610 2024-04-24T05:47:09.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.330+0000 7fdbed400640 1 --2- 172.21.15.123:0/861370393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80d0160 0x7fdbf80ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:09.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.331+0000 7fdbf6800640 1 --2- 172.21.15.123:0/861370393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf80ce610 0x7fdbf806e700 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009870 tx=0x7fdbf002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60309d43ae88aaec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:09.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.331+0000 7fdbf5400640 1 -- 172.21.15.123:0/861370393 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0037070 con 0x7fdbf80ce610 2024-04-24T05:47:09.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.331+0000 7fdbf5400640 1 -- 172.21.15.123:0/861370393 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf0032440 con 0x7fdbf80ce610 2024-04-24T05:47:09.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.331+0000 7fdbf5400640 1 -- 172.21.15.123:0/861370393 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0032810 con 0x7fdbf80ce610 2024-04-24T05:47:09.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.332+0000 7fdbfd200640 1 -- 172.21.15.123:0/861370393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf80ce610 msgr2=0x7fdbf806e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:09.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.332+0000 7fdbfd200640 1 --2- 172.21.15.123:0/861370393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf80ce610 0x7fdbf806e700 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009870 tx=0x7fdbf002a210 comp rx=0 tx=0).stop 2024-04-24T05:47:09.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.332+0000 7fdbfd200640 1 -- 172.21.15.123:0/861370393 shutdown_connections 2024-04-24T05:47:09.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.332+0000 7fdbfd200640 1 --2- 172.21.15.123:0/861370393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf80ce610 0x7fdbf806e700 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.332+0000 7fdbfd200640 1 --2- 172.21.15.123:0/861370393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80d0160 0x7fdbf80ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.332+0000 7fdbfd200640 1 -- 172.21.15.123:0/861370393 >> 172.21.15.123:0/861370393 conn(0x7fdbf80b51f0 msgr2=0x7fdbf80b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:09.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.332+0000 7fdbfd200640 1 -- 172.21.15.123:0/861370393 shutdown_connections 2024-04-24T05:47:09.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.332+0000 7fdbfd200640 1 -- 172.21.15.123:0/861370393 wait complete. 2024-04-24T05:47:09.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.334+0000 7fdbfd200640 1 Processor -- start 2024-04-24T05:47:09.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.334+0000 7fdbfd200640 1 -- start start 2024-04-24T05:47:09.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbfd200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80ae990 0x7fdbf80b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:09.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbfd200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf80ce610 0x7fdbf815d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:09.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbfd200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf80d0160 0x7fdbf815da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:09.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbfd200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf80aed70 con 0x7fdbf80ce610 2024-04-24T05:47:09.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbfd200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf806e270 con 0x7fdbf80d0160 2024-04-24T05:47:09.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbfd200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf80ad040 con 0x7fdbf80ae990 2024-04-24T05:47:09.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbed400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf80ce610 0x7fdbf815d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:09.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbed400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf80ce610 0x7fdbf815d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59524/0 (socket says 172.21.15.123:59524) 2024-04-24T05:47:09.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbed400640 1 -- 172.21.15.123:0/3289041927 learned_addr learned my addr 172.21.15.123:0/3289041927 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:09.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbf5e00640 1 --2- 172.21.15.123:0/3289041927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80ae990 0x7fdbf80b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:09.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbed400640 1 -- 172.21.15.123:0/3289041927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80ae990 msgr2=0x7fdbf80b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:09.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbed400640 1 --2- 172.21.15.123:0/3289041927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80ae990 0x7fdbf80b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbed400640 1 -- 172.21.15.123:0/3289041927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf80d0160 msgr2=0x7fdbf815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:09.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbf6800640 1 --2- 172.21.15.123:0/3289041927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf80d0160 0x7fdbf815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:09.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbed400640 1 --2- 172.21.15.123:0/3289041927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf80d0160 0x7fdbf815da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.335+0000 7fdbed400640 1 -- 172.21.15.123:0/3289041927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf815df80 con 0x7fdbf80ce610 2024-04-24T05:47:09.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.336+0000 7fdbed400640 1 --2- 172.21.15.123:0/3289041927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf80ce610 0x7fdbf815d500 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000c7f0 tx=0x7fdbe000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:09.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.336+0000 7fdbeea00640 1 -- 172.21.15.123:0/3289041927 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0015070 con 0x7fdbf80ce610 2024-04-24T05:47:09.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.336+0000 7fdbeea00640 1 -- 172.21.15.123:0/3289041927 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe0007a30 con 0x7fdbf80ce610 2024-04-24T05:47:09.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.336+0000 7fdbeea00640 1 -- 172.21.15.123:0/3289041927 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0007e00 con 0x7fdbf80ce610 2024-04-24T05:47:09.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.336+0000 7fdbfd200640 1 -- 172.21.15.123:0/3289041927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf81f78a0 con 0x7fdbf80ce610 2024-04-24T05:47:09.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.336+0000 7fdbfd200640 1 -- 172.21.15.123:0/3289041927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf81f7e60 con 0x7fdbf80ce610 2024-04-24T05:47:09.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.337+0000 7fdbfd200640 1 -- 172.21.15.123:0/3289041927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf8068a20 con 0x7fdbf80ce610 2024-04-24T05:47:09.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.338+0000 7fdbeea00640 1 -- 172.21.15.123:0/3289041927 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbe0018070 con 0x7fdbf80ce610 2024-04-24T05:47:09.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.340+0000 7fdbeea00640 1 --2- 172.21.15.123:0/3289041927 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdbd406c8f0 0x7fdbd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:09.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.340+0000 7fdbeea00640 1 -- 172.21.15.123:0/3289041927 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fdbe008bf50 con 0x7fdbf80ce610 2024-04-24T05:47:09.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.340+0000 7fdbf5e00640 1 --2- 172.21.15.123:0/3289041927 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdbd406c8f0 0x7fdbd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:09.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.341+0000 7fdbf5e00640 1 --2- 172.21.15.123:0/3289041927 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdbd406c8f0 0x7fdbd406edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009f20 tx=0x7fdbe80094d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:09.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.344+0000 7fdbeea00640 1 -- 172.21.15.123:0/3289041927 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdbe0059f60 con 0x7fdbf80ce610 2024-04-24T05:47:09.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.542+0000 7fdbfd200640 1 -- 172.21.15.123:0/3289041927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdbf81f81a0 con 0x7fdbf80ce610 2024-04-24T05:47:09.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.543+0000 7fdbeea00640 1 -- 172.21.15.123:0/3289041927 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fdbe0059860 con 0x7fdbf80ce610 2024-04-24T05:47:09.545 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:09.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.546+0000 7fdbfd200640 1 -- 172.21.15.123:0/3289041927 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdbd406c8f0 msgr2=0x7fdbd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:09.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.546+0000 7fdbfd200640 1 --2- 172.21.15.123:0/3289041927 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdbd406c8f0 0x7fdbd406edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009f20 tx=0x7fdbe80094d0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.546+0000 7fdbfd200640 1 -- 172.21.15.123:0/3289041927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf80ce610 msgr2=0x7fdbf815d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:09.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.546+0000 7fdbfd200640 1 --2- 172.21.15.123:0/3289041927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf80ce610 0x7fdbf815d500 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000c7f0 tx=0x7fdbe000cbf0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.546+0000 7fdbfd200640 1 -- 172.21.15.123:0/3289041927 shutdown_connections 2024-04-24T05:47:09.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.546+0000 7fdbfd200640 1 --2- 172.21.15.123:0/3289041927 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdbd406c8f0 0x7fdbd406edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.546+0000 7fdbfd200640 1 --2- 172.21.15.123:0/3289041927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbf80d0160 0x7fdbf815da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.546+0000 7fdbfd200640 1 --2- 172.21.15.123:0/3289041927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf80ce610 0x7fdbf815d500 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.546+0000 7fdbfd200640 1 --2- 172.21.15.123:0/3289041927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbf80ae990 0x7fdbf80b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.546+0000 7fdbfd200640 1 -- 172.21.15.123:0/3289041927 >> 172.21.15.123:0/3289041927 conn(0x7fdbf80b51f0 msgr2=0x7fdbf80ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:09.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.546+0000 7fdbfd200640 1 -- 172.21.15.123:0/3289041927 shutdown_connections 2024-04-24T05:47:09.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.546+0000 7fdbfd200640 1 -- 172.21.15.123:0/3289041927 wait complete. 2024-04-24T05:47:09.566 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:47:09.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.774+0000 7fcce1400640 1 Processor -- start 2024-04-24T05:47:09.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.774+0000 7fcce1400640 1 -- start start 2024-04-24T05:47:09.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.776+0000 7fcce1400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fccdc158cc0 0x7fccdc15ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:09.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.776+0000 7fcce1400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccdc06d9b0 0x7fccdc160490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:09.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.776+0000 7fcce1400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccdc15ff90 con 0x7fccdc06e400 2024-04-24T05:47:09.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.776+0000 7fcce1400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fccdc06fab0 con 0x7fccdc158cc0 2024-04-24T05:47:09.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.776+0000 7fcce1400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fccdc15f430 con 0x7fccdc06d9b0 2024-04-24T05:47:09.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.776+0000 7fccdbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fccdc158cc0 0x7fccdc15ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:09.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.776+0000 7fccdbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fccdc158cc0 0x7fccdc15ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59532/0 (socket says 172.21.15.123:59532) 2024-04-24T05:47:09.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.776+0000 7fccdbe00640 1 -- 172.21.15.123:0/1705933557 learned_addr learned my addr 172.21.15.123:0/1705933557 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:09.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.776+0000 7fcce0a00640 1 --2- 172.21.15.123:0/1705933557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccdc06d9b0 0x7fccdc160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:09.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.776+0000 7fccdbe00640 1 -- 172.21.15.123:0/1705933557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccdc06d9b0 msgr2=0x7fccdc160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:09.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.776+0000 7fccdbe00640 1 --2- 172.21.15.123:0/1705933557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccdc06d9b0 0x7fccdc160490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.776+0000 7fccdbe00640 1 -- 172.21.15.123:0/1705933557 >> v1:172.21.15.155:6789/0 conn(0x7fccdc06e400 legacy=0x7fccdc06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:09.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.776+0000 7fccdbe00640 1 -- 172.21.15.123:0/1705933557 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccdc15f710 con 0x7fccdc158cc0 2024-04-24T05:47:09.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.777+0000 7fcce0a00640 1 --2- 172.21.15.123:0/1705933557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccdc06d9b0 0x7fccdc160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.778+0000 7fccdbe00640 1 --2- 172.21.15.123:0/1705933557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccdc158cc0 0x7fccdc15ec00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009870 tx=0x7fcccc02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f86a91b8583edd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.778+0000 7fccdb400640 1 -- 172.21.15.123:0/1705933557 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc037070 con 0x7fccdc158cc0 2024-04-24T05:47:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.778+0000 7fccdb400640 1 -- 172.21.15.123:0/1705933557 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcccc032440 con 0x7fccdc158cc0 2024-04-24T05:47:09.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.778+0000 7fccdb400640 1 -- 172.21.15.123:0/1705933557 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc032810 con 0x7fccdc158cc0 2024-04-24T05:47:09.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.779+0000 7fcce1400640 1 -- 172.21.15.123:0/1705933557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccdc158cc0 msgr2=0x7fccdc15ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:09.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.779+0000 7fcce1400640 1 --2- 172.21.15.123:0/1705933557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccdc158cc0 0x7fccdc15ec00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009870 tx=0x7fcccc02a2a0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.779+0000 7fcce1400640 1 -- 172.21.15.123:0/1705933557 shutdown_connections 2024-04-24T05:47:09.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.779+0000 7fcce1400640 1 --2- 172.21.15.123:0/1705933557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccdc06d9b0 0x7fccdc160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.779+0000 7fcce1400640 1 --2- 172.21.15.123:0/1705933557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccdc158cc0 0x7fccdc15ec00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.779+0000 7fcce1400640 1 -- 172.21.15.123:0/1705933557 >> 172.21.15.123:0/1705933557 conn(0x7fccdc0723a0 msgr2=0x7fccdc0727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:09.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.779+0000 7fcce1400640 1 -- 172.21.15.123:0/1705933557 shutdown_connections 2024-04-24T05:47:09.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.779+0000 7fcce1400640 1 -- 172.21.15.123:0/1705933557 wait complete. 2024-04-24T05:47:09.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.781+0000 7fcce1400640 1 Processor -- start 2024-04-24T05:47:09.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.781+0000 7fcce1400640 1 -- start start 2024-04-24T05:47:09.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.782+0000 7fcce1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccdc06d9b0 0x7fccdc146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:09.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fcce1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccdc06e400 0x7fccdc159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:09.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fcce1400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccdc158cc0 0x7fccdc15d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:09.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fcce1400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc1590a0 con 0x7fccdc06e400 2024-04-24T05:47:09.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fcce1400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc15f140 con 0x7fccdc06d9b0 2024-04-24T05:47:09.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fcce1400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fccdc1609d0 con 0x7fccdc158cc0 2024-04-24T05:47:09.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fccd3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccdc06e400 0x7fccdc159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:09.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fccd3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccdc06e400 0x7fccdc159220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59538/0 (socket says 172.21.15.123:59538) 2024-04-24T05:47:09.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fccd3e00640 1 -- 172.21.15.123:0/959221147 learned_addr learned my addr 172.21.15.123:0/959221147 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fccd3e00640 1 -- 172.21.15.123:0/959221147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccdc158cc0 msgr2=0x7fccdc15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fcce0a00640 1 --2- 172.21.15.123:0/959221147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccdc158cc0 0x7fccdc15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fccdbe00640 1 --2- 172.21.15.123:0/959221147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccdc06d9b0 0x7fccdc146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fccd3e00640 1 --2- 172.21.15.123:0/959221147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccdc158cc0 0x7fccdc15d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fccd3e00640 1 -- 172.21.15.123:0/959221147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccdc06d9b0 msgr2=0x7fccdc146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fccd3e00640 1 --2- 172.21.15.123:0/959221147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccdc06d9b0 0x7fccdc146350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:09.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.783+0000 7fccd3e00640 1 -- 172.21.15.123:0/959221147 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccdc159ce0 con 0x7fccdc06e400 2024-04-24T05:47:09.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.784+0000 7fccd3e00640 1 --2- 172.21.15.123:0/959221147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccdc06e400 0x7fccdc159220 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c460 tx=0x7fccc400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:09.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.784+0000 7fccd8c00640 1 -- 172.21.15.123:0/959221147 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4016020 con 0x7fccdc06e400 2024-04-24T05:47:09.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.784+0000 7fccd8c00640 1 -- 172.21.15.123:0/959221147 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc4012870 con 0x7fccdc06e400 2024-04-24T05:47:09.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.784+0000 7fcce1400640 1 -- 172.21.15.123:0/959221147 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccdc1f7ab0 con 0x7fccdc06e400 2024-04-24T05:47:09.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.784+0000 7fccd8c00640 1 -- 172.21.15.123:0/959221147 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4012c60 con 0x7fccdc06e400 2024-04-24T05:47:09.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.784+0000 7fcce1400640 1 -- 172.21.15.123:0/959221147 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccdc14cf00 con 0x7fccdc06e400 2024-04-24T05:47:09.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.785+0000 7fcce1400640 1 -- 172.21.15.123:0/959221147 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccdc069130 con 0x7fccdc06e400 2024-04-24T05:47:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.786+0000 7fccd8c00640 1 -- 172.21.15.123:0/959221147 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fccc4019020 con 0x7fccdc06e400 2024-04-24T05:47:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.787+0000 7fccd8c00640 1 --2- 172.21.15.123:0/959221147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fccb806c8f0 0x7fccb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.787+0000 7fccd8c00640 1 -- 172.21.15.123:0/959221147 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fccc408bf70 con 0x7fccdc06e400 2024-04-24T05:47:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.787+0000 7fccdbe00640 1 --2- 172.21.15.123:0/959221147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fccb806c8f0 0x7fccb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.788+0000 7fccdbe00640 1 --2- 172.21.15.123:0/959221147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fccb806c8f0 0x7fccb806edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcccc034000 tx=0x7fcccc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:09.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.791+0000 7fccd8c00640 1 -- 172.21.15.123:0/959221147 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fccc4020020 con 0x7fccdc06e400 2024-04-24T05:47:09.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.994+0000 7fcce1400640 1 -- 172.21.15.123:0/959221147 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fccdc1f7860 con 0x7fccdc06e400 2024-04-24T05:47:09.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.996+0000 7fccd8c00640 1 -- 172.21.15.123:0/959221147 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fccc40182c0 con 0x7fccdc06e400 2024-04-24T05:47:09.997 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:47:10.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.998+0000 7fcce1400640 1 -- 172.21.15.123:0/959221147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fccb806c8f0 msgr2=0x7fccb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:10.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.998+0000 7fcce1400640 1 --2- 172.21.15.123:0/959221147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fccb806c8f0 0x7fccb806edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcccc034000 tx=0x7fcccc02e000 comp rx=0 tx=0).stop 2024-04-24T05:47:10.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.998+0000 7fcce1400640 1 -- 172.21.15.123:0/959221147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccdc06e400 msgr2=0x7fccdc159220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:10.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.998+0000 7fcce1400640 1 --2- 172.21.15.123:0/959221147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccdc06e400 0x7fccdc159220 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c460 tx=0x7fccc400c920 comp rx=0 tx=0).stop 2024-04-24T05:47:10.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.998+0000 7fcce1400640 1 -- 172.21.15.123:0/959221147 shutdown_connections 2024-04-24T05:47:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.998+0000 7fcce1400640 1 --2- 172.21.15.123:0/959221147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fccb806c8f0 0x7fccb806edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.998+0000 7fcce1400640 1 --2- 172.21.15.123:0/959221147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccdc158cc0 0x7fccdc15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.998+0000 7fcce1400640 1 --2- 172.21.15.123:0/959221147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccdc06e400 0x7fccdc159220 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.999+0000 7fcce1400640 1 --2- 172.21.15.123:0/959221147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccdc06d9b0 0x7fccdc146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.999+0000 7fcce1400640 1 -- 172.21.15.123:0/959221147 >> 172.21.15.123:0/959221147 conn(0x7fccdc0723a0 msgr2=0x7fccdc0aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.999+0000 7fcce1400640 1 -- 172.21.15.123:0/959221147 shutdown_connections 2024-04-24T05:47:10.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:09.999+0000 7fcce1400640 1 -- 172.21.15.123:0/959221147 wait complete. 2024-04-24T05:47:10.020 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:47:10.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.218+0000 7f4273c00640 1 Processor -- start 2024-04-24T05:47:10.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.218+0000 7f4273c00640 1 -- start start 2024-04-24T05:47:10.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.219+0000 7f4273c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426c0ae990 0x7f426c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:10.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.219+0000 7f4273c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f426c06c410 0x7f426c0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:10.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.219+0000 7f4273c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f426c0ce610 0x7f426c15f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:10.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.219+0000 7f4273c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f426c0cea70 con 0x7f426c06c410 2024-04-24T05:47:10.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.220+0000 7f4273c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f426c06dcb0 con 0x7f426c0ce610 2024-04-24T05:47:10.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.220+0000 7f4273c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f426c06e270 con 0x7f426c0ae990 2024-04-24T05:47:10.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.220+0000 7f426be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f426c06c410 0x7f426c0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:10.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.220+0000 7f426be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f426c06c410 0x7f426c0ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59544/0 (socket says 172.21.15.123:59544) 2024-04-24T05:47:10.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.220+0000 7f426be00640 1 -- 172.21.15.123:0/1524269453 learned_addr learned my addr 172.21.15.123:0/1524269453 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:10.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.220+0000 7f4271c00640 1 --2- 172.21.15.123:0/1524269453 >> v2:172.21.15.155:3300/0 conn(0x7f426c0ce610 0x7f426c15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:10.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.220+0000 7f4271200640 1 --2- 172.21.15.123:0/1524269453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426c0ae990 0x7f426c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:10.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.220+0000 7f426be00640 1 -- 172.21.15.123:0/1524269453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426c0ae990 msgr2=0x7f426c0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:10.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.220+0000 7f426be00640 1 --2- 172.21.15.123:0/1524269453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426c0ae990 0x7f426c0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.220+0000 7f426be00640 1 -- 172.21.15.123:0/1524269453 >> v2:172.21.15.155:3300/0 conn(0x7f426c0ce610 msgr2=0x7f426c15f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:10.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.220+0000 7f426be00640 1 --2- 172.21.15.123:0/1524269453 >> v2:172.21.15.155:3300/0 conn(0x7f426c0ce610 0x7f426c15f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.220+0000 7f426be00640 1 -- 172.21.15.123:0/1524269453 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426c0d0740 con 0x7f426c06c410 2024-04-24T05:47:10.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.222+0000 7f426be00640 1 --2- 172.21.15.123:0/1524269453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f426c06c410 0x7f426c0ce0d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f426c074250 tx=0x7f425802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58f4e5c5088c3aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:10.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.222+0000 7f426b400640 1 -- 172.21.15.123:0/1524269453 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258032790 con 0x7f426c06c410 2024-04-24T05:47:10.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.222+0000 7f426b400640 1 -- 172.21.15.123:0/1524269453 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4258038070 con 0x7f426c06c410 2024-04-24T05:47:10.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.222+0000 7f426b400640 1 -- 172.21.15.123:0/1524269453 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258032b50 con 0x7f426c06c410 2024-04-24T05:47:10.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.222+0000 7f4273c00640 1 -- 172.21.15.123:0/1524269453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f426c06c410 msgr2=0x7f426c0ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:10.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.222+0000 7f4273c00640 1 --2- 172.21.15.123:0/1524269453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f426c06c410 0x7f426c0ce0d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f426c074250 tx=0x7f425802c890 comp rx=0 tx=0).stop 2024-04-24T05:47:10.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.223+0000 7f4273c00640 1 -- 172.21.15.123:0/1524269453 shutdown_connections 2024-04-24T05:47:10.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.223+0000 7f4273c00640 1 --2- 172.21.15.123:0/1524269453 >> v2:172.21.15.155:3300/0 conn(0x7f426c0ce610 0x7f426c15f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.223+0000 7f4273c00640 1 --2- 172.21.15.123:0/1524269453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f426c06c410 0x7f426c0ce0d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.223+0000 7f4273c00640 1 --2- 172.21.15.123:0/1524269453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426c0ae990 0x7f426c0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.223+0000 7f4273c00640 1 -- 172.21.15.123:0/1524269453 >> 172.21.15.123:0/1524269453 conn(0x7f426c0b51f0 msgr2=0x7f426c0b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:10.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.223+0000 7f4273c00640 1 -- 172.21.15.123:0/1524269453 shutdown_connections 2024-04-24T05:47:10.225 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.223+0000 7f4273c00640 1 -- 172.21.15.123:0/1524269453 wait complete. 2024-04-24T05:47:10.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.224+0000 7f4273c00640 1 Processor -- start 2024-04-24T05:47:10.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.225+0000 7f4273c00640 1 -- start start 2024-04-24T05:47:10.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.225+0000 7f4273c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426c06c410 0x7f426c0ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:10.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.225+0000 7f4273c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f426c0ae990 0x7f426c0c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:10.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.225+0000 7f4273c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f426c0ce610 0x7f426c15a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:10.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.225+0000 7f4273c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f426c0aed70 con 0x7f426c0ce610 2024-04-24T05:47:10.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.225+0000 7f4273c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f426c06df00 con 0x7f426c0ae990 2024-04-24T05:47:10.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.225+0000 7f4273c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f426c06c870 con 0x7f426c06c410 2024-04-24T05:47:10.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.226+0000 7f4271c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f426c0ce610 0x7f426c15a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:10.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.226+0000 7f4271c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f426c0ce610 0x7f426c15a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59560/0 (socket says 172.21.15.123:59560) 2024-04-24T05:47:10.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.226+0000 7f4271c00640 1 -- 172.21.15.123:0/838606828 learned_addr learned my addr 172.21.15.123:0/838606828 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:10.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.226+0000 7f426be00640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f426c0ae990 0x7f426c0c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:10.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.226+0000 7f4271200640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426c06c410 0x7f426c0ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:10.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.226+0000 7f4271c00640 1 -- 172.21.15.123:0/838606828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426c06c410 msgr2=0x7f426c0ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:10.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.226+0000 7f4271c00640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426c06c410 0x7f426c0ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.226+0000 7f4271c00640 1 -- 172.21.15.123:0/838606828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f426c0ae990 msgr2=0x7f426c0c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:10.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.226+0000 7f4271c00640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f426c0ae990 0x7f426c0c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.226+0000 7f4271c00640 1 -- 172.21.15.123:0/838606828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426c15d830 con 0x7f426c0ce610 2024-04-24T05:47:10.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.226+0000 7f426be00640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f426c0ae990 0x7f426c0c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:10.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.227+0000 7f4271c00640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f426c0ce610 0x7f426c15a550 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4264007e60 tx=0x7f426400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:10.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.227+0000 7f4268c00640 1 -- 172.21.15.123:0/838606828 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264014420 con 0x7f426c0ce610 2024-04-24T05:47:10.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.227+0000 7f4268c00640 1 -- 172.21.15.123:0/838606828 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4264019070 con 0x7f426c0ce610 2024-04-24T05:47:10.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.227+0000 7f4268c00640 1 -- 172.21.15.123:0/838606828 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264014820 con 0x7f426c0ce610 2024-04-24T05:47:10.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.227+0000 7f4273c00640 1 -- 172.21.15.123:0/838606828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426c0c87e0 con 0x7f426c0ce610 2024-04-24T05:47:10.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.227+0000 7f4273c00640 1 -- 172.21.15.123:0/838606828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f426c15ddb0 con 0x7f426c0ce610 2024-04-24T05:47:10.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.228+0000 7f4273c00640 1 -- 172.21.15.123:0/838606828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f426c068a20 con 0x7f426c0ce610 2024-04-24T05:47:10.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.229+0000 7f4268c00640 1 -- 172.21.15.123:0/838606828 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42640149c0 con 0x7f426c0ce610 2024-04-24T05:47:10.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.230+0000 7f4268c00640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f424c06c8f0 0x7f424c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:10.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.230+0000 7f4268c00640 1 -- 172.21.15.123:0/838606828 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f426408c7d0 con 0x7f426c0ce610 2024-04-24T05:47:10.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.230+0000 7f4271200640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f424c06c8f0 0x7f424c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:10.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.231+0000 7f4271200640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f424c06c8f0 0x7f424c06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4260006fd0 tx=0x7f4260008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:10.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.235+0000 7f4268c00640 1 -- 172.21.15.123:0/838606828 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f426405a7e0 con 0x7f426c0ce610 2024-04-24T05:47:10.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.430+0000 7f4273c00640 1 -- 172.21.15.123:0/838606828 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f426c06cb80 con 0x7f426c0ce610 2024-04-24T05:47:10.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.431+0000 7f4268c00640 1 -- 172.21.15.123:0/838606828 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f426405a0e0 con 0x7f426c0ce610 2024-04-24T05:47:10.432 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:10.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.433+0000 7f4273c00640 1 -- 172.21.15.123:0/838606828 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f424c06c8f0 msgr2=0x7f424c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:10.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.433+0000 7f4273c00640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f424c06c8f0 0x7f424c06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4260006fd0 tx=0x7f4260008040 comp rx=0 tx=0).stop 2024-04-24T05:47:10.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.433+0000 7f4273c00640 1 -- 172.21.15.123:0/838606828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f426c0ce610 msgr2=0x7f426c15a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:10.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.433+0000 7f4273c00640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f426c0ce610 0x7f426c15a550 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4264007e60 tx=0x7f426400e4b0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.433+0000 7f4273c00640 1 -- 172.21.15.123:0/838606828 shutdown_connections 2024-04-24T05:47:10.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.433+0000 7f4273c00640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f424c06c8f0 0x7f424c06edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.433+0000 7f4273c00640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f426c0ce610 0x7f426c15a550 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.433+0000 7f4273c00640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f426c0ae990 0x7f426c0c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.433+0000 7f4273c00640 1 --2- 172.21.15.123:0/838606828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f426c06c410 0x7f426c0ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.433+0000 7f4273c00640 1 -- 172.21.15.123:0/838606828 >> 172.21.15.123:0/838606828 conn(0x7f426c0b51f0 msgr2=0x7f426c0b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:10.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.433+0000 7f4273c00640 1 -- 172.21.15.123:0/838606828 shutdown_connections 2024-04-24T05:47:10.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.433+0000 7f4273c00640 1 -- 172.21.15.123:0/838606828 wait complete. 2024-04-24T05:47:10.454 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T05:47:10.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.655+0000 7f6124e00640 1 Processor -- start 2024-04-24T05:47:10.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.655+0000 7f6124e00640 1 -- start start 2024-04-24T05:47:10.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.656+0000 7f6124e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f612006d120 0x7f6120160000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:10.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.656+0000 7f6124e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f61200cf770 0x7f61200ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:10.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.656+0000 7f6124e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6120160610 con 0x7f612015c450 2024-04-24T05:47:10.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.656+0000 7f6124e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f61200cdcb0 con 0x7f61200cf770 2024-04-24T05:47:10.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.656+0000 7f6124e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f61201608f0 con 0x7f612006d120 2024-04-24T05:47:10.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.656+0000 7f611f400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f612015c450 0x7f612006ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56712/0 (socket says 172.21.15.123:56712) 2024-04-24T05:47:10.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.656+0000 7f611f400640 1 -- 172.21.15.123:0/2671229534 learned_addr learned my addr 172.21.15.123:0/2671229534 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:10.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.656+0000 7f611fe00640 1 --2- 172.21.15.123:0/2671229534 >> v2:172.21.15.123:3300/0 conn(0x7f61200cf770 0x7f61200ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:10.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.657+0000 7f6116a00640 1 --2- 172.21.15.123:0/2671229534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f612006d120 0x7f6120160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:10.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.657+0000 7f611fe00640 1 -- 172.21.15.123:0/2671229534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f612006d120 msgr2=0x7f6120160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:10.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.657+0000 7f611fe00640 1 --2- 172.21.15.123:0/2671229534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f612006d120 0x7f6120160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.657+0000 7f611fe00640 1 -- 172.21.15.123:0/2671229534 >> v1:172.21.15.123:6789/0 conn(0x7f612015c450 legacy=0x7f612006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:10.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.657+0000 7f611fe00640 1 -- 172.21.15.123:0/2671229534 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6120160bd0 con 0x7f61200cf770 2024-04-24T05:47:10.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.657+0000 7f611ea00640 1 -- 172.21.15.123:0/2671229534 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2932421576 0 0) 0x7f6120160610 con 0x7f612015c450 2024-04-24T05:47:10.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.657+0000 7f6116a00640 1 --2- 172.21.15.123:0/2671229534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f612006d120 0x7f6120160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:10.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.658+0000 7f611fe00640 1 --2- 172.21.15.123:0/2671229534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61200cf770 0x7f61200ce140 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6118009d90 tx=0x7f611802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc40d23616755c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:10.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.658+0000 7f611ea00640 1 -- 172.21.15.123:0/2671229534 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118007740 con 0x7f61200cf770 2024-04-24T05:47:10.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.658+0000 7f611ea00640 1 -- 172.21.15.123:0/2671229534 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61180078e0 con 0x7f61200cf770 2024-04-24T05:47:10.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.658+0000 7f611ea00640 1 -- 172.21.15.123:0/2671229534 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118007cb0 con 0x7f61200cf770 2024-04-24T05:47:10.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.659+0000 7f6124e00640 1 -- 172.21.15.123:0/2671229534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61200cf770 msgr2=0x7f61200ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:10.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.659+0000 7f6124e00640 1 --2- 172.21.15.123:0/2671229534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61200cf770 0x7f61200ce140 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6118009d90 tx=0x7f611802a6f0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.659+0000 7f6124e00640 1 -- 172.21.15.123:0/2671229534 shutdown_connections 2024-04-24T05:47:10.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.659+0000 7f6124e00640 1 --2- 172.21.15.123:0/2671229534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61200cf770 0x7f61200ce140 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.659+0000 7f6124e00640 1 --2- 172.21.15.123:0/2671229534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f612006d120 0x7f6120160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.659+0000 7f6124e00640 1 -- 172.21.15.123:0/2671229534 >> 172.21.15.123:0/2671229534 conn(0x7f61200b1b10 msgr2=0x7f61200b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:10.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.659+0000 7f6124e00640 1 -- 172.21.15.123:0/2671229534 shutdown_connections 2024-04-24T05:47:10.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.659+0000 7f6124e00640 1 -- 172.21.15.123:0/2671229534 wait complete. 2024-04-24T05:47:10.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.661+0000 7f6124e00640 1 Processor -- start 2024-04-24T05:47:10.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.661+0000 7f6124e00640 1 -- start start 2024-04-24T05:47:10.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f6124e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f612006d120 0x7f61200c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:10.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f6124e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61200cf770 0x7f6120157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:10.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f6124e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f612015c450 0x7f61201588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:10.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f6124e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f612015c9b0 con 0x7f612015c450 2024-04-24T05:47:10.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f6124e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f612015c830 con 0x7f612006d120 2024-04-24T05:47:10.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f6124e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f61200737f0 con 0x7f61200cf770 2024-04-24T05:47:10.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f611fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f612015c450 0x7f61201588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:10.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f611fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f612015c450 0x7f61201588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59580/0 (socket says 172.21.15.123:59580) 2024-04-24T05:47:10.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f611fe00640 1 -- 172.21.15.123:0/3066912667 learned_addr learned my addr 172.21.15.123:0/3066912667 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:10.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f611f400640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f612006d120 0x7f61200c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:10.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f6116a00640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61200cf770 0x7f6120157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:10.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f611fe00640 1 -- 172.21.15.123:0/3066912667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61200cf770 msgr2=0x7f6120157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:10.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f611fe00640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61200cf770 0x7f6120157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f611fe00640 1 -- 172.21.15.123:0/3066912667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f612006d120 msgr2=0x7f61200c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:10.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f611fe00640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f612006d120 0x7f61200c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.662+0000 7f611fe00640 1 -- 172.21.15.123:0/3066912667 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612015b570 con 0x7f612015c450 2024-04-24T05:47:10.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.663+0000 7f6116a00640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61200cf770 0x7f6120157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:10.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.663+0000 7f611fe00640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f612015c450 0x7f61201588a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6118009d90 tx=0x7f6118004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:10.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.663+0000 7f6117e00640 1 -- 172.21.15.123:0/3066912667 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118040070 con 0x7f612015c450 2024-04-24T05:47:10.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.663+0000 7f6117e00640 1 -- 172.21.15.123:0/3066912667 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6118031c40 con 0x7f612015c450 2024-04-24T05:47:10.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.663+0000 7f6117e00640 1 -- 172.21.15.123:0/3066912667 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118003990 con 0x7f612015c450 2024-04-24T05:47:10.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.663+0000 7f6124e00640 1 -- 172.21.15.123:0/3066912667 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612009b600 con 0x7f612015c450 2024-04-24T05:47:10.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.663+0000 7f6124e00640 1 -- 172.21.15.123:0/3066912667 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612015b240 con 0x7f612015c450 2024-04-24T05:47:10.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.664+0000 7f6124e00640 1 -- 172.21.15.123:0/3066912667 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f612009b3f0 con 0x7f612015c450 2024-04-24T05:47:10.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.665+0000 7f6117e00640 1 -- 172.21.15.123:0/3066912667 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6118039040 con 0x7f612015c450 2024-04-24T05:47:10.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.666+0000 7f6117e00640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f60fc06c8f0 0x7f60fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:10.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.666+0000 7f6117e00640 1 -- 172.21.15.123:0/3066912667 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f61180acf30 con 0x7f612015c450 2024-04-24T05:47:10.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.667+0000 7f611f400640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f60fc06c8f0 0x7f60fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:10.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.667+0000 7f611f400640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f60fc06c8f0 0x7f60fc06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6110000f60 tx=0x7f6110007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:10.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.672+0000 7f6117e00640 1 -- 172.21.15.123:0/3066912667 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f611807af40 con 0x7f612015c450 2024-04-24T05:47:10.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.885+0000 7f6124e00640 1 -- 172.21.15.123:0/3066912667 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f61200b5fa0 con 0x7f612015c450 2024-04-24T05:47:10.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.886+0000 7f6117e00640 1 -- 172.21.15.123:0/3066912667 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f611807a840 con 0x7f612015c450 2024-04-24T05:47:10.888 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-24T05:47:10.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.888+0000 7f6124e00640 1 -- 172.21.15.123:0/3066912667 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f60fc06c8f0 msgr2=0x7f60fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:10.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.888+0000 7f6124e00640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f60fc06c8f0 0x7f60fc06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6110000f60 tx=0x7f6110007c00 comp rx=0 tx=0).stop 2024-04-24T05:47:10.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.888+0000 7f6124e00640 1 -- 172.21.15.123:0/3066912667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f612015c450 msgr2=0x7f61201588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:10.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.888+0000 7f6124e00640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f612015c450 0x7f61201588a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6118009d90 tx=0x7f6118004a40 comp rx=0 tx=0).stop 2024-04-24T05:47:10.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.889+0000 7f6124e00640 1 -- 172.21.15.123:0/3066912667 shutdown_connections 2024-04-24T05:47:10.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.889+0000 7f6124e00640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f60fc06c8f0 0x7f60fc06edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.889+0000 7f6124e00640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f612015c450 0x7f61201588a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.889+0000 7f6124e00640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61200cf770 0x7f6120157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.889+0000 7f6124e00640 1 --2- 172.21.15.123:0/3066912667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f612006d120 0x7f61200c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:10.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.889+0000 7f6124e00640 1 -- 172.21.15.123:0/3066912667 >> 172.21.15.123:0/3066912667 conn(0x7f61200b1b10 msgr2=0x7f61200af120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:10.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.889+0000 7f6124e00640 1 -- 172.21.15.123:0/3066912667 shutdown_connections 2024-04-24T05:47:10.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:10.889+0000 7f6124e00640 1 -- 172.21.15.123:0/3066912667 wait complete. 2024-04-24T05:47:10.908 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:47:10.908 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T05:47:10.966 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:11.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.167+0000 7f3c76200640 1 Processor -- start 2024-04-24T05:47:11.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.167+0000 7f3c76200640 1 -- start start 2024-04-24T05:47:11.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.168+0000 7f3c76200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3c7015c6c0 0x7f3c700b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:11.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.168+0000 7f3c76200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700b3710 0x7f3c700b5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:11.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.169+0000 7f3c76200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c700b2600 con 0x7f3c700718d0 2024-04-24T05:47:11.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.169+0000 7f3c76200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3c700b28e0 con 0x7f3c7015c6c0 2024-04-24T05:47:11.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.169+0000 7f3c76200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3c700b2bc0 con 0x7f3c700b3710 2024-04-24T05:47:11.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.169+0000 7f3c66a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700b3710 0x7f3c700b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:11.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.169+0000 7f3c66a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700b3710 0x7f3c700b5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:49284/0 (socket says 172.21.15.123:49284) 2024-04-24T05:47:11.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.169+0000 7f3c66a00640 1 -- 172.21.15.123:0/2227263640 learned_addr learned my addr 172.21.15.123:0/2227263640 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:11.171 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.169+0000 7f3c6f400640 1 --2- 172.21.15.123:0/2227263640 >> v2:172.21.15.155:3300/0 conn(0x7f3c7015c6c0 0x7f3c700b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:11.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.170+0000 7f3c6f400640 1 -- 172.21.15.123:0/2227263640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700b3710 msgr2=0x7f3c700b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:11.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.170+0000 7f3c6f400640 1 --2- 172.21.15.123:0/2227263640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700b3710 0x7f3c700b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.170+0000 7f3c6f400640 1 -- 172.21.15.123:0/2227263640 >> v1:172.21.15.155:6789/0 conn(0x7f3c700718d0 legacy=0x7f3c70071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:11.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.170+0000 7f3c6f400640 1 -- 172.21.15.123:0/2227263640 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c700b2310 con 0x7f3c7015c6c0 2024-04-24T05:47:11.172 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.170+0000 7f3c66a00640 1 --2- 172.21.15.123:0/2227263640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700b3710 0x7f3c700b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:47:11.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.172+0000 7f3c6f400640 1 --2- 172.21.15.123:0/2227263640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c7015c6c0 0x7f3c700b1dd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3c600097a0 tx=0x7f3c6002a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ac1c926a61a0f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:11.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.172+0000 7f3c6ea00640 1 -- 172.21.15.123:0/2227263640 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c60037070 con 0x7f3c7015c6c0 2024-04-24T05:47:11.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.173+0000 7f3c6ea00640 1 -- 172.21.15.123:0/2227263640 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6002adb0 con 0x7f3c7015c6c0 2024-04-24T05:47:11.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.173+0000 7f3c6ea00640 1 -- 172.21.15.123:0/2227263640 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c60032660 con 0x7f3c7015c6c0 2024-04-24T05:47:11.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.173+0000 7f3c76200640 1 -- 172.21.15.123:0/2227263640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c7015c6c0 msgr2=0x7f3c700b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:11.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.173+0000 7f3c76200640 1 --2- 172.21.15.123:0/2227263640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c7015c6c0 0x7f3c700b1dd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3c600097a0 tx=0x7f3c6002a060 comp rx=0 tx=0).stop 2024-04-24T05:47:11.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.173+0000 7f3c76200640 1 -- 172.21.15.123:0/2227263640 shutdown_connections 2024-04-24T05:47:11.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.173+0000 7f3c76200640 1 --2- 172.21.15.123:0/2227263640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700b3710 0x7f3c700b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.173+0000 7f3c76200640 1 --2- 172.21.15.123:0/2227263640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c7015c6c0 0x7f3c700b1dd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.173+0000 7f3c76200640 1 -- 172.21.15.123:0/2227263640 >> 172.21.15.123:0/2227263640 conn(0x7f3c700d24b0 msgr2=0x7f3c700d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:11.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.173+0000 7f3c76200640 1 -- 172.21.15.123:0/2227263640 shutdown_connections 2024-04-24T05:47:11.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.173+0000 7f3c76200640 1 -- 172.21.15.123:0/2227263640 wait complete. 2024-04-24T05:47:11.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.175+0000 7f3c76200640 1 Processor -- start 2024-04-24T05:47:11.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.175+0000 7f3c76200640 1 -- start start 2024-04-24T05:47:11.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.176+0000 7f3c76200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c700718d0 0x7f3c7006d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:11.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.176+0000 7f3c76200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700b3710 0x7f3c7006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:11.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.176+0000 7f3c76200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c7015c6c0 0x7f3c70159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:11.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.176+0000 7f3c76200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c700d0ad0 con 0x7f3c700718d0 2024-04-24T05:47:11.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.176+0000 7f3c76200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c700d0950 con 0x7f3c7015c6c0 2024-04-24T05:47:11.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.176+0000 7f3c76200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3c700d0c50 con 0x7f3c700b3710 2024-04-24T05:47:11.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.176+0000 7f3c6f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c700718d0 0x7f3c7006d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:11.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.176+0000 7f3c6f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c700718d0 0x7f3c7006d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59592/0 (socket says 172.21.15.123:59592) 2024-04-24T05:47:11.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.176+0000 7f3c6f400640 1 -- 172.21.15.123:0/2203844878 learned_addr learned my addr 172.21.15.123:0/2203844878 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:11.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.177+0000 7f3c6fe00640 1 --2- 172.21.15.123:0/2203844878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c7015c6c0 0x7f3c70159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:11.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.177+0000 7f3c6f400640 1 -- 172.21.15.123:0/2203844878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700b3710 msgr2=0x7f3c7006ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:11.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.177+0000 7f3c6f400640 1 --2- 172.21.15.123:0/2203844878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700b3710 0x7f3c7006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.177+0000 7f3c6f400640 1 -- 172.21.15.123:0/2203844878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c7015c6c0 msgr2=0x7f3c70159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:11.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.177+0000 7f3c6f400640 1 --2- 172.21.15.123:0/2203844878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c7015c6c0 0x7f3c70159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.177+0000 7f3c6f400640 1 -- 172.21.15.123:0/2203844878 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7006d980 con 0x7f3c700718d0 2024-04-24T05:47:11.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.177+0000 7f3c6fe00640 1 --2- 172.21.15.123:0/2203844878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c7015c6c0 0x7f3c70159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:11.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.177+0000 7f3c6f400640 1 --2- 172.21.15.123:0/2203844878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c700718d0 0x7f3c7006d3a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c600098c0 tx=0x7f3c600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:11.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.177+0000 7f3c67e00640 1 -- 172.21.15.123:0/2203844878 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c60041030 con 0x7f3c700718d0 2024-04-24T05:47:11.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.178+0000 7f3c67e00640 1 -- 172.21.15.123:0/2203844878 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6002abb0 con 0x7f3c700718d0 2024-04-24T05:47:11.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.178+0000 7f3c76200640 1 -- 172.21.15.123:0/2203844878 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7006dc60 con 0x7f3c700718d0 2024-04-24T05:47:11.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.178+0000 7f3c67e00640 1 -- 172.21.15.123:0/2203844878 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c60037040 con 0x7f3c700718d0 2024-04-24T05:47:11.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.178+0000 7f3c76200640 1 -- 172.21.15.123:0/2203844878 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7006e220 con 0x7f3c700718d0 2024-04-24T05:47:11.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.179+0000 7f3c76200640 1 -- 172.21.15.123:0/2203844878 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c70068a20 con 0x7f3c700718d0 2024-04-24T05:47:11.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.180+0000 7f3c67e00640 1 -- 172.21.15.123:0/2203844878 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c60044070 con 0x7f3c700718d0 2024-04-24T05:47:11.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.180+0000 7f3c67e00640 1 --2- 172.21.15.123:0/2203844878 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3c4c06c920 0x7f3c4c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:11.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.181+0000 7f3c67e00640 1 -- 172.21.15.123:0/2203844878 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f3c600ac5a0 con 0x7f3c700718d0 2024-04-24T05:47:11.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.181+0000 7f3c66a00640 1 --2- 172.21.15.123:0/2203844878 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3c4c06c920 0x7f3c4c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:11.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.182+0000 7f3c66a00640 1 --2- 172.21.15.123:0/2203844878 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3c4c06c920 0x7f3c4c06ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009630 tx=0x7f3c58006e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:11.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.185+0000 7f3c67e00640 1 -- 172.21.15.123:0/2203844878 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c60044e40 con 0x7f3c700718d0 2024-04-24T05:47:11.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.375+0000 7f3c76200640 1 -- 172.21.15.123:0/2203844878 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c70074e70 con 0x7f3c700718d0 2024-04-24T05:47:11.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.378+0000 7f3c67e00640 1 -- 172.21.15.123:0/2203844878 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7f3c6007a520 con 0x7f3c700718d0 2024-04-24T05:47:11.380 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:11.380 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:11.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.380+0000 7f3c76200640 1 -- 172.21.15.123:0/2203844878 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3c4c06c920 msgr2=0x7f3c4c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:11.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.380+0000 7f3c76200640 1 --2- 172.21.15.123:0/2203844878 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3c4c06c920 0x7f3c4c06ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009630 tx=0x7f3c58006e90 comp rx=0 tx=0).stop 2024-04-24T05:47:11.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.380+0000 7f3c76200640 1 -- 172.21.15.123:0/2203844878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c700718d0 msgr2=0x7f3c7006d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:11.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.380+0000 7f3c76200640 1 --2- 172.21.15.123:0/2203844878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c700718d0 0x7f3c7006d3a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c600098c0 tx=0x7f3c600046c0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.381+0000 7f3c76200640 1 -- 172.21.15.123:0/2203844878 shutdown_connections 2024-04-24T05:47:11.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.381+0000 7f3c76200640 1 --2- 172.21.15.123:0/2203844878 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f3c4c06c920 0x7f3c4c06ede0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.381+0000 7f3c76200640 1 --2- 172.21.15.123:0/2203844878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c7015c6c0 0x7f3c70159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.381+0000 7f3c76200640 1 --2- 172.21.15.123:0/2203844878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700b3710 0x7f3c7006ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.381+0000 7f3c76200640 1 --2- 172.21.15.123:0/2203844878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c700718d0 0x7f3c7006d3a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.381+0000 7f3c76200640 1 -- 172.21.15.123:0/2203844878 >> 172.21.15.123:0/2203844878 conn(0x7f3c700d24b0 msgr2=0x7f3c700cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:11.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.381+0000 7f3c76200640 1 -- 172.21.15.123:0/2203844878 shutdown_connections 2024-04-24T05:47:11.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.381+0000 7f3c76200640 1 -- 172.21.15.123:0/2203844878 wait complete. 2024-04-24T05:47:11.401 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:47:11.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:47:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.601+0000 7f0206200640 1 Processor -- start 2024-04-24T05:47:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.601+0000 7f0206200640 1 -- start start 2024-04-24T05:47:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.602+0000 7f0206200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0200158e00 0x7f020006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.603+0000 7f0206200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f020006bb30 con 0x7f020006ce30 2024-04-24T05:47:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.603+0000 7f0206200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f020006be10 con 0x7f020006e3c0 2024-04-24T05:47:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.603+0000 7f0206200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f020006f590 con 0x7f0200158e00 2024-04-24T05:47:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.603+0000 7f01f7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f020006ce30 0x7f020006dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56714/0 (socket says 172.21.15.123:56714) 2024-04-24T05:47:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.603+0000 7f01f7e00640 1 -- 172.21.15.123:0/1075708543 learned_addr learned my addr 172.21.15.123:0/1075708543 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:11.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.603+0000 7f01ff400640 1 --2- 172.21.15.123:0/1075708543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0200158e00 0x7f020006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.603+0000 7f01fea00640 1 -- 172.21.15.123:0/1075708543 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3991392814 0 0) 0x7f020006bb30 con 0x7f020006ce30 2024-04-24T05:47:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.603+0000 7f01fea00640 1 -- 172.21.15.123:0/1075708543 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01e4003520 con 0x7f020006ce30 2024-04-24T05:47:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.604+0000 7f01fea00640 1 -- 172.21.15.123:0/1075708543 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3428757692 0 0) 0x7f01e4003520 con 0x7f020006ce30 2024-04-24T05:47:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.604+0000 7f01fea00640 1 -- 172.21.15.123:0/1075708543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0200158e00 msgr2=0x7f020006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.604+0000 7f01fea00640 1 --2- 172.21.15.123:0/1075708543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0200158e00 0x7f020006c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.604+0000 7f01fea00640 1 -- 172.21.15.123:0/1075708543 >> v1:172.21.15.155:6789/0 conn(0x7f020006e3c0 legacy=0x7f020006e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.604+0000 7f01fea00640 1 -- 172.21.15.123:0/1075708543 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020006c0f0 con 0x7f020006ce30 2024-04-24T05:47:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.604+0000 7f01fea00640 1 -- 172.21.15.123:0/1075708543 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f01e8001670 con 0x7f020006ce30 2024-04-24T05:47:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.604+0000 7f01fea00640 1 -- 172.21.15.123:0/1075708543 >> v1:172.21.15.123:6789/0 conn(0x7f020006ce30 legacy=0x7f020006dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.604+0000 7f01fea00640 1 --2- 172.21.15.123:0/1075708543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e40039e0 0x7f01e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.604+0000 7f01fea00640 1 --2- 172.21.15.123:0/1075708543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01e4004800 0x7f01e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.604+0000 7f01fea00640 1 --2- 172.21.15.123:0/1075708543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0200158e00 0x7f01e4005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.604+0000 7f01fea00640 1 -- 172.21.15.123:0/1075708543 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f020006f590 con 0x7f01e4004800 2024-04-24T05:47:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.604+0000 7f01fea00640 1 -- 172.21.15.123:0/1075708543 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f01e400bb50 con 0x7f01e40039e0 2024-04-24T05:47:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.604+0000 7f01fea00640 1 -- 172.21.15.123:0/1075708543 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f01e400bd50 con 0x7f0200158e00 2024-04-24T05:47:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.604+0000 7f01f7e00640 1 --2- 172.21.15.123:0/1075708543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01e4004800 0x7f01e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.605+0000 7f01ff400640 1 --2- 172.21.15.123:0/1075708543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e40039e0 0x7f01e4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.605+0000 7f01f7e00640 1 -- 172.21.15.123:0/1075708543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0200158e00 msgr2=0x7f01e4005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.605+0000 7f01f7e00640 1 --2- 172.21.15.123:0/1075708543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0200158e00 0x7f01e4005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.605+0000 7f01f7e00640 1 -- 172.21.15.123:0/1075708543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e40039e0 msgr2=0x7f01e4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.605+0000 7f01f7e00640 1 --2- 172.21.15.123:0/1075708543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e40039e0 0x7f01e4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.605+0000 7f01f7e00640 1 -- 172.21.15.123:0/1075708543 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e4003520 con 0x7f01e4004800 2024-04-24T05:47:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.606+0000 7f01f7e00640 1 --2- 172.21.15.123:0/1075708543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01e4004800 0x7f01e4004c80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f01e8007fd0 tx=0x7f01e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca10e4718dcb7d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.606+0000 7f01fea00640 1 -- 172.21.15.123:0/1075708543 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8007c20 con 0x7f01e4004800 2024-04-24T05:47:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.606+0000 7f01fea00640 1 -- 172.21.15.123:0/1075708543 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e8005af0 con 0x7f01e4004800 2024-04-24T05:47:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.606+0000 7f01fea00640 1 -- 172.21.15.123:0/1075708543 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8007070 con 0x7f01e4004800 2024-04-24T05:47:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.607+0000 7f0206200640 1 -- 172.21.15.123:0/1075708543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01e4004800 msgr2=0x7f01e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.607+0000 7f0206200640 1 --2- 172.21.15.123:0/1075708543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01e4004800 0x7f01e4004c80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f01e8007fd0 tx=0x7f01e4005830 comp rx=0 tx=0).stop 2024-04-24T05:47:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.607+0000 7f01ffe00640 1 -- 172.21.15.123:0/1075708543 reap_dead start 2024-04-24T05:47:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.607+0000 7f0206200640 1 -- 172.21.15.123:0/1075708543 shutdown_connections 2024-04-24T05:47:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.607+0000 7f0206200640 1 -- 172.21.15.123:0/1075708543 >> 172.21.15.123:0/1075708543 conn(0x7f02000b51f0 msgr2=0x7f02000b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.607+0000 7f0206200640 1 -- 172.21.15.123:0/1075708543 shutdown_connections 2024-04-24T05:47:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.607+0000 7f0206200640 1 -- 172.21.15.123:0/1075708543 wait complete. 2024-04-24T05:47:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.609+0000 7f0206200640 1 Processor -- start 2024-04-24T05:47:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.609+0000 7f0206200640 1 -- start start 2024-04-24T05:47:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f0206200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01e4004800 0x7f0200153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f0206200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0200099be0 0x7f020015eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f0206200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0200156d10 0x7f0200157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f0206200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f020006f070 con 0x7f01e4004800 2024-04-24T05:47:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f0206200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f020006c460 con 0x7f0200156d10 2024-04-24T05:47:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f0206200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f020006f1f0 con 0x7f0200099be0 2024-04-24T05:47:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f01ff400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01e4004800 0x7f0200153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f01ff400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01e4004800 0x7f0200153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59614/0 (socket says 172.21.15.123:59614) 2024-04-24T05:47:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f01ff400640 1 -- 172.21.15.123:0/3672742750 learned_addr learned my addr 172.21.15.123:0/3672742750 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f01f7e00640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0200099be0 0x7f020015eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f01ff400640 1 -- 172.21.15.123:0/3672742750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0200099be0 msgr2=0x7f020015eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f01ff400640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0200099be0 0x7f020015eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f01ffe00640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0200156d10 0x7f0200157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f01ff400640 1 -- 172.21.15.123:0/3672742750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0200156d10 msgr2=0x7f0200157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f01ff400640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0200156d10 0x7f0200157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f01ff400640 1 -- 172.21.15.123:0/3672742750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02000730d0 con 0x7f01e4004800 2024-04-24T05:47:11.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.610+0000 7f01f7e00640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0200099be0 0x7f020015eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:11.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.611+0000 7f01ff400640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01e4004800 0x7f0200153a60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f01f00116d0 tx=0x7f01f0011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:11.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.611+0000 7f01f7400640 1 -- 172.21.15.123:0/3672742750 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f000f9c0 con 0x7f01e4004800 2024-04-24T05:47:11.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.611+0000 7f01f7400640 1 -- 172.21.15.123:0/3672742750 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f000fb60 con 0x7f01e4004800 2024-04-24T05:47:11.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.611+0000 7f01f7400640 1 -- 172.21.15.123:0/3672742750 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0015460 con 0x7f01e4004800 2024-04-24T05:47:11.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.611+0000 7f0206200640 1 -- 172.21.15.123:0/3672742750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02000733b0 con 0x7f01e4004800 2024-04-24T05:47:11.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.611+0000 7f0206200640 1 -- 172.21.15.123:0/3672742750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0200161790 con 0x7f01e4004800 2024-04-24T05:47:11.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.612+0000 7f0206200640 1 -- 172.21.15.123:0/3672742750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02001576d0 con 0x7f01e4004800 2024-04-24T05:47:11.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.613+0000 7f01f7400640 1 -- 172.21.15.123:0/3672742750 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01f0015600 con 0x7f01e4004800 2024-04-24T05:47:11.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.614+0000 7f01f7400640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f01dc06c8f0 0x7f01dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:11.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.614+0000 7f01f7400640 1 -- 172.21.15.123:0/3672742750 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f01f008c550 con 0x7f01e4004800 2024-04-24T05:47:11.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.615+0000 7f01f7e00640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f01dc06c8f0 0x7f01dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:11.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.615+0000 7f01f7e00640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f01dc06c8f0 0x7f01dc06edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01e8007fd0 tx=0x7f01e8009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:11.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.619+0000 7f01f7400640 1 -- 172.21.15.123:0/3672742750 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f01f0019030 con 0x7f01e4004800 2024-04-24T05:47:11.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.821+0000 7f0206200640 1 -- 172.21.15.123:0/3672742750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0200068a20 con 0x7f01e4004800 2024-04-24T05:47:11.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.822+0000 7f01f7400640 1 -- 172.21.15.123:0/3672742750 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f01f005a570 con 0x7f01e4004800 2024-04-24T05:47:11.823 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:47:11.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.824+0000 7f0206200640 1 -- 172.21.15.123:0/3672742750 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f01dc06c8f0 msgr2=0x7f01dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:11.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.824+0000 7f0206200640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f01dc06c8f0 0x7f01dc06edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01e8007fd0 tx=0x7f01e8009030 comp rx=0 tx=0).stop 2024-04-24T05:47:11.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.824+0000 7f0206200640 1 -- 172.21.15.123:0/3672742750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01e4004800 msgr2=0x7f0200153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:11.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.824+0000 7f0206200640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01e4004800 0x7f0200153a60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f01f00116d0 tx=0x7f01f0011b90 comp rx=0 tx=0).stop 2024-04-24T05:47:11.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.824+0000 7f0206200640 1 -- 172.21.15.123:0/3672742750 shutdown_connections 2024-04-24T05:47:11.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.824+0000 7f0206200640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f01dc06c8f0 0x7f01dc06edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.825+0000 7f0206200640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0200156d10 0x7f0200157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.825+0000 7f0206200640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0200099be0 0x7f020015eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.825+0000 7f0206200640 1 --2- 172.21.15.123:0/3672742750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01e4004800 0x7f0200153a60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:11.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.825+0000 7f0206200640 1 -- 172.21.15.123:0/3672742750 >> 172.21.15.123:0/3672742750 conn(0x7f02000b51f0 msgr2=0x7f0200153350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:11.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.825+0000 7f0206200640 1 -- 172.21.15.123:0/3672742750 shutdown_connections 2024-04-24T05:47:11.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:11.825+0000 7f0206200640 1 -- 172.21.15.123:0/3672742750 wait complete. 2024-04-24T05:47:11.846 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:47:12.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.051+0000 7f7147600640 1 Processor -- start 2024-04-24T05:47:12.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.052+0000 7f7147600640 1 -- start start 2024-04-24T05:47:12.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.053+0000 7f7147600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f71400cd2c0 0x7f71400d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.053+0000 7f7147600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71400cf170 con 0x7f714006c510 2024-04-24T05:47:12.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.053+0000 7f7147600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71400cf450 con 0x7f7140071c50 2024-04-24T05:47:12.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.053+0000 7f7147600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f71400cf730 con 0x7f71400cd2c0 2024-04-24T05:47:12.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.053+0000 7f7144c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f714006c510 0x7f714006c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56716/0 (socket says 172.21.15.123:56716) 2024-04-24T05:47:12.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.053+0000 7f7144c00640 1 -- 172.21.15.123:0/2748248459 learned_addr learned my addr 172.21.15.123:0/2748248459 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:12.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.053+0000 7f7145600640 1 --2- 172.21.15.123:0/2748248459 >> v2:172.21.15.123:3300/0 conn(0x7f71400cd2c0 0x7f71400d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.054+0000 7f7145600640 1 -- 172.21.15.123:0/2748248459 >> v1:172.21.15.155:6789/0 conn(0x7f7140071c50 legacy=0x7f71400ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:12.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.054+0000 7f7145600640 1 -- 172.21.15.123:0/2748248459 >> v1:172.21.15.123:6789/0 conn(0x7f714006c510 legacy=0x7f714006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.054+0000 7f7145600640 1 -- 172.21.15.123:0/2748248459 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71400cfa10 con 0x7f71400cd2c0 2024-04-24T05:47:12.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.055+0000 7f7145600640 1 --2- 172.21.15.123:0/2748248459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71400cd2c0 0x7f71400d0210 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f71380097a0 tx=0x7f713802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406d375682e48b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:12.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.055+0000 7f713f400640 1 -- 172.21.15.123:0/2748248459 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138037070 con 0x7f71400cd2c0 2024-04-24T05:47:12.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.055+0000 7f713f400640 1 -- 172.21.15.123:0/2748248459 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7138032440 con 0x7f71400cd2c0 2024-04-24T05:47:12.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.055+0000 7f713f400640 1 -- 172.21.15.123:0/2748248459 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138032810 con 0x7f71400cd2c0 2024-04-24T05:47:12.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.056+0000 7f7147600640 1 -- 172.21.15.123:0/2748248459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71400cd2c0 msgr2=0x7f71400d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.056+0000 7f7147600640 1 --2- 172.21.15.123:0/2748248459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71400cd2c0 0x7f71400d0210 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f71380097a0 tx=0x7f713802a1e0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.056+0000 7f7147600640 1 -- 172.21.15.123:0/2748248459 shutdown_connections 2024-04-24T05:47:12.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.056+0000 7f7147600640 1 --2- 172.21.15.123:0/2748248459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71400cd2c0 0x7f71400d0210 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.056+0000 7f7147600640 1 -- 172.21.15.123:0/2748248459 >> 172.21.15.123:0/2748248459 conn(0x7f71400b51f0 msgr2=0x7f71400b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:12.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.056+0000 7f7147600640 1 -- 172.21.15.123:0/2748248459 shutdown_connections 2024-04-24T05:47:12.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.056+0000 7f7147600640 1 -- 172.21.15.123:0/2748248459 wait complete. 2024-04-24T05:47:12.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.058+0000 7f7147600640 1 Processor -- start 2024-04-24T05:47:12.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.058+0000 7f7147600640 1 -- start start 2024-04-24T05:47:12.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.059+0000 7f7147600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f714006c510 0x7f71401ee390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.059+0000 7f7147600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7140071c50 0x7f71400c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.059+0000 7f7147600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71400cd2c0 0x7f71400c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.059+0000 7f7147600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f714006d180 con 0x7f71400cd2c0 2024-04-24T05:47:12.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.059+0000 7f7147600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f714006d000 con 0x7f7140071c50 2024-04-24T05:47:12.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.059+0000 7f7147600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f71400ced00 con 0x7f714006c510 2024-04-24T05:47:12.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.059+0000 7f7145600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71400cd2c0 0x7f71400c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.059+0000 7f7145600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71400cd2c0 0x7f71400c79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59636/0 (socket says 172.21.15.123:59636) 2024-04-24T05:47:12.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.059+0000 7f7145600640 1 -- 172.21.15.123:0/145776782 learned_addr learned my addr 172.21.15.123:0/145776782 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:12.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.059+0000 7f7145600640 1 -- 172.21.15.123:0/145776782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f714006c510 msgr2=0x7f71401ee390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.059+0000 7f7145600640 1 --2- 172.21.15.123:0/145776782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f714006c510 0x7f71401ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.059+0000 7f7145600640 1 -- 172.21.15.123:0/145776782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7140071c50 msgr2=0x7f71400c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.059+0000 7f7145600640 1 --2- 172.21.15.123:0/145776782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7140071c50 0x7f71400c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.060+0000 7f7145600640 1 -- 172.21.15.123:0/145776782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71401f4910 con 0x7f71400cd2c0 2024-04-24T05:47:12.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.060+0000 7f7145600640 1 --2- 172.21.15.123:0/145776782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71400cd2c0 0x7f71400c79a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f71380097a0 tx=0x7f71380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:12.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.060+0000 7f713cc00640 1 -- 172.21.15.123:0/145776782 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138037040 con 0x7f71400cd2c0 2024-04-24T05:47:12.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.060+0000 7f713cc00640 1 -- 172.21.15.123:0/145776782 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7138042070 con 0x7f71400cd2c0 2024-04-24T05:47:12.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.060+0000 7f713cc00640 1 -- 172.21.15.123:0/145776782 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138030b30 con 0x7f71400cd2c0 2024-04-24T05:47:12.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.060+0000 7f7147600640 1 -- 172.21.15.123:0/145776782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71401f4bf0 con 0x7f71400cd2c0 2024-04-24T05:47:12.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.060+0000 7f7147600640 1 -- 172.21.15.123:0/145776782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71401f5490 con 0x7f71400cd2c0 2024-04-24T05:47:12.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.061+0000 7f7147600640 1 -- 172.21.15.123:0/145776782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71401f4dc0 con 0x7f71400cd2c0 2024-04-24T05:47:12.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.062+0000 7f713cc00640 1 -- 172.21.15.123:0/145776782 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7138030cd0 con 0x7f71400cd2c0 2024-04-24T05:47:12.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.063+0000 7f713cc00640 1 --2- 172.21.15.123:0/145776782 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f711c06c940 0x7f711c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.063+0000 7f713cc00640 1 -- 172.21.15.123:0/145776782 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f71380acaa0 con 0x7f71400cd2c0 2024-04-24T05:47:12.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.063+0000 7f7144c00640 1 --2- 172.21.15.123:0/145776782 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f711c06c940 0x7f711c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.064+0000 7f7144c00640 1 --2- 172.21.15.123:0/145776782 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f711c06c940 0x7f711c06ee00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f71300012e0 tx=0x7f7130001630 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:12.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.068+0000 7f713cc00640 1 -- 172.21.15.123:0/145776782 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f713807ab30 con 0x7f71400cd2c0 2024-04-24T05:47:12.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.270+0000 7f7147600640 1 -- 172.21.15.123:0/145776782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7140068a20 con 0x7f71400cd2c0 2024-04-24T05:47:12.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.271+0000 7f713cc00640 1 -- 172.21.15.123:0/145776782 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f713807a430 con 0x7f71400cd2c0 2024-04-24T05:47:12.272 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:12.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.273+0000 7f7147600640 1 -- 172.21.15.123:0/145776782 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f711c06c940 msgr2=0x7f711c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.273+0000 7f7147600640 1 --2- 172.21.15.123:0/145776782 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f711c06c940 0x7f711c06ee00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f71300012e0 tx=0x7f7130001630 comp rx=0 tx=0).stop 2024-04-24T05:47:12.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.273+0000 7f7147600640 1 -- 172.21.15.123:0/145776782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71400cd2c0 msgr2=0x7f71400c79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.273+0000 7f7147600640 1 --2- 172.21.15.123:0/145776782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71400cd2c0 0x7f71400c79a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f71380097a0 tx=0x7f71380046c0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.274+0000 7f7147600640 1 -- 172.21.15.123:0/145776782 shutdown_connections 2024-04-24T05:47:12.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.274+0000 7f7147600640 1 --2- 172.21.15.123:0/145776782 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f711c06c940 0x7f711c06ee00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.274+0000 7f7147600640 1 --2- 172.21.15.123:0/145776782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71400cd2c0 0x7f71400c79a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.274+0000 7f7147600640 1 --2- 172.21.15.123:0/145776782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7140071c50 0x7f71400c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.274+0000 7f7147600640 1 --2- 172.21.15.123:0/145776782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f714006c510 0x7f71401ee390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.274+0000 7f7147600640 1 -- 172.21.15.123:0/145776782 >> 172.21.15.123:0/145776782 conn(0x7f71400b51f0 msgr2=0x7f7140072f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:12.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.274+0000 7f7147600640 1 -- 172.21.15.123:0/145776782 shutdown_connections 2024-04-24T05:47:12.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.274+0000 7f7147600640 1 -- 172.21.15.123:0/145776782 wait complete. 2024-04-24T05:47:12.293 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:47:12.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.493+0000 7f10f0200640 1 Processor -- start 2024-04-24T05:47:12.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.493+0000 7f10f0200640 1 -- start start 2024-04-24T05:47:12.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.495+0000 7f10f0200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e806c510 0x7f10e8071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.495+0000 7f10f0200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f10e806c8f0 0x7f10e806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.495+0000 7f10f0200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e80cf170 con 0x7f10e80ccbb0 2024-04-24T05:47:12.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.495+0000 7f10f0200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f10e80cf450 con 0x7f10e806c8f0 2024-04-24T05:47:12.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.495+0000 7f10f0200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f10e80cf730 con 0x7f10e806c510 2024-04-24T05:47:12.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.495+0000 7f10ee200640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f10e80ccbb0 0x7f10e80d0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56718/0 (socket says 172.21.15.123:56718) 2024-04-24T05:47:12.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.495+0000 7f10ee200640 1 -- 172.21.15.123:0/1984849281 learned_addr learned my addr 172.21.15.123:0/1984849281 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:12.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10ece00640 1 --2- 172.21.15.123:0/1984849281 >> v2:172.21.15.155:3300/0 conn(0x7f10e806c8f0 0x7f10e806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10ed800640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e806c510 0x7f10e8071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 -- 172.21.15.123:0/1984849281 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1089398926 0 0) 0x7f10e80cf170 con 0x7f10e80ccbb0 2024-04-24T05:47:12.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 -- 172.21.15.123:0/1984849281 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10d0003520 con 0x7f10e80ccbb0 2024-04-24T05:47:12.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 -- 172.21.15.123:0/1984849281 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2528117407 0 0) 0x7f10d0003520 con 0x7f10e80ccbb0 2024-04-24T05:47:12.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 -- 172.21.15.123:0/1984849281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e806c510 msgr2=0x7f10e8071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:12.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e806c510 0x7f10e8071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 -- 172.21.15.123:0/1984849281 >> v2:172.21.15.155:3300/0 conn(0x7f10e806c8f0 msgr2=0x7f10e806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:12.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 --2- 172.21.15.123:0/1984849281 >> v2:172.21.15.155:3300/0 conn(0x7f10e806c8f0 0x7f10e806cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 -- 172.21.15.123:0/1984849281 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e80cfa10 con 0x7f10e80ccbb0 2024-04-24T05:47:12.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 -- 172.21.15.123:0/1984849281 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f10e00019f0 con 0x7f10e80ccbb0 2024-04-24T05:47:12.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10ed800640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e806c510 0x7f10e8071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:47:12.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 -- 172.21.15.123:0/1984849281 >> v1:172.21.15.123:6789/0 conn(0x7f10e80ccbb0 legacy=0x7f10e80d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e806c510 0x7f10d00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d0003ef0 0x7f10d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10d0004d40 0x7f10d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 -- 172.21.15.123:0/1984849281 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10e80cf450 con 0x7f10d0004d40 2024-04-24T05:47:12.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 -- 172.21.15.123:0/1984849281 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f10e80cf730 con 0x7f10d0003ef0 2024-04-24T05:47:12.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.496+0000 7f10e7e00640 1 -- 172.21.15.123:0/1984849281 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f10d000b9f0 con 0x7f10e806c510 2024-04-24T05:47:12.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.497+0000 7f10ee200640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10d0004d40 0x7f10d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.497+0000 7f10ece00640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d0003ef0 0x7f10d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.497+0000 7f10ed800640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e806c510 0x7f10d00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.497+0000 7f10ee200640 1 -- 172.21.15.123:0/1984849281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e806c510 msgr2=0x7f10d00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:12.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.497+0000 7f10ee200640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e806c510 0x7f10d00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.497+0000 7f10ee200640 1 -- 172.21.15.123:0/1984849281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d0003ef0 msgr2=0x7f10d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:12.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.497+0000 7f10ee200640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d0003ef0 0x7f10d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.497+0000 7f10ee200640 1 -- 172.21.15.123:0/1984849281 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d0003520 con 0x7f10d0004d40 2024-04-24T05:47:12.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.497+0000 7f10ed800640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10e806c510 0x7f10d00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:12.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.497+0000 7f10ece00640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d0003ef0 0x7f10d0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.498+0000 7f10ee200640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10d0004d40 0x7f10d00051c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f10e000a8a0 tx=0x7f10d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59340d476bd8cf44 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.498+0000 7f10e7e00640 1 -- 172.21.15.123:0/1984849281 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e000a4c0 con 0x7f10d0004d40 2024-04-24T05:47:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.498+0000 7f10e7e00640 1 -- 172.21.15.123:0/1984849281 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10e002f070 con 0x7f10d0004d40 2024-04-24T05:47:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.498+0000 7f10e7e00640 1 -- 172.21.15.123:0/1984849281 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e002bd70 con 0x7f10d0004d40 2024-04-24T05:47:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.499+0000 7f10f0200640 1 -- 172.21.15.123:0/1984849281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10d0004d40 msgr2=0x7f10d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.499+0000 7f10f0200640 1 --2- 172.21.15.123:0/1984849281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10d0004d40 0x7f10d00051c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f10e000a8a0 tx=0x7f10d0005830 comp rx=0 tx=0).stop 2024-04-24T05:47:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.499+0000 7f10ee200640 1 -- 172.21.15.123:0/1984849281 reap_dead start 2024-04-24T05:47:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.499+0000 7f10f0200640 1 -- 172.21.15.123:0/1984849281 shutdown_connections 2024-04-24T05:47:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.499+0000 7f10f0200640 1 -- 172.21.15.123:0/1984849281 >> 172.21.15.123:0/1984849281 conn(0x7f10e80b51f0 msgr2=0x7f10e80b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.499+0000 7f10f0200640 1 -- 172.21.15.123:0/1984849281 shutdown_connections 2024-04-24T05:47:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.499+0000 7f10f0200640 1 -- 172.21.15.123:0/1984849281 wait complete. 2024-04-24T05:47:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.501+0000 7f10f0200640 1 Processor -- start 2024-04-24T05:47:12.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.501+0000 7f10f0200640 1 -- start start 2024-04-24T05:47:12.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.502+0000 7f10f0200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10d0004d40 0x7f10e80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.502+0000 7f10f0200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10e80c4b30 0x7f10e80c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.502+0000 7f10f0200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10e81f54f0 0x7f10e8073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.502+0000 7f10f0200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10e80cee80 con 0x7f10e81f54f0 2024-04-24T05:47:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.502+0000 7f10f0200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f10e80ced00 con 0x7f10e80c4b30 2024-04-24T05:47:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.502+0000 7f10f0200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f10e8071c50 con 0x7f10d0004d40 2024-04-24T05:47:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.502+0000 7f10ee200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10e81f54f0 0x7f10e8073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.503+0000 7f10ee200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10e81f54f0 0x7f10e8073400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59648/0 (socket says 172.21.15.123:59648) 2024-04-24T05:47:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.503+0000 7f10ee200640 1 -- 172.21.15.123:0/2554104927 learned_addr learned my addr 172.21.15.123:0/2554104927 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.503+0000 7f10ed800640 1 --2- 172.21.15.123:0/2554104927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10d0004d40 0x7f10e80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.503+0000 7f10ece00640 1 --2- 172.21.15.123:0/2554104927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10e80c4b30 0x7f10e80c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.503+0000 7f10ee200640 1 -- 172.21.15.123:0/2554104927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10d0004d40 msgr2=0x7f10e80c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.503+0000 7f10ee200640 1 --2- 172.21.15.123:0/2554104927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10d0004d40 0x7f10e80c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.503+0000 7f10ee200640 1 -- 172.21.15.123:0/2554104927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10e80c4b30 msgr2=0x7f10e80c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.503+0000 7f10ee200640 1 --2- 172.21.15.123:0/2554104927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10e80c4b30 0x7f10e80c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.503+0000 7f10ee200640 1 -- 172.21.15.123:0/2554104927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e80c67d0 con 0x7f10e81f54f0 2024-04-24T05:47:12.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.503+0000 7f10ee200640 1 --2- 172.21.15.123:0/2554104927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10e81f54f0 0x7f10e8073400 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f10e000a9c0 tx=0x7f10e0001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:12.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.503+0000 7f10e5600640 1 -- 172.21.15.123:0/2554104927 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e000a4c0 con 0x7f10e81f54f0 2024-04-24T05:47:12.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.504+0000 7f10e5600640 1 -- 172.21.15.123:0/2554104927 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10e00019e0 con 0x7f10e81f54f0 2024-04-24T05:47:12.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.504+0000 7f10f0200640 1 -- 172.21.15.123:0/2554104927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e80c51f0 con 0x7f10e81f54f0 2024-04-24T05:47:12.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.504+0000 7f10e5600640 1 -- 172.21.15.123:0/2554104927 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e0002010 con 0x7f10e81f54f0 2024-04-24T05:47:12.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.504+0000 7f10f0200640 1 -- 172.21.15.123:0/2554104927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e80c57b0 con 0x7f10e81f54f0 2024-04-24T05:47:12.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.505+0000 7f10f0200640 1 -- 172.21.15.123:0/2554104927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e80724f0 con 0x7f10e81f54f0 2024-04-24T05:47:12.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.506+0000 7f10e5600640 1 -- 172.21.15.123:0/2554104927 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10e0047060 con 0x7f10e81f54f0 2024-04-24T05:47:12.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.507+0000 7f10e5600640 1 --2- 172.21.15.123:0/2554104927 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f10c806c8f0 0x7f10c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.507+0000 7f10e5600640 1 -- 172.21.15.123:0/2554104927 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f10e00abf10 con 0x7f10e81f54f0 2024-04-24T05:47:12.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.508+0000 7f10ed800640 1 --2- 172.21.15.123:0/2554104927 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f10c806c8f0 0x7f10c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.508+0000 7f10ed800640 1 --2- 172.21.15.123:0/2554104927 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f10c806c8f0 0x7f10c806edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f10dc006fd0 tx=0x7f10dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:12.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.512+0000 7f10e5600640 1 -- 172.21.15.123:0/2554104927 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10e80724f0 con 0x7f10e81f54f0 2024-04-24T05:47:12.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.711+0000 7f10f0200640 1 -- 172.21.15.123:0/2554104927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f10e80b0c80 con 0x7f10e81f54f0 2024-04-24T05:47:12.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.712+0000 7f10e5600640 1 -- 172.21.15.123:0/2554104927 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f10e80c67d0 con 0x7f10e81f54f0 2024-04-24T05:47:12.714 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:47:12.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.715+0000 7f10f0200640 1 -- 172.21.15.123:0/2554104927 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f10c806c8f0 msgr2=0x7f10c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.715+0000 7f10f0200640 1 --2- 172.21.15.123:0/2554104927 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f10c806c8f0 0x7f10c806edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f10dc006fd0 tx=0x7f10dc008040 comp rx=0 tx=0).stop 2024-04-24T05:47:12.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.715+0000 7f10f0200640 1 -- 172.21.15.123:0/2554104927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10e81f54f0 msgr2=0x7f10e8073400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.715+0000 7f10f0200640 1 --2- 172.21.15.123:0/2554104927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10e81f54f0 0x7f10e8073400 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f10e000a9c0 tx=0x7f10e0001dc0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.715+0000 7f10f0200640 1 -- 172.21.15.123:0/2554104927 shutdown_connections 2024-04-24T05:47:12.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.715+0000 7f10f0200640 1 --2- 172.21.15.123:0/2554104927 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f10c806c8f0 0x7f10c806edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.715+0000 7f10f0200640 1 --2- 172.21.15.123:0/2554104927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10e81f54f0 0x7f10e8073400 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.715+0000 7f10f0200640 1 --2- 172.21.15.123:0/2554104927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10e80c4b30 0x7f10e80c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.715+0000 7f10f0200640 1 --2- 172.21.15.123:0/2554104927 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10d0004d40 0x7f10e80c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.715+0000 7f10f0200640 1 -- 172.21.15.123:0/2554104927 >> 172.21.15.123:0/2554104927 conn(0x7f10e80b51f0 msgr2=0x7f10e8072730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:12.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.715+0000 7f10f0200640 1 -- 172.21.15.123:0/2554104927 shutdown_connections 2024-04-24T05:47:12.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.715+0000 7f10f0200640 1 -- 172.21.15.123:0/2554104927 wait complete. 2024-04-24T05:47:12.734 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:47:12.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.939+0000 7f8586000640 1 Processor -- start 2024-04-24T05:47:12.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.940+0000 7f8586000640 1 -- start start 2024-04-24T05:47:12.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.941+0000 7f8586000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85800d0160 0x7f85800ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.941+0000 7f8586000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f85800ce610 0x7f858006e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.941+0000 7f8586000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85800d0740 con 0x7f85800ae990 2024-04-24T05:47:12.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.941+0000 7f8586000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f85800cea70 con 0x7f85800ce610 2024-04-24T05:47:12.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.941+0000 7f8586000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f858006dcb0 con 0x7f85800d0160 2024-04-24T05:47:12.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.942+0000 7f857f400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f85800ae990 0x7f858006c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56730/0 (socket says 172.21.15.123:56730) 2024-04-24T05:47:12.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.942+0000 7f857f400640 1 -- 172.21.15.123:0/1233210130 learned_addr learned my addr 172.21.15.123:0/1233210130 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:12.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.942+0000 7f8577e00640 1 --2- 172.21.15.123:0/1233210130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85800d0160 0x7f85800ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.942+0000 7f857fe00640 1 --2- 172.21.15.123:0/1233210130 >> v2:172.21.15.155:3300/0 conn(0x7f85800ce610 0x7f858006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.942+0000 7f857ea00640 1 -- 172.21.15.123:0/1233210130 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1699550885 0 0) 0x7f85800d0740 con 0x7f85800ae990 2024-04-24T05:47:12.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.942+0000 7f857ea00640 1 -- 172.21.15.123:0/1233210130 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8564003520 con 0x7f85800ae990 2024-04-24T05:47:12.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857ea00640 1 -- 172.21.15.123:0/1233210130 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2044108537 0 0) 0x7f8564003520 con 0x7f85800ae990 2024-04-24T05:47:12.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857ea00640 1 -- 172.21.15.123:0/1233210130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85800d0160 msgr2=0x7f85800ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:12.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857ea00640 1 --2- 172.21.15.123:0/1233210130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85800d0160 0x7f85800ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857ea00640 1 -- 172.21.15.123:0/1233210130 >> v2:172.21.15.155:3300/0 conn(0x7f85800ce610 msgr2=0x7f858006e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:12.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857ea00640 1 --2- 172.21.15.123:0/1233210130 >> v2:172.21.15.155:3300/0 conn(0x7f85800ce610 0x7f858006e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857ea00640 1 -- 172.21.15.123:0/1233210130 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85800d0a20 con 0x7f85800ae990 2024-04-24T05:47:12.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857ea00640 1 -- 172.21.15.123:0/1233210130 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f8570001670 con 0x7f85800ae990 2024-04-24T05:47:12.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857ea00640 1 -- 172.21.15.123:0/1233210130 >> v1:172.21.15.123:6789/0 conn(0x7f85800ae990 legacy=0x7f858006c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857ea00640 1 --2- 172.21.15.123:0/1233210130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85640039b0 0x7f8564003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857ea00640 1 --2- 172.21.15.123:0/1233210130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85800d0160 0x7f8564004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857ea00640 1 --2- 172.21.15.123:0/1233210130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8564004310 0x7f8564004d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857ea00640 1 -- 172.21.15.123:0/1233210130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f858006dcb0 con 0x7f85640039b0 2024-04-24T05:47:12.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857ea00640 1 -- 172.21.15.123:0/1233210130 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85640054e0 con 0x7f8564004310 2024-04-24T05:47:12.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857ea00640 1 -- 172.21.15.123:0/1233210130 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f856400ffd0 con 0x7f85800d0160 2024-04-24T05:47:12.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857f400640 1 --2- 172.21.15.123:0/1233210130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85640039b0 0x7f8564003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f857fe00640 1 --2- 172.21.15.123:0/1233210130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8564004310 0x7f8564004d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.943+0000 7f8577e00640 1 --2- 172.21.15.123:0/1233210130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85800d0160 0x7f8564004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.944+0000 7f857f400640 1 -- 172.21.15.123:0/1233210130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85800d0160 msgr2=0x7f8564004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:12.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.944+0000 7f857f400640 1 --2- 172.21.15.123:0/1233210130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85800d0160 0x7f8564004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.944+0000 7f857f400640 1 -- 172.21.15.123:0/1233210130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8564004310 msgr2=0x7f8564004d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:12.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.944+0000 7f857f400640 1 --2- 172.21.15.123:0/1233210130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8564004310 0x7f8564004d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.944+0000 7f857f400640 1 -- 172.21.15.123:0/1233210130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8564003520 con 0x7f85640039b0 2024-04-24T05:47:12.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.944+0000 7f857fe00640 1 --2- 172.21.15.123:0/1233210130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8564004310 0x7f8564004d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:47:12.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.945+0000 7f857f400640 1 --2- 172.21.15.123:0/1233210130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85640039b0 0x7f8564003dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f857000c210 tx=0x7f8564005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ad1f61b31d5e3f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:12.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.946+0000 7f857ea00640 1 -- 172.21.15.123:0/1233210130 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570007d50 con 0x7f85640039b0 2024-04-24T05:47:12.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.946+0000 7f857ea00640 1 -- 172.21.15.123:0/1233210130 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8570005ad0 con 0x7f85640039b0 2024-04-24T05:47:12.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.946+0000 7f857ea00640 1 -- 172.21.15.123:0/1233210130 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570007070 con 0x7f85640039b0 2024-04-24T05:47:12.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.946+0000 7f8586000640 1 -- 172.21.15.123:0/1233210130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85640039b0 msgr2=0x7f8564003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.946+0000 7f8586000640 1 --2- 172.21.15.123:0/1233210130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85640039b0 0x7f8564003dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f857000c210 tx=0x7f8564005830 comp rx=0 tx=0).stop 2024-04-24T05:47:12.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.946+0000 7f857fe00640 1 -- 172.21.15.123:0/1233210130 reap_dead start 2024-04-24T05:47:12.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.946+0000 7f8586000640 1 -- 172.21.15.123:0/1233210130 shutdown_connections 2024-04-24T05:47:12.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.946+0000 7f8586000640 1 -- 172.21.15.123:0/1233210130 >> 172.21.15.123:0/1233210130 conn(0x7f85800b51f0 msgr2=0x7f85800b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:12.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.946+0000 7f8586000640 1 -- 172.21.15.123:0/1233210130 shutdown_connections 2024-04-24T05:47:12.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.946+0000 7f8586000640 1 -- 172.21.15.123:0/1233210130 wait complete. 2024-04-24T05:47:12.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.949+0000 7f8586000640 1 Processor -- start 2024-04-24T05:47:12.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.949+0000 7f8586000640 1 -- start start 2024-04-24T05:47:12.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.950+0000 7f8586000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85640039b0 0x7f85800c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.950+0000 7f8586000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85800b2400 0x7f85800cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.950+0000 7f8586000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85800b2880 0x7f85800cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.950+0000 7f8586000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85800aed70 con 0x7f85800b2880 2024-04-24T05:47:12.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.950+0000 7f8586000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f858006e270 con 0x7f85800b2400 2024-04-24T05:47:12.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.950+0000 7f8586000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f85800ad040 con 0x7f85640039b0 2024-04-24T05:47:12.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.950+0000 7f857fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85800b2880 0x7f85800cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.950+0000 7f857fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85800b2880 0x7f85800cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59656/0 (socket says 172.21.15.123:59656) 2024-04-24T05:47:12.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.950+0000 7f857fe00640 1 -- 172.21.15.123:0/3824946902 learned_addr learned my addr 172.21.15.123:0/3824946902 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:12.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.951+0000 7f857f400640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85640039b0 0x7f85800c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.951+0000 7f857fe00640 1 -- 172.21.15.123:0/3824946902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85640039b0 msgr2=0x7f85800c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.951+0000 7f8577e00640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85800b2400 0x7f85800cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.951+0000 7f857fe00640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85640039b0 0x7f85800c5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.951+0000 7f857fe00640 1 -- 172.21.15.123:0/3824946902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85800b2400 msgr2=0x7f85800cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:12.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.951+0000 7f857fe00640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85800b2400 0x7f85800cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:12.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.951+0000 7f857fe00640 1 -- 172.21.15.123:0/3824946902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85800c97a0 con 0x7f85800b2880 2024-04-24T05:47:12.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.951+0000 7f857f400640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85640039b0 0x7f85800c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:12.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.951+0000 7f8577e00640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85800b2400 0x7f85800cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:12.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.951+0000 7f857fe00640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85800b2880 0x7f85800cbae0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8564007b70 tx=0x7f8578010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:12.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.951+0000 7f8577400640 1 -- 172.21.15.123:0/3824946902 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578010db0 con 0x7f85800b2880 2024-04-24T05:47:12.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.952+0000 7f8577400640 1 -- 172.21.15.123:0/3824946902 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8578003990 con 0x7f85800b2880 2024-04-24T05:47:12.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.952+0000 7f8577400640 1 -- 172.21.15.123:0/3824946902 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578003dc0 con 0x7f85800b2880 2024-04-24T05:47:12.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.952+0000 7f8586000640 1 -- 172.21.15.123:0/3824946902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85800c9a80 con 0x7f85800b2880 2024-04-24T05:47:12.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.952+0000 7f8586000640 1 -- 172.21.15.123:0/3824946902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85800ca040 con 0x7f85800b2880 2024-04-24T05:47:12.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.953+0000 7f8586000640 1 -- 172.21.15.123:0/3824946902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8580068a20 con 0x7f85800b2880 2024-04-24T05:47:12.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.954+0000 7f8577400640 1 -- 172.21.15.123:0/3824946902 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8578005d10 con 0x7f85800b2880 2024-04-24T05:47:12.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.955+0000 7f8577400640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f855c06c8f0 0x7f855c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:12.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.955+0000 7f8577400640 1 -- 172.21.15.123:0/3824946902 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f857808c6c0 con 0x7f85800b2880 2024-04-24T05:47:12.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.955+0000 7f857f400640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f855c06c8f0 0x7f855c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:12.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.956+0000 7f857f400640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f855c06c8f0 0x7f855c06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8570033000 tx=0x7f8570009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:12.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:12.959+0000 7f8577400640 1 -- 172.21.15.123:0/3824946902 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8578025020 con 0x7f85800b2880 2024-04-24T05:47:13.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.162+0000 7f8586000640 1 -- 172.21.15.123:0/3824946902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f858006cb80 con 0x7f85800b2880 2024-04-24T05:47:13.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.162+0000 7f8577400640 1 -- 172.21.15.123:0/3824946902 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f857805a640 con 0x7f85800b2880 2024-04-24T05:47:13.164 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:13.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.165+0000 7f8586000640 1 -- 172.21.15.123:0/3824946902 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f855c06c8f0 msgr2=0x7f855c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:13.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.165+0000 7f8586000640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f855c06c8f0 0x7f855c06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8570033000 tx=0x7f8570009030 comp rx=0 tx=0).stop 2024-04-24T05:47:13.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.165+0000 7f8586000640 1 -- 172.21.15.123:0/3824946902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85800b2880 msgr2=0x7f85800cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:13.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.165+0000 7f8586000640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85800b2880 0x7f85800cbae0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8564007b70 tx=0x7f8578010590 comp rx=0 tx=0).stop 2024-04-24T05:47:13.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.165+0000 7f8586000640 1 -- 172.21.15.123:0/3824946902 shutdown_connections 2024-04-24T05:47:13.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.165+0000 7f8586000640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f855c06c8f0 0x7f855c06edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:13.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.165+0000 7f8586000640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85800b2880 0x7f85800cbae0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:13.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.165+0000 7f8586000640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85800b2400 0x7f85800cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:13.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.165+0000 7f8586000640 1 --2- 172.21.15.123:0/3824946902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85640039b0 0x7f85800c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:13.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.165+0000 7f8586000640 1 -- 172.21.15.123:0/3824946902 >> 172.21.15.123:0/3824946902 conn(0x7f85800b51f0 msgr2=0x7f85800ad310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:13.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.165+0000 7f8586000640 1 -- 172.21.15.123:0/3824946902 shutdown_connections 2024-04-24T05:47:13.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.166+0000 7f8586000640 1 -- 172.21.15.123:0/3824946902 wait complete. 2024-04-24T05:47:13.187 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T05:47:13.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.372+0000 7fa6f1400640 1 Processor -- start 2024-04-24T05:47:13.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.372+0000 7fa6f1400640 1 -- start start 2024-04-24T05:47:13.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.373+0000 7fa6f1400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ec06d120 0x7fa6ec160000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:13.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.374+0000 7fa6f1400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6ec160610 con 0x7fa6ec0cf770 2024-04-24T05:47:13.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.374+0000 7fa6f1400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6ec1608f0 con 0x7fa6ec15c450 2024-04-24T05:47:13.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.374+0000 7fa6f1400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ec0cdcb0 con 0x7fa6ec06d120 2024-04-24T05:47:13.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.374+0000 7fa6f0a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa6ec0cf770 0x7fa6ec0ce140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56736/0 (socket says 172.21.15.123:56736) 2024-04-24T05:47:13.376 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.374+0000 7fa6f0a00640 1 -- 172.21.15.123:0/4168808248 learned_addr learned my addr 172.21.15.123:0/4168808248 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:13.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.374+0000 7fa6eb400640 1 --2- 172.21.15.123:0/4168808248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ec06d120 0x7fa6ec160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:13.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 -- 172.21.15.123:0/4168808248 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2923510738 0 0) 0x7fa6ec160610 con 0x7fa6ec0cf770 2024-04-24T05:47:13.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 -- 172.21.15.123:0/4168808248 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6d0003520 con 0x7fa6ec0cf770 2024-04-24T05:47:13.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 -- 172.21.15.123:0/4168808248 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1020407503 0 0) 0x7fa6d0003520 con 0x7fa6ec0cf770 2024-04-24T05:47:13.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 -- 172.21.15.123:0/4168808248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ec06d120 msgr2=0x7fa6ec160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:13.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 --2- 172.21.15.123:0/4168808248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ec06d120 0x7fa6ec160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:13.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 -- 172.21.15.123:0/4168808248 >> v1:172.21.15.155:6789/0 conn(0x7fa6ec15c450 legacy=0x7fa6ec06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:13.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 -- 172.21.15.123:0/4168808248 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec160bd0 con 0x7fa6ec0cf770 2024-04-24T05:47:13.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 -- 172.21.15.123:0/4168808248 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7fa6e4001670 con 0x7fa6ec0cf770 2024-04-24T05:47:13.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 -- 172.21.15.123:0/4168808248 >> v1:172.21.15.123:6789/0 conn(0x7fa6ec0cf770 legacy=0x7fa6ec0ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:13.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 --2- 172.21.15.123:0/4168808248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d00039e0 0x7fa6d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:13.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 --2- 172.21.15.123:0/4168808248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ec06d120 0x7fa6d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:13.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 --2- 172.21.15.123:0/4168808248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6d0004d40 0x7fa6d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:13.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 -- 172.21.15.123:0/4168808248 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec0cdcb0 con 0x7fa6d00039e0 2024-04-24T05:47:13.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 -- 172.21.15.123:0/4168808248 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d000bb60 con 0x7fa6d0004d40 2024-04-24T05:47:13.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6eaa00640 1 -- 172.21.15.123:0/4168808248 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d000bd60 con 0x7fa6ec06d120 2024-04-24T05:47:13.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.375+0000 7fa6ebe00640 1 --2- 172.21.15.123:0/4168808248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d00039e0 0x7fa6d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:13.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.376+0000 7fa6ebe00640 1 -- 172.21.15.123:0/4168808248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ec06d120 msgr2=0x7fa6d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:13.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.376+0000 7fa6ebe00640 1 --2- 172.21.15.123:0/4168808248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6ec06d120 0x7fa6d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:13.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.376+0000 7fa6ebe00640 1 -- 172.21.15.123:0/4168808248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6d0004d40 msgr2=0x7fa6d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:13.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.376+0000 7fa6ebe00640 1 --2- 172.21.15.123:0/4168808248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6d0004d40 0x7fa6d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:13.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.376+0000 7fa6ebe00640 1 -- 172.21.15.123:0/4168808248 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d0003520 con 0x7fa6d00039e0 2024-04-24T05:47:13.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.377+0000 7fa6ebe00640 1 --2- 172.21.15.123:0/4168808248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d00039e0 0x7fa6d0003e20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc001e20 tx=0x7fa6d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2c016b9d73c25b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:13.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.377+0000 7fa6eaa00640 1 -- 172.21.15.123:0/4168808248 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc001a70 con 0x7fa6d00039e0 2024-04-24T05:47:13.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.377+0000 7fa6eaa00640 1 -- 172.21.15.123:0/4168808248 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6dc02ad70 con 0x7fa6d00039e0 2024-04-24T05:47:13.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.377+0000 7fa6eaa00640 1 -- 172.21.15.123:0/4168808248 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc003bb0 con 0x7fa6d00039e0 2024-04-24T05:47:13.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.378+0000 7fa6f1400640 1 -- 172.21.15.123:0/4168808248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d00039e0 msgr2=0x7fa6d0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:13.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.378+0000 7fa6f1400640 1 --2- 172.21.15.123:0/4168808248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d00039e0 0x7fa6d0003e20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc001e20 tx=0x7fa6d0005830 comp rx=0 tx=0).stop 2024-04-24T05:47:13.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.378+0000 7fa6f0a00640 1 -- 172.21.15.123:0/4168808248 reap_dead start 2024-04-24T05:47:13.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.378+0000 7fa6f1400640 1 -- 172.21.15.123:0/4168808248 shutdown_connections 2024-04-24T05:47:13.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.378+0000 7fa6f1400640 1 -- 172.21.15.123:0/4168808248 >> 172.21.15.123:0/4168808248 conn(0x7fa6ec0b1b10 msgr2=0x7fa6ec0b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:13.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.378+0000 7fa6f1400640 1 -- 172.21.15.123:0/4168808248 shutdown_connections 2024-04-24T05:47:13.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.378+0000 7fa6f1400640 1 -- 172.21.15.123:0/4168808248 wait complete. 2024-04-24T05:47:13.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.380+0000 7fa6f1400640 1 Processor -- start 2024-04-24T05:47:13.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.380+0000 7fa6f1400640 1 -- start start 2024-04-24T05:47:13.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.381+0000 7fa6f1400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6d00039e0 0x7fa6ec0a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:13.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.381+0000 7fa6f1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ec0cbb40 0x7fa6ec0b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.381+0000 7fa6f1400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6ec0b3180 0x7fa6ec0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.381+0000 7fa6f1400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec15c9b0 con 0x7fa6ec0cbb40 2024-04-24T05:47:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.381+0000 7fa6f1400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec15c830 con 0x7fa6ec0b3180 2024-04-24T05:47:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.381+0000 7fa6f1400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ec0737f0 con 0x7fa6d00039e0 2024-04-24T05:47:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.381+0000 7fa6eb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ec0cbb40 0x7fa6ec0b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.381+0000 7fa6eb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ec0cbb40 0x7fa6ec0b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59672/0 (socket says 172.21.15.123:59672) 2024-04-24T05:47:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.381+0000 7fa6eb400640 1 -- 172.21.15.123:0/284618928 learned_addr learned my addr 172.21.15.123:0/284618928 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.382+0000 7fa6ebe00640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6d00039e0 0x7fa6ec0a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.382+0000 7fa6f0a00640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6ec0b3180 0x7fa6ec0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.382+0000 7fa6eb400640 1 -- 172.21.15.123:0/284618928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6d00039e0 msgr2=0x7fa6ec0a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.382+0000 7fa6eb400640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6d00039e0 0x7fa6ec0a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.382+0000 7fa6eb400640 1 -- 172.21.15.123:0/284618928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6ec0b3180 msgr2=0x7fa6ec0b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:13.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.382+0000 7fa6eb400640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6ec0b3180 0x7fa6ec0b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:13.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.382+0000 7fa6eb400640 1 -- 172.21.15.123:0/284618928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec0cc150 con 0x7fa6ec0cbb40 2024-04-24T05:47:13.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.382+0000 7fa6f0a00640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6ec0b3180 0x7fa6ec0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:13.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.382+0000 7fa6eb400640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ec0cbb40 0x7fa6ec0b2c40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800e7f0 tx=0x7fa6d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:13.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.382+0000 7fa6e3e00640 1 -- 172.21.15.123:0/284618928 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8017070 con 0x7fa6ec0cbb40 2024-04-24T05:47:13.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.383+0000 7fa6e3e00640 1 -- 172.21.15.123:0/284618928 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d800c940 con 0x7fa6ec0cbb40 2024-04-24T05:47:13.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.383+0000 7fa6f1400640 1 -- 172.21.15.123:0/284618928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ec06c1d0 con 0x7fa6ec0cbb40 2024-04-24T05:47:13.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.383+0000 7fa6e3e00640 1 -- 172.21.15.123:0/284618928 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800eea0 con 0x7fa6ec0cbb40 2024-04-24T05:47:13.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.383+0000 7fa6f1400640 1 -- 172.21.15.123:0/284618928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ec06c4b0 con 0x7fa6ec0cbb40 2024-04-24T05:47:13.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.384+0000 7fa6f1400640 1 -- 172.21.15.123:0/284618928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ec075b70 con 0x7fa6ec0cbb40 2024-04-24T05:47:13.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.385+0000 7fa6e3e00640 1 -- 172.21.15.123:0/284618928 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6d801a070 con 0x7fa6ec0cbb40 2024-04-24T05:47:13.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.386+0000 7fa6e3e00640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa6cc06c8f0 0x7fa6cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:13.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.386+0000 7fa6e3e00640 1 -- 172.21.15.123:0/284618928 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fa6d808c0d0 con 0x7fa6ec0cbb40 2024-04-24T05:47:13.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.386+0000 7fa6ebe00640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa6cc06c8f0 0x7fa6cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:13.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.387+0000 7fa6ebe00640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa6cc06c8f0 0x7fa6cc06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc032000 tx=0x7fa6dc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:13.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.391+0000 7fa6e3e00640 1 -- 172.21.15.123:0/284618928 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6d805a0e0 con 0x7fa6ec0cbb40 2024-04-24T05:47:13.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.602+0000 7fa6f1400640 1 -- 172.21.15.123:0/284618928 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa6ec0b5fa0 con 0x7fa6ec0cbb40 2024-04-24T05:47:13.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.603+0000 7fa6e3e00640 1 -- 172.21.15.123:0/284618928 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fa6d80599e0 con 0x7fa6ec0cbb40 2024-04-24T05:47:13.605 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-24T05:47:13.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.606+0000 7fa6f1400640 1 -- 172.21.15.123:0/284618928 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa6cc06c8f0 msgr2=0x7fa6cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:13.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.606+0000 7fa6f1400640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa6cc06c8f0 0x7fa6cc06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc032000 tx=0x7fa6dc02e000 comp rx=0 tx=0).stop 2024-04-24T05:47:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.606+0000 7fa6f1400640 1 -- 172.21.15.123:0/284618928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ec0cbb40 msgr2=0x7fa6ec0b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.606+0000 7fa6f1400640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ec0cbb40 0x7fa6ec0b2c40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800e7f0 tx=0x7fa6d800ecb0 comp rx=0 tx=0).stop 2024-04-24T05:47:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.606+0000 7fa6f1400640 1 -- 172.21.15.123:0/284618928 shutdown_connections 2024-04-24T05:47:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.606+0000 7fa6f1400640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa6cc06c8f0 0x7fa6cc06edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.606+0000 7fa6f1400640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6ec0b3180 0x7fa6ec0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.606+0000 7fa6f1400640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6ec0cbb40 0x7fa6ec0b2c40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.606+0000 7fa6f1400640 1 --2- 172.21.15.123:0/284618928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6d00039e0 0x7fa6ec0a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.606+0000 7fa6f1400640 1 -- 172.21.15.123:0/284618928 >> 172.21.15.123:0/284618928 conn(0x7fa6ec0b1b10 msgr2=0x7fa6ec09b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.606+0000 7fa6f1400640 1 -- 172.21.15.123:0/284618928 shutdown_connections 2024-04-24T05:47:13.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:13.606+0000 7fa6f1400640 1 -- 172.21.15.123:0/284618928 wait complete. 2024-04-24T05:47:13.628 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2024-04-24T05:47:13.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:13.628 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-24T05:47:13.725 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T05:47:13.726 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-04-24T05:47:13.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:13.726 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-24T05:47:13.804 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:47:13.804 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T05:47:13.861 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:47:13.861 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T05:47:13.915 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:14.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.109+0000 7f65d1e00640 1 Processor -- start 2024-04-24T05:47:14.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.109+0000 7f65d1e00640 1 -- start start 2024-04-24T05:47:14.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.110+0000 7f65d1e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f65cc06f090 0x7f65cc072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.110+0000 7f65d1e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65cc071cd0 con 0x7f65cc0b2990 2024-04-24T05:47:14.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.110+0000 7f65d1e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65cc071fb0 con 0x7f65cc06f470 2024-04-24T05:47:14.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.110+0000 7f65d1e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f65cc072290 con 0x7f65cc06f090 2024-04-24T05:47:14.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.111+0000 7f65caa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f65cc06f090 0x7f65cc072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.111+0000 7f65cb400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f65cc0b2990 0x7f65cc0adfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56742/0 (socket says 172.21.15.123:56742) 2024-04-24T05:47:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.111+0000 7f65cb400640 1 -- 172.21.15.123:0/1724803439 learned_addr learned my addr 172.21.15.123:0/1724803439 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.111+0000 7f65caa00640 1 -- 172.21.15.123:0/1724803439 >> v1:172.21.15.155:6789/0 conn(0x7f65cc06f470 legacy=0x7f65cc151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.111+0000 7f65caa00640 1 -- 172.21.15.123:0/1724803439 >> v1:172.21.15.123:6789/0 conn(0x7f65cc0b2990 legacy=0x7f65cc0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.111+0000 7f65caa00640 1 -- 172.21.15.123:0/1724803439 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65cc072570 con 0x7f65cc06f090 2024-04-24T05:47:14.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.111+0000 7f65ca000640 1 -- 172.21.15.123:0/1724803439 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1673521293 0 0) 0x7f65cc071cd0 con 0x7f65cc0b2990 2024-04-24T05:47:14.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.112+0000 7f65caa00640 1 --2- 172.21.15.123:0/1724803439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65cc06f090 0x7f65cc072fc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0097a0 tx=0x7f65bc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c4d5ae323e7335 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:14.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.113+0000 7f65ca000640 1 -- 172.21.15.123:0/1724803439 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc037070 con 0x7f65cc06f090 2024-04-24T05:47:14.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.113+0000 7f65ca000640 1 -- 172.21.15.123:0/1724803439 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65bc02ae50 con 0x7f65cc06f090 2024-04-24T05:47:14.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.113+0000 7f65ca000640 1 -- 172.21.15.123:0/1724803439 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc032660 con 0x7f65cc06f090 2024-04-24T05:47:14.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.113+0000 7f65d1e00640 1 -- 172.21.15.123:0/1724803439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65cc06f090 msgr2=0x7f65cc072fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.113+0000 7f65d1e00640 1 --2- 172.21.15.123:0/1724803439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65cc06f090 0x7f65cc072fc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0097a0 tx=0x7f65bc02a140 comp rx=0 tx=0).stop 2024-04-24T05:47:14.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.113+0000 7f65d1e00640 1 -- 172.21.15.123:0/1724803439 shutdown_connections 2024-04-24T05:47:14.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.113+0000 7f65d1e00640 1 --2- 172.21.15.123:0/1724803439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65cc06f090 0x7f65cc072fc0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.113+0000 7f65d1e00640 1 -- 172.21.15.123:0/1724803439 >> 172.21.15.123:0/1724803439 conn(0x7f65cc0b1e70 msgr2=0x7f65cc0b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:14.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.113+0000 7f65d1e00640 1 -- 172.21.15.123:0/1724803439 shutdown_connections 2024-04-24T05:47:14.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.113+0000 7f65d1e00640 1 -- 172.21.15.123:0/1724803439 wait complete. 2024-04-24T05:47:14.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.115+0000 7f65d1e00640 1 Processor -- start 2024-04-24T05:47:14.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.115+0000 7f65d1e00640 1 -- start start 2024-04-24T05:47:14.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.116+0000 7f65d1e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65cc06f090 0x7f65cc149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.116+0000 7f65d1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65cc06f470 0x7f65cc09a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.116+0000 7f65d1e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65cc0b2990 0x7f65cc150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.116+0000 7f65d1e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc15c690 con 0x7f65cc06f470 2024-04-24T05:47:14.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.116+0000 7f65d1e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc15c510 con 0x7f65cc0b2990 2024-04-24T05:47:14.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.116+0000 7f65d1e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f65cc072a90 con 0x7f65cc06f090 2024-04-24T05:47:14.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.116+0000 7f65c2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65cc06f470 0x7f65cc09a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:14.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.116+0000 7f65c2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65cc06f470 0x7f65cc09a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59704/0 (socket says 172.21.15.123:59704) 2024-04-24T05:47:14.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.116+0000 7f65c2000640 1 -- 172.21.15.123:0/1964123993 learned_addr learned my addr 172.21.15.123:0/1964123993 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:14.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.117+0000 7f65c2000640 1 -- 172.21.15.123:0/1964123993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65cc06f090 msgr2=0x7f65cc149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.117+0000 7f65caa00640 1 --2- 172.21.15.123:0/1964123993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65cc06f090 0x7f65cc149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:14.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.117+0000 7f65c2000640 1 --2- 172.21.15.123:0/1964123993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65cc06f090 0x7f65cc149900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.117+0000 7f65c2000640 1 -- 172.21.15.123:0/1964123993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65cc0b2990 msgr2=0x7f65cc150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.117+0000 7f65c2000640 1 --2- 172.21.15.123:0/1964123993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65cc0b2990 0x7f65cc150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.117+0000 7f65c2000640 1 -- 172.21.15.123:0/1964123993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65cc09af80 con 0x7f65cc06f470 2024-04-24T05:47:14.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.117+0000 7f65c2000640 1 --2- 172.21.15.123:0/1964123993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65cc06f470 0x7f65cc09a4c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f65b400c460 tx=0x7f65b400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:14.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.117+0000 7f65c3e00640 1 -- 172.21.15.123:0/1964123993 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4016020 con 0x7f65cc06f470 2024-04-24T05:47:14.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.117+0000 7f65c3e00640 1 -- 172.21.15.123:0/1964123993 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b4012870 con 0x7f65cc06f470 2024-04-24T05:47:14.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.117+0000 7f65c3e00640 1 -- 172.21.15.123:0/1964123993 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4012c60 con 0x7f65cc06f470 2024-04-24T05:47:14.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.117+0000 7f65d1e00640 1 -- 172.21.15.123:0/1964123993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65cc09b260 con 0x7f65cc06f470 2024-04-24T05:47:14.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.117+0000 7f65d1e00640 1 -- 172.21.15.123:0/1964123993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65cc09b820 con 0x7f65cc06f470 2024-04-24T05:47:14.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.119+0000 7f65d1e00640 1 -- 172.21.15.123:0/1964123993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65cc068a20 con 0x7f65cc06f470 2024-04-24T05:47:14.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.119+0000 7f65c3e00640 1 -- 172.21.15.123:0/1964123993 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65b4019020 con 0x7f65cc06f470 2024-04-24T05:47:14.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.120+0000 7f65c3e00640 1 --2- 172.21.15.123:0/1964123993 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f65a806c8f0 0x7f65a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.120+0000 7f65c3e00640 1 -- 172.21.15.123:0/1964123993 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f65b408bd40 con 0x7f65cc06f470 2024-04-24T05:47:14.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.121+0000 7f65caa00640 1 --2- 172.21.15.123:0/1964123993 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f65a806c8f0 0x7f65a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:14.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.121+0000 7f65caa00640 1 --2- 172.21.15.123:0/1964123993 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f65a806c8f0 0x7f65a806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f65bc034000 tx=0x7f65bc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:14.126 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.125+0000 7f65c3e00640 1 -- 172.21.15.123:0/1964123993 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65b4020030 con 0x7f65cc06f470 2024-04-24T05:47:14.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.325+0000 7f65d1e00640 1 -- 172.21.15.123:0/1964123993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65cc167ca0 con 0x7f65cc06f470 2024-04-24T05:47:14.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.329+0000 7f65c3e00640 1 -- 172.21.15.123:0/1964123993 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7f65b4059cc0 con 0x7f65cc06f470 2024-04-24T05:47:14.331 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:14.331 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:14.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.332+0000 7f65d1e00640 1 -- 172.21.15.123:0/1964123993 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f65a806c8f0 msgr2=0x7f65a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.332+0000 7f65d1e00640 1 --2- 172.21.15.123:0/1964123993 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f65a806c8f0 0x7f65a806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f65bc034000 tx=0x7f65bc02e000 comp rx=0 tx=0).stop 2024-04-24T05:47:14.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.332+0000 7f65d1e00640 1 -- 172.21.15.123:0/1964123993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65cc06f470 msgr2=0x7f65cc09a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.332+0000 7f65d1e00640 1 --2- 172.21.15.123:0/1964123993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65cc06f470 0x7f65cc09a4c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f65b400c460 tx=0x7f65b400c920 comp rx=0 tx=0).stop 2024-04-24T05:47:14.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.332+0000 7f65d1e00640 1 -- 172.21.15.123:0/1964123993 shutdown_connections 2024-04-24T05:47:14.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.332+0000 7f65d1e00640 1 --2- 172.21.15.123:0/1964123993 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f65a806c8f0 0x7f65a806edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.332+0000 7f65d1e00640 1 --2- 172.21.15.123:0/1964123993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65cc0b2990 0x7f65cc150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.332+0000 7f65d1e00640 1 --2- 172.21.15.123:0/1964123993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65cc06f470 0x7f65cc09a4c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.332+0000 7f65d1e00640 1 --2- 172.21.15.123:0/1964123993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65cc06f090 0x7f65cc149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.332+0000 7f65d1e00640 1 -- 172.21.15.123:0/1964123993 >> 172.21.15.123:0/1964123993 conn(0x7f65cc0b1e70 msgr2=0x7f65cc0b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:14.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.332+0000 7f65d1e00640 1 -- 172.21.15.123:0/1964123993 shutdown_connections 2024-04-24T05:47:14.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.332+0000 7f65d1e00640 1 -- 172.21.15.123:0/1964123993 wait complete. 2024-04-24T05:47:14.353 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:47:14.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:47:14.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.562+0000 7f4faac00640 1 Processor -- start 2024-04-24T05:47:14.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.563+0000 7f4faac00640 1 -- start start 2024-04-24T05:47:14.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.564+0000 7f4faac00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4fa40af210 0x7f4fa4165f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.564+0000 7f4faac00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fa406dbc0 con 0x7f4fa415c470 2024-04-24T05:47:14.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.564+0000 7f4faac00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fa406de30 con 0x7f4fa415b980 2024-04-24T05:47:14.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.564+0000 7f4faac00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4fa4071b80 con 0x7f4fa40af210 2024-04-24T05:47:14.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.564+0000 7f4faa200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4fa40af210 0x7f4fa4165f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:14.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.565+0000 7f4faa200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4fa40af210 0x7f4fa4165f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59712/0 (socket says 172.21.15.123:59712) 2024-04-24T05:47:14.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.565+0000 7f4faa200640 1 -- 172.21.15.123:0/2963110911 learned_addr learned my addr 172.21.15.123:0/2963110911 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:14.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.565+0000 7f4fa8e00640 1 --1- 172.21.15.123:0/2963110911 >> v1:172.21.15.123:6789/0 conn(0x7f4fa415c470 0x7f4fa40aeb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56754/0 (socket says 172.21.15.123:56754) 2024-04-24T05:47:14.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.565+0000 7f4faa200640 1 -- 172.21.15.123:0/2963110911 >> v1:172.21.15.155:6789/0 conn(0x7f4fa415b980 legacy=0x7f4fa415bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.565+0000 7f4faa200640 1 -- 172.21.15.123:0/2963110911 >> v1:172.21.15.123:6789/0 conn(0x7f4fa415c470 legacy=0x7f4fa40aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.565+0000 7f4faa200640 1 -- 172.21.15.123:0/2963110911 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa406e110 con 0x7f4fa40af210 2024-04-24T05:47:14.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.566+0000 7f4faa200640 1 --2- 172.21.15.123:0/2963110911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa40af210 0x7f4fa4165f40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009e50 tx=0x7f4f9c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf58d2c5438acf0e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:14.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.566+0000 7f4fa3e00640 1 -- 172.21.15.123:0/2963110911 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c036070 con 0x7f4fa40af210 2024-04-24T05:47:14.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.566+0000 7f4fa3e00640 1 -- 172.21.15.123:0/2963110911 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9c031600 con 0x7f4fa40af210 2024-04-24T05:47:14.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.566+0000 7f4fa3e00640 1 -- 172.21.15.123:0/2963110911 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c0319d0 con 0x7f4fa40af210 2024-04-24T05:47:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.567+0000 7f4faac00640 1 -- 172.21.15.123:0/2963110911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa40af210 msgr2=0x7f4fa4165f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.567+0000 7f4faac00640 1 --2- 172.21.15.123:0/2963110911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa40af210 0x7f4fa4165f40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009e50 tx=0x7f4f9c02a730 comp rx=0 tx=0).stop 2024-04-24T05:47:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.567+0000 7f4faac00640 1 -- 172.21.15.123:0/2963110911 shutdown_connections 2024-04-24T05:47:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.567+0000 7f4faac00640 1 --2- 172.21.15.123:0/2963110911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa40af210 0x7f4fa4165f40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.567+0000 7f4faac00640 1 -- 172.21.15.123:0/2963110911 >> 172.21.15.123:0/2963110911 conn(0x7f4fa40b1cf0 msgr2=0x7f4fa40b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.567+0000 7f4faac00640 1 -- 172.21.15.123:0/2963110911 shutdown_connections 2024-04-24T05:47:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.567+0000 7f4faac00640 1 -- 172.21.15.123:0/2963110911 wait complete. 2024-04-24T05:47:14.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.569+0000 7f4faac00640 1 Processor -- start 2024-04-24T05:47:14.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.569+0000 7f4faac00640 1 -- start start 2024-04-24T05:47:14.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.570+0000 7f4faac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fa40af210 0x7f4fa414bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.570+0000 7f4faac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa415b980 0x7f4fa40ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.570+0000 7f4faac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa415c470 0x7f4fa40cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.570+0000 7f4faac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa406f920 con 0x7f4fa415c470 2024-04-24T05:47:14.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.570+0000 7f4faac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa40af830 con 0x7f4fa40af210 2024-04-24T05:47:14.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.570+0000 7f4faac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4fa406faa0 con 0x7f4fa415b980 2024-04-24T05:47:14.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.570+0000 7f4faa200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa415c470 0x7f4fa40cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:14.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.570+0000 7f4faa200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa415c470 0x7f4fa40cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59728/0 (socket says 172.21.15.123:59728) 2024-04-24T05:47:14.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.570+0000 7f4faa200640 1 -- 172.21.15.123:0/5846747 learned_addr learned my addr 172.21.15.123:0/5846747 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:14.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.571+0000 7f4faa200640 1 -- 172.21.15.123:0/5846747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa415b980 msgr2=0x7f4fa40ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.571+0000 7f4faa200640 1 --2- 172.21.15.123:0/5846747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa415b980 0x7f4fa40ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.571+0000 7f4faa200640 1 -- 172.21.15.123:0/5846747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fa40af210 msgr2=0x7f4fa414bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.571+0000 7f4faa200640 1 --2- 172.21.15.123:0/5846747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fa40af210 0x7f4fa414bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.571+0000 7f4faa200640 1 -- 172.21.15.123:0/5846747 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa4157790 con 0x7f4fa415c470 2024-04-24T05:47:14.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.571+0000 7f4faa200640 1 --2- 172.21.15.123:0/5846747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa415c470 0x7f4fa40cd150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009f70 tx=0x7f4f9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:14.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.571+0000 7f4fa1600640 1 -- 172.21.15.123:0/5846747 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c03f020 con 0x7f4fa415c470 2024-04-24T05:47:14.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.571+0000 7f4fa1600640 1 -- 172.21.15.123:0/5846747 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9c037040 con 0x7f4fa415c470 2024-04-24T05:47:14.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.571+0000 7f4fa1600640 1 -- 172.21.15.123:0/5846747 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c036070 con 0x7f4fa415c470 2024-04-24T05:47:14.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.571+0000 7f4faac00640 1 -- 172.21.15.123:0/5846747 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa4157a70 con 0x7f4fa415c470 2024-04-24T05:47:14.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.571+0000 7f4faac00640 1 -- 172.21.15.123:0/5846747 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa4158030 con 0x7f4fa415c470 2024-04-24T05:47:14.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.573+0000 7f4faac00640 1 -- 172.21.15.123:0/5846747 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa4068a20 con 0x7f4fa415c470 2024-04-24T05:47:14.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.573+0000 7f4fa1600640 1 -- 172.21.15.123:0/5846747 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f9c042020 con 0x7f4fa415c470 2024-04-24T05:47:14.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.574+0000 7f4fa1600640 1 --2- 172.21.15.123:0/5846747 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4f8406c920 0x7f4f8406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.574+0000 7f4fa1600640 1 -- 172.21.15.123:0/5846747 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f4f9c0ac430 con 0x7f4fa415c470 2024-04-24T05:47:14.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.575+0000 7f4fa9800640 1 --2- 172.21.15.123:0/5846747 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4f8406c920 0x7f4f8406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:14.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.576+0000 7f4fa9800640 1 --2- 172.21.15.123:0/5846747 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4f8406c920 0x7f4f8406ede0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4f98000f60 tx=0x7f4f980079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:14.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.579+0000 7f4fa1600640 1 -- 172.21.15.123:0/5846747 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f9c049030 con 0x7f4fa415c470 2024-04-24T05:47:14.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.777+0000 7f4faac00640 1 -- 172.21.15.123:0/5846747 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4fa406bed0 con 0x7f4fa415c470 2024-04-24T05:47:14.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.778+0000 7f4fa1600640 1 -- 172.21.15.123:0/5846747 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f4f9c07a3b0 con 0x7f4fa415c470 2024-04-24T05:47:14.780 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:47:14.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.781+0000 7f4faac00640 1 -- 172.21.15.123:0/5846747 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4f8406c920 msgr2=0x7f4f8406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.781+0000 7f4faac00640 1 --2- 172.21.15.123:0/5846747 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4f8406c920 0x7f4f8406ede0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4f98000f60 tx=0x7f4f980079b0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.781+0000 7f4faac00640 1 -- 172.21.15.123:0/5846747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa415c470 msgr2=0x7f4fa40cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.781+0000 7f4faac00640 1 --2- 172.21.15.123:0/5846747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa415c470 0x7f4fa40cd150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009f70 tx=0x7f4f9c004a40 comp rx=0 tx=0).stop 2024-04-24T05:47:14.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.781+0000 7f4faac00640 1 -- 172.21.15.123:0/5846747 shutdown_connections 2024-04-24T05:47:14.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.781+0000 7f4faac00640 1 --2- 172.21.15.123:0/5846747 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f4f8406c920 0x7f4f8406ede0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.781+0000 7f4faac00640 1 --2- 172.21.15.123:0/5846747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa415c470 0x7f4fa40cd150 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.781+0000 7f4faac00640 1 --2- 172.21.15.123:0/5846747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fa415b980 0x7f4fa40ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.781+0000 7f4faac00640 1 --2- 172.21.15.123:0/5846747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fa40af210 0x7f4fa414bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.781+0000 7f4faac00640 1 -- 172.21.15.123:0/5846747 >> 172.21.15.123:0/5846747 conn(0x7f4fa40b1cf0 msgr2=0x7f4fa4073390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:14.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.781+0000 7f4faac00640 1 -- 172.21.15.123:0/5846747 shutdown_connections 2024-04-24T05:47:14.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.781+0000 7f4faac00640 1 -- 172.21.15.123:0/5846747 wait complete. 2024-04-24T05:47:14.800 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:47:14.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.987+0000 7ff43f200640 1 Processor -- start 2024-04-24T05:47:14.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.988+0000 7ff43f200640 1 -- start start 2024-04-24T05:47:14.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.989+0000 7ff43f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff43815c2f0 0x7ff438141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.989+0000 7ff43f200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff43806ba00 con 0x7ff4380cc5d0 2024-04-24T05:47:14.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.989+0000 7ff43f200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff438142530 con 0x7ff4380ad6e0 2024-04-24T05:47:14.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.989+0000 7ff43f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff43806bd30 con 0x7ff43815c2f0 2024-04-24T05:47:14.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.989+0000 7ff43d400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff4380cc5d0 0x7ff4380acfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56770/0 (socket says 172.21.15.123:56770) 2024-04-24T05:47:14.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.990+0000 7ff43d400640 1 -- 172.21.15.123:0/621022979 learned_addr learned my addr 172.21.15.123:0/621022979 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:14.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.990+0000 7ff43de00640 1 --2- 172.21.15.123:0/621022979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff43815c2f0 0x7ff438141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:14.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.990+0000 7ff43ca00640 1 -- 172.21.15.123:0/621022979 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2055571987 0 0) 0x7ff43806ba00 con 0x7ff4380cc5d0 2024-04-24T05:47:14.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.990+0000 7ff43ca00640 1 -- 172.21.15.123:0/621022979 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff420003520 con 0x7ff4380cc5d0 2024-04-24T05:47:14.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.990+0000 7ff43ca00640 1 -- 172.21.15.123:0/621022979 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3490240973 0 0) 0x7ff420003520 con 0x7ff4380cc5d0 2024-04-24T05:47:14.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.990+0000 7ff43ca00640 1 -- 172.21.15.123:0/621022979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff43815c2f0 msgr2=0x7ff438141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:14.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.990+0000 7ff43ca00640 1 --2- 172.21.15.123:0/621022979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff43815c2f0 0x7ff438141ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.990+0000 7ff43ca00640 1 -- 172.21.15.123:0/621022979 >> v1:172.21.15.155:6789/0 conn(0x7ff4380ad6e0 legacy=0x7ff43806c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.990+0000 7ff43ca00640 1 -- 172.21.15.123:0/621022979 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff438142810 con 0x7ff4380cc5d0 2024-04-24T05:47:14.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.990+0000 7ff43ca00640 1 -- 172.21.15.123:0/621022979 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7ff428001670 con 0x7ff4380cc5d0 2024-04-24T05:47:14.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.990+0000 7ff43ca00640 1 -- 172.21.15.123:0/621022979 >> v1:172.21.15.123:6789/0 conn(0x7ff4380cc5d0 legacy=0x7ff4380acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.990+0000 7ff43ca00640 1 --2- 172.21.15.123:0/621022979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff43815c2f0 0x7ff4200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.990+0000 7ff43ca00640 1 --2- 172.21.15.123:0/621022979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff420003fb0 0x7ff420004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.991+0000 7ff43ca00640 1 --2- 172.21.15.123:0/621022979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420004d40 0x7ff4200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.991+0000 7ff43ca00640 1 -- 172.21.15.123:0/621022979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff43806bd30 con 0x7ff420003fb0 2024-04-24T05:47:14.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.991+0000 7ff43ca00640 1 -- 172.21.15.123:0/621022979 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff42000bbb0 con 0x7ff420004d40 2024-04-24T05:47:14.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.991+0000 7ff43ca00640 1 -- 172.21.15.123:0/621022979 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff42000bdb0 con 0x7ff43815c2f0 2024-04-24T05:47:14.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.991+0000 7ff43d400640 1 --2- 172.21.15.123:0/621022979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff420003fb0 0x7ff420004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:14.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.991+0000 7ff43d400640 1 -- 172.21.15.123:0/621022979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff43815c2f0 msgr2=0x7ff4200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:14.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.991+0000 7ff43de00640 1 --2- 172.21.15.123:0/621022979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff43815c2f0 0x7ff4200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:14.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.991+0000 7ff43e800640 1 --2- 172.21.15.123:0/621022979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420004d40 0x7ff4200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:14.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.991+0000 7ff43d400640 1 --2- 172.21.15.123:0/621022979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff43815c2f0 0x7ff4200039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.991+0000 7ff43d400640 1 -- 172.21.15.123:0/621022979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420004d40 msgr2=0x7ff4200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:14.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.991+0000 7ff43d400640 1 --2- 172.21.15.123:0/621022979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420004d40 0x7ff4200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:14.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.991+0000 7ff43d400640 1 -- 172.21.15.123:0/621022979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff420003520 con 0x7ff420003fb0 2024-04-24T05:47:14.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.991+0000 7ff43e800640 1 --2- 172.21.15.123:0/621022979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420004d40 0x7ff4200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:14.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.992+0000 7ff43d400640 1 --2- 172.21.15.123:0/621022979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff420003fb0 0x7ff420004800 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff42800c0e0 tx=0x7ff420005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b759a82fe1290d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:14.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.993+0000 7ff43ca00640 1 -- 172.21.15.123:0/621022979 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428007c20 con 0x7ff420003fb0 2024-04-24T05:47:14.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.993+0000 7ff43ca00640 1 -- 172.21.15.123:0/621022979 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff428005af0 con 0x7ff420003fb0 2024-04-24T05:47:14.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.993+0000 7ff43ca00640 1 -- 172.21.15.123:0/621022979 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428007070 con 0x7ff420003fb0 2024-04-24T05:47:14.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.993+0000 7ff43f200640 1 -- 172.21.15.123:0/621022979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff420003fb0 msgr2=0x7ff420004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:14.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.993+0000 7ff43f200640 1 --2- 172.21.15.123:0/621022979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff420003fb0 0x7ff420004800 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff42800c0e0 tx=0x7ff420005830 comp rx=0 tx=0).stop 2024-04-24T05:47:14.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.993+0000 7ff43e800640 1 -- 172.21.15.123:0/621022979 reap_dead start 2024-04-24T05:47:14.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.993+0000 7ff43f200640 1 -- 172.21.15.123:0/621022979 shutdown_connections 2024-04-24T05:47:14.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.993+0000 7ff43f200640 1 -- 172.21.15.123:0/621022979 >> 172.21.15.123:0/621022979 conn(0x7ff4380b1cf0 msgr2=0x7ff4380b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:14.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.993+0000 7ff43f200640 1 -- 172.21.15.123:0/621022979 shutdown_connections 2024-04-24T05:47:14.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.993+0000 7ff43f200640 1 -- 172.21.15.123:0/621022979 wait complete. 2024-04-24T05:47:14.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.995+0000 7ff43f200640 1 Processor -- start 2024-04-24T05:47:14.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.995+0000 7ff43f200640 1 -- start start 2024-04-24T05:47:14.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.996+0000 7ff43f200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420003fb0 0x7ff438145560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.996+0000 7ff43f200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff438158460 0x7ff438159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.996+0000 7ff43f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4381588e0 0x7ff43815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:14.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.996+0000 7ff43f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff43815c850 con 0x7ff4381588e0 2024-04-24T05:47:14.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.996+0000 7ff43f200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff43815c6d0 con 0x7ff420003fb0 2024-04-24T05:47:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.996+0000 7ff43f200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff438141ce0 con 0x7ff438158460 2024-04-24T05:47:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.996+0000 7ff43e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4381588e0 0x7ff43815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.996+0000 7ff43e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4381588e0 0x7ff43815a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59742/0 (socket says 172.21.15.123:59742) 2024-04-24T05:47:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.996+0000 7ff43e800640 1 -- 172.21.15.123:0/629221025 learned_addr learned my addr 172.21.15.123:0/629221025 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.997+0000 7ff43d400640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff438158460 0x7ff438159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.997+0000 7ff43e800640 1 -- 172.21.15.123:0/629221025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff438158460 msgr2=0x7ff438159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.997+0000 7ff43de00640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420003fb0 0x7ff438145560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.997+0000 7ff43e800640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff438158460 0x7ff438159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.997+0000 7ff43e800640 1 -- 172.21.15.123:0/629221025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420003fb0 msgr2=0x7ff438145560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.997+0000 7ff43e800640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420003fb0 0x7ff438145560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.997+0000 7ff43e800640 1 -- 172.21.15.123:0/629221025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff438145aa0 con 0x7ff4381588e0 2024-04-24T05:47:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.997+0000 7ff43de00640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420003fb0 0x7ff438145560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.997+0000 7ff43e800640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4381588e0 0x7ff43815a500 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff438074180 tx=0x7ff43400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:15.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.997+0000 7ff432000640 1 -- 172.21.15.123:0/629221025 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434006d50 con 0x7ff4381588e0 2024-04-24T05:47:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.997+0000 7ff432000640 1 -- 172.21.15.123:0/629221025 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff434003970 con 0x7ff4381588e0 2024-04-24T05:47:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.997+0000 7ff432000640 1 -- 172.21.15.123:0/629221025 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434003d60 con 0x7ff4381588e0 2024-04-24T05:47:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.997+0000 7ff43f200640 1 -- 172.21.15.123:0/629221025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43815ae00 con 0x7ff4381588e0 2024-04-24T05:47:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.998+0000 7ff43f200640 1 -- 172.21.15.123:0/629221025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43815b3c0 con 0x7ff4381588e0 2024-04-24T05:47:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:14.999+0000 7ff43f200640 1 -- 172.21.15.123:0/629221025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4380ae350 con 0x7ff4381588e0 2024-04-24T05:47:15.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.000+0000 7ff432000640 1 -- 172.21.15.123:0/629221025 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff434005070 con 0x7ff4381588e0 2024-04-24T05:47:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.000+0000 7ff432000640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff41c06c920 0x7ff41c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.001+0000 7ff432000640 1 -- 172.21.15.123:0/629221025 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7ff43408bbb0 con 0x7ff4381588e0 2024-04-24T05:47:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.001+0000 7ff43de00640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff41c06c920 0x7ff41c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.001+0000 7ff43de00640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff41c06c920 0x7ff41c06ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0099e0 tx=0x7ff42c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:15.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.005+0000 7ff432000640 1 -- 172.21.15.123:0/629221025 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4380ae350 con 0x7ff4381588e0 2024-04-24T05:47:15.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.193+0000 7ff43f200640 1 -- 172.21.15.123:0/629221025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff438068a20 con 0x7ff4381588e0 2024-04-24T05:47:15.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.193+0000 7ff432000640 1 -- 172.21.15.123:0/629221025 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7ff43408f0a0 con 0x7ff4381588e0 2024-04-24T05:47:15.195 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.196+0000 7ff43f200640 1 -- 172.21.15.123:0/629221025 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff41c06c920 msgr2=0x7ff41c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.196+0000 7ff43f200640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff41c06c920 0x7ff41c06ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0099e0 tx=0x7ff42c008040 comp rx=0 tx=0).stop 2024-04-24T05:47:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.196+0000 7ff43f200640 1 -- 172.21.15.123:0/629221025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4381588e0 msgr2=0x7ff43815a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.196+0000 7ff43f200640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4381588e0 0x7ff43815a500 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff438074180 tx=0x7ff43400dc00 comp rx=0 tx=0).stop 2024-04-24T05:47:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.196+0000 7ff43f200640 1 -- 172.21.15.123:0/629221025 shutdown_connections 2024-04-24T05:47:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.196+0000 7ff43f200640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff41c06c920 0x7ff41c06ede0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.196+0000 7ff43f200640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4381588e0 0x7ff43815a500 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.196+0000 7ff43f200640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff438158460 0x7ff438159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.196+0000 7ff43f200640 1 --2- 172.21.15.123:0/629221025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff420003fb0 0x7ff438145560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.196+0000 7ff43f200640 1 -- 172.21.15.123:0/629221025 >> 172.21.15.123:0/629221025 conn(0x7ff4380b1cf0 msgr2=0x7ff43806c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:15.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.196+0000 7ff43f200640 1 -- 172.21.15.123:0/629221025 shutdown_connections 2024-04-24T05:47:15.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.196+0000 7ff43f200640 1 -- 172.21.15.123:0/629221025 wait complete. 2024-04-24T05:47:15.216 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:47:15.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.410+0000 7fc7fd000640 1 Processor -- start 2024-04-24T05:47:15.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.411+0000 7fc7fd000640 1 -- start start 2024-04-24T05:47:15.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.412+0000 7fc7fd000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc7f815e970 0x7fc7f80d44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.412+0000 7fc7fd000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc7f80d3e10 0x7fc7f80d4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.412+0000 7fc7fd000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f80b39b0 0x7fc7f80d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.412+0000 7fc7fd000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc7f80d09c0 con 0x7fc7f80d3e10 2024-04-24T05:47:15.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.412+0000 7fc7fd000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc7f80cfdf0 con 0x7fc7f815e970 2024-04-24T05:47:15.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.412+0000 7fc7fd000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f80d0090 con 0x7fc7f80b39b0 2024-04-24T05:47:15.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.412+0000 7fc7ed400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc7f80d3e10 0x7fc7f80d4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.412+0000 7fc7ed400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc7f80d3e10 0x7fc7f80d4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59750/0 (socket says 172.21.15.123:59750) 2024-04-24T05:47:15.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.413+0000 7fc7ed400640 1 -- 172.21.15.123:0/740525939 learned_addr learned my addr 172.21.15.123:0/740525939 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:15.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.413+0000 7fc7f5e00640 1 --2- 172.21.15.123:0/740525939 >> v2:172.21.15.155:3300/0 conn(0x7fc7f815e970 0x7fc7f80d44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.413+0000 7fc7f6800640 1 --2- 172.21.15.123:0/740525939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f80b39b0 0x7fc7f80d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.413+0000 7fc7ed400640 1 -- 172.21.15.123:0/740525939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f80b39b0 msgr2=0x7fc7f80d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:15.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.413+0000 7fc7ed400640 1 --2- 172.21.15.123:0/740525939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f80b39b0 0x7fc7f80d2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.413+0000 7fc7ed400640 1 -- 172.21.15.123:0/740525939 >> v2:172.21.15.155:3300/0 conn(0x7fc7f815e970 msgr2=0x7fc7f80d44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:15.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.413+0000 7fc7ed400640 1 --2- 172.21.15.123:0/740525939 >> v2:172.21.15.155:3300/0 conn(0x7fc7f815e970 0x7fc7f80d44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.413+0000 7fc7ed400640 1 -- 172.21.15.123:0/740525939 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f80d0370 con 0x7fc7f80d3e10 2024-04-24T05:47:15.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.413+0000 7fc7f5e00640 1 --2- 172.21.15.123:0/740525939 >> v2:172.21.15.155:3300/0 conn(0x7fc7f815e970 0x7fc7f80d44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:15.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.414+0000 7fc7ed400640 1 --2- 172.21.15.123:0/740525939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80d3e10 0x7fc7f80d4a30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc7f808ca20 tx=0x7fc7e002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ea5a85ea1118f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:15.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.414+0000 7fc7f5400640 1 -- 172.21.15.123:0/740525939 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0037070 con 0x7fc7f80d3e10 2024-04-24T05:47:15.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.414+0000 7fc7f5400640 1 -- 172.21.15.123:0/740525939 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e0032440 con 0x7fc7f80d3e10 2024-04-24T05:47:15.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.414+0000 7fc7f5400640 1 -- 172.21.15.123:0/740525939 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0032810 con 0x7fc7f80d3e10 2024-04-24T05:47:15.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.415+0000 7fc7fd000640 1 -- 172.21.15.123:0/740525939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80d3e10 msgr2=0x7fc7f80d4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:15.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.415+0000 7fc7fd000640 1 --2- 172.21.15.123:0/740525939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80d3e10 0x7fc7f80d4a30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc7f808ca20 tx=0x7fc7e002a140 comp rx=0 tx=0).stop 2024-04-24T05:47:15.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.415+0000 7fc7fd000640 1 -- 172.21.15.123:0/740525939 shutdown_connections 2024-04-24T05:47:15.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.415+0000 7fc7fd000640 1 --2- 172.21.15.123:0/740525939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f80b39b0 0x7fc7f80d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.415+0000 7fc7fd000640 1 --2- 172.21.15.123:0/740525939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80d3e10 0x7fc7f80d4a30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.415+0000 7fc7fd000640 1 --2- 172.21.15.123:0/740525939 >> v2:172.21.15.155:3300/0 conn(0x7fc7f815e970 0x7fc7f80d44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.415+0000 7fc7fd000640 1 -- 172.21.15.123:0/740525939 >> 172.21.15.123:0/740525939 conn(0x7fc7f80b9a40 msgr2=0x7fc7f80b75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:15.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.415+0000 7fc7fd000640 1 -- 172.21.15.123:0/740525939 shutdown_connections 2024-04-24T05:47:15.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.415+0000 7fc7fd000640 1 -- 172.21.15.123:0/740525939 wait complete. 2024-04-24T05:47:15.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.417+0000 7fc7fd000640 1 Processor -- start 2024-04-24T05:47:15.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.417+0000 7fc7fd000640 1 -- start start 2024-04-24T05:47:15.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.418+0000 7fc7fd000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80b39b0 0x7fc7f81576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.418+0000 7fc7fd000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f80d3e10 0x7fc7f80b6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.418+0000 7fc7fd000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7f815e970 0x7fc7f806d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.418+0000 7fc7fd000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f80b12b0 con 0x7fc7f80b39b0 2024-04-24T05:47:15.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.418+0000 7fc7fd000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f815ed50 con 0x7fc7f815e970 2024-04-24T05:47:15.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.418+0000 7fc7fd000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f80d53a0 con 0x7fc7f80d3e10 2024-04-24T05:47:15.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.418+0000 7fc7f5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80b39b0 0x7fc7f81576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.418+0000 7fc7f5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80b39b0 0x7fc7f81576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59764/0 (socket says 172.21.15.123:59764) 2024-04-24T05:47:15.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.418+0000 7fc7f5e00640 1 -- 172.21.15.123:0/191655102 learned_addr learned my addr 172.21.15.123:0/191655102 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:15.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.419+0000 7fc7f6800640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7f815e970 0x7fc7f806d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.419+0000 7fc7ed400640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f80d3e10 0x7fc7f80b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.419+0000 7fc7f5e00640 1 -- 172.21.15.123:0/191655102 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f80d3e10 msgr2=0x7fc7f80b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:15.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.419+0000 7fc7f5e00640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f80d3e10 0x7fc7f80b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.419+0000 7fc7f5e00640 1 -- 172.21.15.123:0/191655102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7f815e970 msgr2=0x7fc7f806d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:15.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.419+0000 7fc7f5e00640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7f815e970 0x7fc7f806d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.419+0000 7fc7f5e00640 1 -- 172.21.15.123:0/191655102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f81f4ac0 con 0x7fc7f80b39b0 2024-04-24T05:47:15.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.419+0000 7fc7ed400640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f80d3e10 0x7fc7f80b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:15.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.419+0000 7fc7f6800640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7f815e970 0x7fc7f806d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:15.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.419+0000 7fc7f5e00640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80b39b0 0x7fc7f81576e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800c820 tx=0x7fc7e800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:15.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.419+0000 7fc7eea00640 1 -- 172.21.15.123:0/191655102 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8015070 con 0x7fc7f80b39b0 2024-04-24T05:47:15.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.419+0000 7fc7eea00640 1 -- 172.21.15.123:0/191655102 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e8007a30 con 0x7fc7f80b39b0 2024-04-24T05:47:15.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.419+0000 7fc7fd000640 1 -- 172.21.15.123:0/191655102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f81f4da0 con 0x7fc7f80b39b0 2024-04-24T05:47:15.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.419+0000 7fc7eea00640 1 -- 172.21.15.123:0/191655102 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8007de0 con 0x7fc7f80b39b0 2024-04-24T05:47:15.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.420+0000 7fc7fd000640 1 -- 172.21.15.123:0/191655102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f81f5360 con 0x7fc7f80b39b0 2024-04-24T05:47:15.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.421+0000 7fc7fd000640 1 -- 172.21.15.123:0/191655102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f8068a20 con 0x7fc7f80b39b0 2024-04-24T05:47:15.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.422+0000 7fc7eea00640 1 -- 172.21.15.123:0/191655102 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7e8018070 con 0x7fc7f80b39b0 2024-04-24T05:47:15.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.423+0000 7fc7eea00640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc7d406c920 0x7fc7d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.423+0000 7fc7eea00640 1 -- 172.21.15.123:0/191655102 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fc7e808bdc0 con 0x7fc7f80b39b0 2024-04-24T05:47:15.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.423+0000 7fc7ed400640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc7d406c920 0x7fc7d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.424+0000 7fc7ed400640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc7d406c920 0x7fc7d406ede0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0034000 tx=0x7fc7e002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:15.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.427+0000 7fc7eea00640 1 -- 172.21.15.123:0/191655102 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7e8059de0 con 0x7fc7f80b39b0 2024-04-24T05:47:15.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.629+0000 7fc7fd000640 1 -- 172.21.15.123:0/191655102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc7f8161560 con 0x7fc7f80b39b0 2024-04-24T05:47:15.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.630+0000 7fc7eea00640 1 -- 172.21.15.123:0/191655102 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fc7e80596e0 con 0x7fc7f80b39b0 2024-04-24T05:47:15.632 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:47:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.633+0000 7fc7fd000640 1 -- 172.21.15.123:0/191655102 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc7d406c920 msgr2=0x7fc7d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.633+0000 7fc7fd000640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc7d406c920 0x7fc7d406ede0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0034000 tx=0x7fc7e002e000 comp rx=0 tx=0).stop 2024-04-24T05:47:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.633+0000 7fc7fd000640 1 -- 172.21.15.123:0/191655102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80b39b0 msgr2=0x7fc7f81576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.633+0000 7fc7fd000640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80b39b0 0x7fc7f81576e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800c820 tx=0x7fc7e800cce0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.633+0000 7fc7fd000640 1 -- 172.21.15.123:0/191655102 shutdown_connections 2024-04-24T05:47:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.633+0000 7fc7fd000640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fc7d406c920 0x7fc7d406ede0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.633+0000 7fc7fd000640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7f815e970 0x7fc7f806d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.633+0000 7fc7fd000640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7f80d3e10 0x7fc7f80b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.633+0000 7fc7fd000640 1 --2- 172.21.15.123:0/191655102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f80b39b0 0x7fc7f81576e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.633+0000 7fc7fd000640 1 -- 172.21.15.123:0/191655102 >> 172.21.15.123:0/191655102 conn(0x7fc7f80b9a40 msgr2=0x7fc7f815dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:15.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.633+0000 7fc7fd000640 1 -- 172.21.15.123:0/191655102 shutdown_connections 2024-04-24T05:47:15.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.633+0000 7fc7fd000640 1 -- 172.21.15.123:0/191655102 wait complete. 2024-04-24T05:47:15.654 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:47:15.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.838+0000 7ff57d600640 1 Processor -- start 2024-04-24T05:47:15.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.838+0000 7ff57d600640 1 -- start start 2024-04-24T05:47:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.840+0000 7ff57d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57815b980 0x7ff57815bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.840+0000 7ff57d600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff57806dbc0 con 0x7ff5780af210 2024-04-24T05:47:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.840+0000 7ff57d600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff57806de30 con 0x7ff57815c2a0 2024-04-24T05:47:15.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.840+0000 7ff57d600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff578071b80 con 0x7ff57815b980 2024-04-24T05:47:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.840+0000 7ff57cc00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff5780af210 0x7ff578164f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56780/0 (socket says 172.21.15.123:56780) 2024-04-24T05:47:15.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.840+0000 7ff57cc00640 1 -- 172.21.15.123:0/3353778613 learned_addr learned my addr 172.21.15.123:0/3353778613 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577e00640 1 --2- 172.21.15.123:0/3353778613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57815b980 0x7ff57815bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.840+0000 7ff577400640 1 -- 172.21.15.123:0/3353778613 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1355870823 0 0) 0x7ff57806dbc0 con 0x7ff5780af210 2024-04-24T05:47:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577400640 1 -- 172.21.15.123:0/3353778613 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff55c003520 con 0x7ff5780af210 2024-04-24T05:47:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577400640 1 -- 172.21.15.123:0/3353778613 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2925665548 0 0) 0x7ff55c003520 con 0x7ff5780af210 2024-04-24T05:47:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577400640 1 -- 172.21.15.123:0/3353778613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57815b980 msgr2=0x7ff57815bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577400640 1 --2- 172.21.15.123:0/3353778613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57815b980 0x7ff57815bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577400640 1 -- 172.21.15.123:0/3353778613 >> v1:172.21.15.155:6789/0 conn(0x7ff57815c2a0 legacy=0x7ff5780aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577400640 1 -- 172.21.15.123:0/3353778613 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57806e110 con 0x7ff5780af210 2024-04-24T05:47:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577400640 1 -- 172.21.15.123:0/3353778613 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7ff5700019f0 con 0x7ff5780af210 2024-04-24T05:47:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577400640 1 -- 172.21.15.123:0/3353778613 >> v1:172.21.15.123:6789/0 conn(0x7ff5780af210 legacy=0x7ff578164f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577400640 1 --2- 172.21.15.123:0/3353778613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff55c0039e0 0x7ff55c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577400640 1 --2- 172.21.15.123:0/3353778613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57815b980 0x7ff55c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577400640 1 --2- 172.21.15.123:0/3353778613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff55c004d40 0x7ff55c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577400640 1 -- 172.21.15.123:0/3353778613 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c00dc80 con 0x7ff55c004d40 2024-04-24T05:47:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577400640 1 -- 172.21.15.123:0/3353778613 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c00de00 con 0x7ff55c0039e0 2024-04-24T05:47:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff577400640 1 -- 172.21.15.123:0/3353778613 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff55c00e000 con 0x7ff57815b980 2024-04-24T05:47:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.841+0000 7ff57cc00640 1 --2- 172.21.15.123:0/3353778613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff55c004d40 0x7ff55c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.842+0000 7ff577e00640 1 --2- 172.21.15.123:0/3353778613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff55c0039e0 0x7ff55c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.842+0000 7ff57cc00640 1 -- 172.21.15.123:0/3353778613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57815b980 msgr2=0x7ff55c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.842+0000 7ff57cc00640 1 --2- 172.21.15.123:0/3353778613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57815b980 0x7ff55c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.842+0000 7ff57cc00640 1 -- 172.21.15.123:0/3353778613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff55c0039e0 msgr2=0x7ff55c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.842+0000 7ff57cc00640 1 --2- 172.21.15.123:0/3353778613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff55c0039e0 0x7ff55c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.842+0000 7ff57cc00640 1 -- 172.21.15.123:0/3353778613 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c003520 con 0x7ff55c004d40 2024-04-24T05:47:15.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.843+0000 7ff57cc00640 1 --2- 172.21.15.123:0/3353778613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff55c004d40 0x7ff55c0051c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff57000a790 tx=0x7ff55c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff6a977160595e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:15.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.843+0000 7ff577400640 1 -- 172.21.15.123:0/3353778613 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57000a3b0 con 0x7ff55c004d40 2024-04-24T05:47:15.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.843+0000 7ff577400640 1 -- 172.21.15.123:0/3353778613 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57002f070 con 0x7ff55c004d40 2024-04-24T05:47:15.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.843+0000 7ff577400640 1 -- 172.21.15.123:0/3353778613 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57002bbe0 con 0x7ff55c004d40 2024-04-24T05:47:15.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.844+0000 7ff57d600640 1 -- 172.21.15.123:0/3353778613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff55c004d40 msgr2=0x7ff55c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:15.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.844+0000 7ff57d600640 1 --2- 172.21.15.123:0/3353778613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff55c004d40 0x7ff55c0051c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff57000a790 tx=0x7ff55c005830 comp rx=0 tx=0).stop 2024-04-24T05:47:15.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.844+0000 7ff57cc00640 1 -- 172.21.15.123:0/3353778613 reap_dead start 2024-04-24T05:47:15.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.844+0000 7ff57d600640 1 -- 172.21.15.123:0/3353778613 shutdown_connections 2024-04-24T05:47:15.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.844+0000 7ff57d600640 1 -- 172.21.15.123:0/3353778613 >> 172.21.15.123:0/3353778613 conn(0x7ff5780b1cf0 msgr2=0x7ff5780b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:15.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.844+0000 7ff57d600640 1 -- 172.21.15.123:0/3353778613 shutdown_connections 2024-04-24T05:47:15.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.844+0000 7ff57d600640 1 -- 172.21.15.123:0/3353778613 wait complete. 2024-04-24T05:47:15.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.846+0000 7ff57d600640 1 Processor -- start 2024-04-24T05:47:15.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.846+0000 7ff57d600640 1 -- start start 2024-04-24T05:47:15.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff57d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff55c004d40 0x7ff578154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff57d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff578152330 0x7ff578153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff57d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff578153d70 0x7ff578154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff57d600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5780af830 con 0x7ff578152330 2024-04-24T05:47:15.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff57d600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff57815c700 con 0x7ff578153d70 2024-04-24T05:47:15.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff57d600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff57806f920 con 0x7ff55c004d40 2024-04-24T05:47:15.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff56fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff578152330 0x7ff578153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff56fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff578152330 0x7ff578153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59778/0 (socket says 172.21.15.123:59778) 2024-04-24T05:47:15.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff56fe00640 1 -- 172.21.15.123:0/4011933550 learned_addr learned my addr 172.21.15.123:0/4011933550 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:15.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff577e00640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff55c004d40 0x7ff578154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff57cc00640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff578153d70 0x7ff578154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff56fe00640 1 -- 172.21.15.123:0/4011933550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff55c004d40 msgr2=0x7ff578154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff56fe00640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff55c004d40 0x7ff578154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff56fe00640 1 -- 172.21.15.123:0/4011933550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff578153d70 msgr2=0x7ff578154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff56fe00640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff578153d70 0x7ff578154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff56fe00640 1 -- 172.21.15.123:0/4011933550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57809a150 con 0x7ff578152330 2024-04-24T05:47:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.847+0000 7ff57cc00640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff578153d70 0x7ff578154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.848+0000 7ff56fe00640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff578152330 0x7ff578153830 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff56000c410 tx=0x7ff56000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.848+0000 7ff574c00640 1 -- 172.21.15.123:0/4011933550 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560011790 con 0x7ff578152330 2024-04-24T05:47:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.848+0000 7ff574c00640 1 -- 172.21.15.123:0/4011933550 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff560017070 con 0x7ff578152330 2024-04-24T05:47:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.848+0000 7ff57d600640 1 -- 172.21.15.123:0/4011933550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57809a430 con 0x7ff578152330 2024-04-24T05:47:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.848+0000 7ff574c00640 1 -- 172.21.15.123:0/4011933550 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560011bb0 con 0x7ff578152330 2024-04-24T05:47:15.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.848+0000 7ff57d600640 1 -- 172.21.15.123:0/4011933550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff578140b80 con 0x7ff578152330 2024-04-24T05:47:15.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.849+0000 7ff57d600640 1 -- 172.21.15.123:0/4011933550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff578069130 con 0x7ff578152330 2024-04-24T05:47:15.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.850+0000 7ff574c00640 1 -- 172.21.15.123:0/4011933550 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff56001c460 con 0x7ff578152330 2024-04-24T05:47:15.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.851+0000 7ff574c00640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff55406c920 0x7ff55406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:15.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.851+0000 7ff574c00640 1 -- 172.21.15.123:0/4011933550 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7ff56008bd60 con 0x7ff578152330 2024-04-24T05:47:15.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.851+0000 7ff577e00640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff55406c920 0x7ff55406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:15.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.852+0000 7ff577e00640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff55406c920 0x7ff55406ede0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff568009a30 tx=0x7ff568008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:15.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:15.855+0000 7ff574c00640 1 -- 172.21.15.123:0/4011933550 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff578069130 con 0x7ff578152330 2024-04-24T05:47:16.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.052+0000 7ff57d600640 1 -- 172.21.15.123:0/4011933550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff578068a20 con 0x7ff578152330 2024-04-24T05:47:16.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.053+0000 7ff574c00640 1 -- 172.21.15.123:0/4011933550 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7ff560059ce0 con 0x7ff578152330 2024-04-24T05:47:16.055 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:16.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.056+0000 7ff57d600640 1 -- 172.21.15.123:0/4011933550 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff55406c920 msgr2=0x7ff55406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:16.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.056+0000 7ff57d600640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff55406c920 0x7ff55406ede0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff568009a30 tx=0x7ff568008040 comp rx=0 tx=0).stop 2024-04-24T05:47:16.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.056+0000 7ff57d600640 1 -- 172.21.15.123:0/4011933550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff578152330 msgr2=0x7ff578153830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:16.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.056+0000 7ff57d600640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff578152330 0x7ff578153830 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff56000c410 tx=0x7ff56000c8d0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.056+0000 7ff57d600640 1 -- 172.21.15.123:0/4011933550 shutdown_connections 2024-04-24T05:47:16.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.056+0000 7ff57d600640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff55406c920 0x7ff55406ede0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.056+0000 7ff57d600640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff578153d70 0x7ff578154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.056+0000 7ff57d600640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff578152330 0x7ff578153830 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.056+0000 7ff57d600640 1 --2- 172.21.15.123:0/4011933550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff55c004d40 0x7ff578154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.056+0000 7ff57d600640 1 -- 172.21.15.123:0/4011933550 >> 172.21.15.123:0/4011933550 conn(0x7ff5780b1cf0 msgr2=0x7ff57815b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:16.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.056+0000 7ff57d600640 1 -- 172.21.15.123:0/4011933550 shutdown_connections 2024-04-24T05:47:16.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.056+0000 7ff57d600640 1 -- 172.21.15.123:0/4011933550 wait complete. 2024-04-24T05:47:16.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T05:47:16.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.275+0000 7fb187e00640 1 Processor -- start 2024-04-24T05:47:16.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.275+0000 7fb187e00640 1 -- start start 2024-04-24T05:47:16.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.277+0000 7fb187e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb188074350 0x7fb18806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:16.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.277+0000 7fb187e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb18806fb50 0x7fb188165610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:16.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.277+0000 7fb187e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb18806f470 con 0x7fb18815a470 2024-04-24T05:47:16.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.277+0000 7fb187e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb18806f710 con 0x7fb188074350 2024-04-24T05:47:16.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.277+0000 7fb187e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb1880b40c0 con 0x7fb18806fb50 2024-04-24T05:47:16.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.277+0000 7fb186a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb18815a470 0x7fb18806e820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56784/0 (socket says 172.21.15.123:56784) 2024-04-24T05:47:16.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.277+0000 7fb186a00640 1 -- 172.21.15.123:0/1316907117 learned_addr learned my addr 172.21.15.123:0/1316907117 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:16.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.277+0000 7fb186000640 1 --2- 172.21.15.123:0/1316907117 >> v2:172.21.15.123:3300/0 conn(0x7fb188074350 0x7fb18806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:16.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.278+0000 7fb185600640 1 -- 172.21.15.123:0/1316907117 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4281858761 0 0) 0x7fb18806f470 con 0x7fb18815a470 2024-04-24T05:47:16.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.278+0000 7fb185600640 1 -- 172.21.15.123:0/1316907117 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb16c003520 con 0x7fb18815a470 2024-04-24T05:47:16.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.278+0000 7fb186000640 1 -- 172.21.15.123:0/1316907117 >> v2:172.21.15.155:3300/0 conn(0x7fb18806fb50 msgr2=0x7fb188165610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:16.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.278+0000 7fb186000640 1 --2- 172.21.15.123:0/1316907117 >> v2:172.21.15.155:3300/0 conn(0x7fb18806fb50 0x7fb188165610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.278+0000 7fb186000640 1 -- 172.21.15.123:0/1316907117 >> v1:172.21.15.123:6789/0 conn(0x7fb18815a470 legacy=0x7fb18806e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:16.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.278+0000 7fb186000640 1 -- 172.21.15.123:0/1316907117 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1880b43a0 con 0x7fb188074350 2024-04-24T05:47:16.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.279+0000 7fb186000640 1 --2- 172.21.15.123:0/1316907117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb188074350 0x7fb18806ef30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb1880cd010 tx=0x7fb17402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822111ee09be49d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:16.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.279+0000 7fb185600640 1 -- 172.21.15.123:0/1316907117 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174037070 con 0x7fb188074350 2024-04-24T05:47:16.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.279+0000 7fb185600640 1 -- 172.21.15.123:0/1316907117 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb174032440 con 0x7fb188074350 2024-04-24T05:47:16.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.279+0000 7fb185600640 1 -- 172.21.15.123:0/1316907117 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17402ae60 con 0x7fb188074350 2024-04-24T05:47:16.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.280+0000 7fb187e00640 1 -- 172.21.15.123:0/1316907117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb188074350 msgr2=0x7fb18806ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:16.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.280+0000 7fb187e00640 1 --2- 172.21.15.123:0/1316907117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb188074350 0x7fb18806ef30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb1880cd010 tx=0x7fb17402a110 comp rx=0 tx=0).stop 2024-04-24T05:47:16.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.280+0000 7fb187e00640 1 -- 172.21.15.123:0/1316907117 shutdown_connections 2024-04-24T05:47:16.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.280+0000 7fb187e00640 1 --2- 172.21.15.123:0/1316907117 >> v2:172.21.15.155:3300/0 conn(0x7fb18806fb50 0x7fb188165610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.280+0000 7fb187e00640 1 --2- 172.21.15.123:0/1316907117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb188074350 0x7fb18806ef30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.280+0000 7fb187e00640 1 -- 172.21.15.123:0/1316907117 >> 172.21.15.123:0/1316907117 conn(0x7fb1880d0590 msgr2=0x7fb1880d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:16.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.280+0000 7fb187e00640 1 -- 172.21.15.123:0/1316907117 shutdown_connections 2024-04-24T05:47:16.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.280+0000 7fb187e00640 1 -- 172.21.15.123:0/1316907117 wait complete. 2024-04-24T05:47:16.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.282+0000 7fb187e00640 1 Processor -- start 2024-04-24T05:47:16.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.282+0000 7fb187e00640 1 -- start start 2024-04-24T05:47:16.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.283+0000 7fb187e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb18806fb50 0x7fb1880a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:16.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.283+0000 7fb187e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb188074350 0x7fb1880aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:16.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.283+0000 7fb187e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb18815a470 0x7fb1881f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:16.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.283+0000 7fb187e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb18806cba0 con 0x7fb18806fb50 2024-04-24T05:47:16.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.283+0000 7fb187e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb1880b4960 con 0x7fb188074350 2024-04-24T05:47:16.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.283+0000 7fb187e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb18806cd20 con 0x7fb18815a470 2024-04-24T05:47:16.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.283+0000 7fb186a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb18806fb50 0x7fb1880a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:16.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.283+0000 7fb186a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb18806fb50 0x7fb1880a9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59808/0 (socket says 172.21.15.123:59808) 2024-04-24T05:47:16.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.283+0000 7fb186a00640 1 -- 172.21.15.123:0/1419499036 learned_addr learned my addr 172.21.15.123:0/1419499036 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:16.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.284+0000 7fb186a00640 1 -- 172.21.15.123:0/1419499036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb18815a470 msgr2=0x7fb1881f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:16.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.284+0000 7fb186a00640 1 --2- 172.21.15.123:0/1419499036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb18815a470 0x7fb1881f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.284+0000 7fb186000640 1 --2- 172.21.15.123:0/1419499036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb188074350 0x7fb1880aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:16.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.284+0000 7fb186a00640 1 -- 172.21.15.123:0/1419499036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb188074350 msgr2=0x7fb1880aaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:16.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.284+0000 7fb186a00640 1 --2- 172.21.15.123:0/1419499036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb188074350 0x7fb1880aaf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.284+0000 7fb186a00640 1 -- 172.21.15.123:0/1419499036 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1880aa5c0 con 0x7fb18806fb50 2024-04-24T05:47:16.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.284+0000 7fb186a00640 1 --2- 172.21.15.123:0/1419499036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb18806fb50 0x7fb1880a9e50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb17800c390 tx=0x7fb17800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:16.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.284+0000 7fb17f400640 1 -- 172.21.15.123:0/1419499036 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178017070 con 0x7fb18806fb50 2024-04-24T05:47:16.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.284+0000 7fb17f400640 1 -- 172.21.15.123:0/1419499036 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb178005ce0 con 0x7fb18806fb50 2024-04-24T05:47:16.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.284+0000 7fb187e00640 1 -- 172.21.15.123:0/1419499036 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb188153270 con 0x7fb18806fb50 2024-04-24T05:47:16.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.284+0000 7fb17f400640 1 -- 172.21.15.123:0/1419499036 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1780072f0 con 0x7fb18806fb50 2024-04-24T05:47:16.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.284+0000 7fb187e00640 1 -- 172.21.15.123:0/1419499036 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb188153830 con 0x7fb18806fb50 2024-04-24T05:47:16.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.286+0000 7fb187e00640 1 -- 172.21.15.123:0/1419499036 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb188075b70 con 0x7fb18806fb50 2024-04-24T05:47:16.288 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.286+0000 7fb17f400640 1 -- 172.21.15.123:0/1419499036 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb17801a070 con 0x7fb18806fb50 2024-04-24T05:47:16.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.288+0000 7fb17f400640 1 --2- 172.21.15.123:0/1419499036 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb16806c8f0 0x7fb16806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:16.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.288+0000 7fb17f400640 1 -- 172.21.15.123:0/1419499036 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fb17808b420 con 0x7fb18806fb50 2024-04-24T05:47:16.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.288+0000 7fb186000640 1 --2- 172.21.15.123:0/1419499036 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb16806c8f0 0x7fb16806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:16.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.289+0000 7fb186000640 1 --2- 172.21.15.123:0/1419499036 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb16806c8f0 0x7fb16806edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb174034000 tx=0x7fb17402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:16.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.292+0000 7fb17f400640 1 -- 172.21.15.123:0/1419499036 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb1780593a0 con 0x7fb18806fb50 2024-04-24T05:47:16.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.491+0000 7fb187e00640 1 -- 172.21.15.123:0/1419499036 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb18806e1a0 con 0x7fb18806fb50 2024-04-24T05:47:16.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.492+0000 7fb17f400640 1 -- 172.21.15.123:0/1419499036 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fb178058ca0 con 0x7fb18806fb50 2024-04-24T05:47:16.493 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-24T05:47:16.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.494+0000 7fb187e00640 1 -- 172.21.15.123:0/1419499036 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb16806c8f0 msgr2=0x7fb16806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:16.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.494+0000 7fb187e00640 1 --2- 172.21.15.123:0/1419499036 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb16806c8f0 0x7fb16806edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb174034000 tx=0x7fb17402e000 comp rx=0 tx=0).stop 2024-04-24T05:47:16.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.494+0000 7fb187e00640 1 -- 172.21.15.123:0/1419499036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb18806fb50 msgr2=0x7fb1880a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:16.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.494+0000 7fb187e00640 1 --2- 172.21.15.123:0/1419499036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb18806fb50 0x7fb1880a9e50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb17800c390 tx=0x7fb17800c850 comp rx=0 tx=0).stop 2024-04-24T05:47:16.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.494+0000 7fb187e00640 1 -- 172.21.15.123:0/1419499036 shutdown_connections 2024-04-24T05:47:16.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.494+0000 7fb187e00640 1 --2- 172.21.15.123:0/1419499036 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb16806c8f0 0x7fb16806edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.494+0000 7fb187e00640 1 --2- 172.21.15.123:0/1419499036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb18815a470 0x7fb1881f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.494+0000 7fb187e00640 1 --2- 172.21.15.123:0/1419499036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb188074350 0x7fb1880aaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.494+0000 7fb187e00640 1 --2- 172.21.15.123:0/1419499036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb18806fb50 0x7fb1880a9e50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.494+0000 7fb187e00640 1 -- 172.21.15.123:0/1419499036 >> 172.21.15.123:0/1419499036 conn(0x7fb1880d0590 msgr2=0x7fb1880b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:16.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.495+0000 7fb187e00640 1 -- 172.21.15.123:0/1419499036 shutdown_connections 2024-04-24T05:47:16.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.495+0000 7fb187e00640 1 -- 172.21.15.123:0/1419499036 wait complete. 2024-04-24T05:47:16.514 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:47:16.514 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T05:47:16.562 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:16.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.795+0000 7f4811c00640 1 Processor -- start 2024-04-24T05:47:16.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.795+0000 7f4811c00640 1 -- start start 2024-04-24T05:47:16.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.796+0000 7f4811c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f480c0adee0 0x7f480c0acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:16.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.796+0000 7f4811c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f480c0ae2e0 0x7f480c0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:16.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.796+0000 7f4811c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f480c160120 con 0x7f480c15c5d0 2024-04-24T05:47:16.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.796+0000 7f4811c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f480c160470 con 0x7f480c0ae2e0 2024-04-24T05:47:16.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.796+0000 7f4811c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f480c161010 con 0x7f480c0adee0 2024-04-24T05:47:16.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.797+0000 7f4811200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f480c0ae2e0 0x7f480c0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:16.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.797+0000 7f4811200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f480c0ae2e0 0x7f480c0ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59820/0 (socket says 172.21.15.123:59820) 2024-04-24T05:47:16.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.797+0000 7f4811200640 1 -- 172.21.15.123:0/1823531671 learned_addr learned my addr 172.21.15.123:0/1823531671 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:16.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.797+0000 7f4803400640 1 --2- 172.21.15.123:0/1823531671 >> v2:172.21.15.155:3300/0 conn(0x7f480c0adee0 0x7f480c0acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:16.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.797+0000 7f4811200640 1 -- 172.21.15.123:0/1823531671 >> v2:172.21.15.155:3300/0 conn(0x7f480c0adee0 msgr2=0x7f480c0acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:16.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.797+0000 7f4811200640 1 --2- 172.21.15.123:0/1823531671 >> v2:172.21.15.155:3300/0 conn(0x7f480c0adee0 0x7f480c0acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.797+0000 7f4811200640 1 -- 172.21.15.123:0/1823531671 >> v1:172.21.15.155:6789/0 conn(0x7f480c15c5d0 legacy=0x7f480c160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:16.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.797+0000 7f4811200640 1 -- 172.21.15.123:0/1823531671 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c06dd80 con 0x7f480c0ae2e0 2024-04-24T05:47:16.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.798+0000 7f4811200640 1 --2- 172.21.15.123:0/1823531671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f480c0ae2e0 0x7f480c0ad480 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f480c160080 tx=0x7f480402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7378c5f22e69db5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:16.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.799+0000 7f480b400640 1 -- 172.21.15.123:0/1823531671 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804036070 con 0x7f480c0ae2e0 2024-04-24T05:47:16.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.799+0000 7f480b400640 1 -- 172.21.15.123:0/1823531671 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4804031600 con 0x7f480c0ae2e0 2024-04-24T05:47:16.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.799+0000 7f480b400640 1 -- 172.21.15.123:0/1823531671 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040319d0 con 0x7f480c0ae2e0 2024-04-24T05:47:16.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.799+0000 7f4811c00640 1 -- 172.21.15.123:0/1823531671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f480c0ae2e0 msgr2=0x7f480c0ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:16.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.799+0000 7f4811c00640 1 --2- 172.21.15.123:0/1823531671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f480c0ae2e0 0x7f480c0ad480 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f480c160080 tx=0x7f480402a6f0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.799+0000 7f4811c00640 1 -- 172.21.15.123:0/1823531671 shutdown_connections 2024-04-24T05:47:16.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.799+0000 7f4811c00640 1 --2- 172.21.15.123:0/1823531671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f480c0ae2e0 0x7f480c0ad480 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.799+0000 7f4811c00640 1 --2- 172.21.15.123:0/1823531671 >> v2:172.21.15.155:3300/0 conn(0x7f480c0adee0 0x7f480c0acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.799+0000 7f4811c00640 1 -- 172.21.15.123:0/1823531671 >> 172.21.15.123:0/1823531671 conn(0x7f480c0d0070 msgr2=0x7f480c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:16.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.799+0000 7f4811c00640 1 -- 172.21.15.123:0/1823531671 shutdown_connections 2024-04-24T05:47:16.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.799+0000 7f4811c00640 1 -- 172.21.15.123:0/1823531671 wait complete. 2024-04-24T05:47:16.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.801+0000 7f4811c00640 1 Processor -- start 2024-04-24T05:47:16.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.801+0000 7f4811c00640 1 -- start start 2024-04-24T05:47:16.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.802+0000 7f4811c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f480c0adee0 0x7f480c1527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:16.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.802+0000 7f4811c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480c0ae2e0 0x7f480c072bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:16.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.802+0000 7f4811c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f480c15c5d0 0x7f480c073110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:16.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.802+0000 7f4811c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f480c0d0b60 con 0x7f480c0adee0 2024-04-24T05:47:16.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.802+0000 7f4811c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f480c15c9b0 con 0x7f480c15c5d0 2024-04-24T05:47:16.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.802+0000 7f4811c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f480c0d0ce0 con 0x7f480c0ae2e0 2024-04-24T05:47:16.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.802+0000 7f480be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f480c0adee0 0x7f480c1527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:16.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.802+0000 7f480be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f480c0adee0 0x7f480c1527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59832/0 (socket says 172.21.15.123:59832) 2024-04-24T05:47:16.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.802+0000 7f480be00640 1 -- 172.21.15.123:0/521299742 learned_addr learned my addr 172.21.15.123:0/521299742 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:16.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.803+0000 7f480be00640 1 -- 172.21.15.123:0/521299742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480c0ae2e0 msgr2=0x7f480c072bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:16.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.803+0000 7f480be00640 1 --2- 172.21.15.123:0/521299742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480c0ae2e0 0x7f480c072bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.803+0000 7f480be00640 1 -- 172.21.15.123:0/521299742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f480c15c5d0 msgr2=0x7f480c073110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:16.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.803+0000 7f480be00640 1 --2- 172.21.15.123:0/521299742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f480c15c5d0 0x7f480c073110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:16.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.803+0000 7f480be00640 1 -- 172.21.15.123:0/521299742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c1f89c0 con 0x7f480c0adee0 2024-04-24T05:47:16.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.803+0000 7f480be00640 1 --2- 172.21.15.123:0/521299742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f480c0adee0 0x7f480c1527f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00c430 tx=0x7f47fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:16.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.803+0000 7f4808c00640 1 -- 172.21.15.123:0/521299742 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc011790 con 0x7f480c0adee0 2024-04-24T05:47:16.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.803+0000 7f4808c00640 1 -- 172.21.15.123:0/521299742 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47fc017070 con 0x7f480c0adee0 2024-04-24T05:47:16.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.803+0000 7f4811c00640 1 -- 172.21.15.123:0/521299742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480c1f7c00 con 0x7f480c0adee0 2024-04-24T05:47:16.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.803+0000 7f4808c00640 1 -- 172.21.15.123:0/521299742 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc011bb0 con 0x7f480c0adee0 2024-04-24T05:47:16.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.803+0000 7f4811c00640 1 -- 172.21.15.123:0/521299742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480c1f81c0 con 0x7f480c0adee0 2024-04-24T05:47:16.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.805+0000 7f4811c00640 1 -- 172.21.15.123:0/521299742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f480c1530a0 con 0x7f480c0adee0 2024-04-24T05:47:16.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.805+0000 7f4808c00640 1 -- 172.21.15.123:0/521299742 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47fc01c460 con 0x7f480c0adee0 2024-04-24T05:47:16.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.806+0000 7f4808c00640 1 --2- 172.21.15.123:0/521299742 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f47e806c8f0 0x7f47e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:16.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.806+0000 7f4808c00640 1 -- 172.21.15.123:0/521299742 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f47fc08bef0 con 0x7f480c0adee0 2024-04-24T05:47:16.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.806+0000 7f4803400640 1 --2- 172.21.15.123:0/521299742 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f47e806c8f0 0x7f47e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:16.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.807+0000 7f4803400640 1 --2- 172.21.15.123:0/521299742 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f47e806c8f0 0x7f47e806edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47f4006fd0 tx=0x7f47f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:16.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.811+0000 7f4808c00640 1 -- 172.21.15.123:0/521299742 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f47fc059e70 con 0x7f480c0adee0 2024-04-24T05:47:16.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.996+0000 7f4811c00640 1 -- 172.21.15.123:0/521299742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f480c15c140 con 0x7f480c0adee0 2024-04-24T05:47:16.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:16.998+0000 7f4808c00640 1 -- 172.21.15.123:0/521299742 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7f47fc059770 con 0x7f480c0adee0 2024-04-24T05:47:17.000 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:17.000 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.000+0000 7f4811c00640 1 -- 172.21.15.123:0/521299742 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f47e806c8f0 msgr2=0x7f47e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.000+0000 7f4811c00640 1 --2- 172.21.15.123:0/521299742 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f47e806c8f0 0x7f47e806edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47f4006fd0 tx=0x7f47f4008040 comp rx=0 tx=0).stop 2024-04-24T05:47:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.000+0000 7f4811c00640 1 -- 172.21.15.123:0/521299742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f480c0adee0 msgr2=0x7f480c1527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.000+0000 7f4811c00640 1 --2- 172.21.15.123:0/521299742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f480c0adee0 0x7f480c1527f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00c430 tx=0x7f47fc00c8f0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.000+0000 7f4811c00640 1 -- 172.21.15.123:0/521299742 shutdown_connections 2024-04-24T05:47:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.000+0000 7f4811c00640 1 --2- 172.21.15.123:0/521299742 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f47e806c8f0 0x7f47e806edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.000+0000 7f4811c00640 1 --2- 172.21.15.123:0/521299742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f480c15c5d0 0x7f480c073110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.000+0000 7f4811c00640 1 --2- 172.21.15.123:0/521299742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f480c0ae2e0 0x7f480c072bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.000+0000 7f4811c00640 1 --2- 172.21.15.123:0/521299742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f480c0adee0 0x7f480c1527f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.000+0000 7f4811c00640 1 -- 172.21.15.123:0/521299742 >> 172.21.15.123:0/521299742 conn(0x7f480c0d0070 msgr2=0x7f480c0b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.001+0000 7f4811c00640 1 -- 172.21.15.123:0/521299742 shutdown_connections 2024-04-24T05:47:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.001+0000 7f4811c00640 1 -- 172.21.15.123:0/521299742 wait complete. 2024-04-24T05:47:17.020 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:47:17.021 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:47:17.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.222+0000 7fcec9c00640 1 Processor -- start 2024-04-24T05:47:17.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.222+0000 7fcec9c00640 1 -- start start 2024-04-24T05:47:17.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.224+0000 7fcec9c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcec406dcb0 0x7fcec406e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:17.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.224+0000 7fcec9c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcec406e960 0x7fcec406ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:17.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.224+0000 7fcec9c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcec406bb30 con 0x7fcec4158e00 2024-04-24T05:47:17.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.224+0000 7fcec9c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fcec406f590 con 0x7fcec406e960 2024-04-24T05:47:17.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.224+0000 7fcec9c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcec406c460 con 0x7fcec406dcb0 2024-04-24T05:47:17.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.224+0000 7fcec3400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcec406e960 0x7fcec406ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:17.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.224+0000 7fcec2a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fcec4158e00 0x7fcec406c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56786/0 (socket says 172.21.15.123:56786) 2024-04-24T05:47:17.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.224+0000 7fcec2a00640 1 -- 172.21.15.123:0/3963603868 learned_addr learned my addr 172.21.15.123:0/3963603868 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:17.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.224+0000 7fceba000640 1 --2- 172.21.15.123:0/3963603868 >> v2:172.21.15.155:3300/0 conn(0x7fcec406dcb0 0x7fcec406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:17.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.225+0000 7fcec3400640 1 -- 172.21.15.123:0/3963603868 >> v2:172.21.15.155:3300/0 conn(0x7fcec406dcb0 msgr2=0x7fcec406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:17.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.225+0000 7fcec3400640 1 --2- 172.21.15.123:0/3963603868 >> v2:172.21.15.155:3300/0 conn(0x7fcec406dcb0 0x7fcec406e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.225+0000 7fcec3400640 1 -- 172.21.15.123:0/3963603868 >> v1:172.21.15.123:6789/0 conn(0x7fcec4158e00 legacy=0x7fcec406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:17.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.225+0000 7fcec3400640 1 -- 172.21.15.123:0/3963603868 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec406be10 con 0x7fcec406e960 2024-04-24T05:47:17.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.225+0000 7fcec2000640 1 -- 172.21.15.123:0/3963603868 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1454765762 0 0) 0x7fcec406bb30 con 0x7fcec4158e00 2024-04-24T05:47:17.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.226+0000 7fcec3400640 1 --2- 172.21.15.123:0/3963603868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec406e960 0x7fcec406ed70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcec406f710 tx=0x7fcebc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba69b5effb0265 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:17.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.226+0000 7fcec2000640 1 -- 172.21.15.123:0/3963603868 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc031600 con 0x7fcec406e960 2024-04-24T05:47:17.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.226+0000 7fcec2000640 1 -- 172.21.15.123:0/3963603868 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcebc0317a0 con 0x7fcec406e960 2024-04-24T05:47:17.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.226+0000 7fcec2000640 1 -- 172.21.15.123:0/3963603868 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc031b70 con 0x7fcec406e960 2024-04-24T05:47:17.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.227+0000 7fcec9c00640 1 -- 172.21.15.123:0/3963603868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec406e960 msgr2=0x7fcec406ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:17.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.227+0000 7fcec9c00640 1 --2- 172.21.15.123:0/3963603868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec406e960 0x7fcec406ed70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcec406f710 tx=0x7fcebc02a5d0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.227+0000 7fcec9c00640 1 -- 172.21.15.123:0/3963603868 shutdown_connections 2024-04-24T05:47:17.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.227+0000 7fcec9c00640 1 --2- 172.21.15.123:0/3963603868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec406e960 0x7fcec406ed70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.227+0000 7fcec9c00640 1 --2- 172.21.15.123:0/3963603868 >> v2:172.21.15.155:3300/0 conn(0x7fcec406dcb0 0x7fcec406e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.227+0000 7fcec9c00640 1 -- 172.21.15.123:0/3963603868 >> 172.21.15.123:0/3963603868 conn(0x7fcec40b51f0 msgr2=0x7fcec40b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:17.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.227+0000 7fcec9c00640 1 -- 172.21.15.123:0/3963603868 shutdown_connections 2024-04-24T05:47:17.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.227+0000 7fcec9c00640 1 -- 172.21.15.123:0/3963603868 wait complete. 2024-04-24T05:47:17.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.228+0000 7fcec9c00640 1 Processor -- start 2024-04-24T05:47:17.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.229+0000 7fcec9c00640 1 -- start start 2024-04-24T05:47:17.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec9c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec406dcb0 0x7fcec40999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:17.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec9c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcec406e960 0x7fcec40af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:17.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec9c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec4158e00 0x7fcec409a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:17.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec9c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcec406d000 con 0x7fcec4158e00 2024-04-24T05:47:17.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec9c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcec406f2b0 con 0x7fcec406dcb0 2024-04-24T05:47:17.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec9c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcec41591e0 con 0x7fcec406e960 2024-04-24T05:47:17.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec4158e00 0x7fcec409a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:17.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec4158e00 0x7fcec409a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59852/0 (socket says 172.21.15.123:59852) 2024-04-24T05:47:17.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec3400640 1 -- 172.21.15.123:0/906155232 learned_addr learned my addr 172.21.15.123:0/906155232 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:17.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec2a00640 1 --2- 172.21.15.123:0/906155232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec406dcb0 0x7fcec40999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:17.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fceba000640 1 --2- 172.21.15.123:0/906155232 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcec406e960 0x7fcec40af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:17.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec3400640 1 -- 172.21.15.123:0/906155232 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcec406e960 msgr2=0x7fcec40af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:17.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec3400640 1 --2- 172.21.15.123:0/906155232 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcec406e960 0x7fcec40af310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec3400640 1 -- 172.21.15.123:0/906155232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec406dcb0 msgr2=0x7fcec40999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:17.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec3400640 1 --2- 172.21.15.123:0/906155232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec406dcb0 0x7fcec40999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.230+0000 7fcec3400640 1 -- 172.21.15.123:0/906155232 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec40af850 con 0x7fcec4158e00 2024-04-24T05:47:17.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.231+0000 7fcec3400640 1 --2- 172.21.15.123:0/906155232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec4158e00 0x7fcec409a7d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcebc02abc0 tx=0x7fcebc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:17.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.231+0000 7fcebb400640 1 -- 172.21.15.123:0/906155232 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc03f070 con 0x7fcec4158e00 2024-04-24T05:47:17.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.231+0000 7fcebb400640 1 -- 172.21.15.123:0/906155232 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcebc030b60 con 0x7fcec4158e00 2024-04-24T05:47:17.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.231+0000 7fcebb400640 1 -- 172.21.15.123:0/906155232 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc009450 con 0x7fcec4158e00 2024-04-24T05:47:17.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.231+0000 7fcec9c00640 1 -- 172.21.15.123:0/906155232 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec409e1b0 con 0x7fcec4158e00 2024-04-24T05:47:17.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.231+0000 7fcec9c00640 1 -- 172.21.15.123:0/906155232 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec409d910 con 0x7fcec4158e00 2024-04-24T05:47:17.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.232+0000 7fcec9c00640 1 -- 172.21.15.123:0/906155232 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec4068a20 con 0x7fcec4158e00 2024-04-24T05:47:17.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.233+0000 7fcebb400640 1 -- 172.21.15.123:0/906155232 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcebc03b410 con 0x7fcec4158e00 2024-04-24T05:47:17.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.235+0000 7fcebb400640 1 --2- 172.21.15.123:0/906155232 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fcea006c8f0 0x7fcea006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:17.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.235+0000 7fcebb400640 1 -- 172.21.15.123:0/906155232 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fcebc0ad580 con 0x7fcec4158e00 2024-04-24T05:47:17.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.235+0000 7fcec2a00640 1 --2- 172.21.15.123:0/906155232 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fcea006c8f0 0x7fcea006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:17.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.236+0000 7fcec2a00640 1 --2- 172.21.15.123:0/906155232 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fcea006c8f0 0x7fcea006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fceb4000f60 tx=0x7fceb4007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:17.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.239+0000 7fcebb400640 1 -- 172.21.15.123:0/906155232 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcec40af850 con 0x7fcec4158e00 2024-04-24T05:47:17.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.437+0000 7fcec9c00640 1 -- 172.21.15.123:0/906155232 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcec4067be0 con 0x7fcec4158e00 2024-04-24T05:47:17.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.438+0000 7fcebb400640 1 -- 172.21.15.123:0/906155232 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fcebc07af40 con 0x7fcec4158e00 2024-04-24T05:47:17.440 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:47:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.441+0000 7fcec9c00640 1 -- 172.21.15.123:0/906155232 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fcea006c8f0 msgr2=0x7fcea006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.441+0000 7fcec9c00640 1 --2- 172.21.15.123:0/906155232 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fcea006c8f0 0x7fcea006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fceb4000f60 tx=0x7fceb4007c00 comp rx=0 tx=0).stop 2024-04-24T05:47:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.441+0000 7fcec9c00640 1 -- 172.21.15.123:0/906155232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec4158e00 msgr2=0x7fcec409a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.441+0000 7fcec9c00640 1 --2- 172.21.15.123:0/906155232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec4158e00 0x7fcec409a7d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcebc02abc0 tx=0x7fcebc004a40 comp rx=0 tx=0).stop 2024-04-24T05:47:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.441+0000 7fcec9c00640 1 -- 172.21.15.123:0/906155232 shutdown_connections 2024-04-24T05:47:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.441+0000 7fcec9c00640 1 --2- 172.21.15.123:0/906155232 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fcea006c8f0 0x7fcea006edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.441+0000 7fcec9c00640 1 --2- 172.21.15.123:0/906155232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcec4158e00 0x7fcec409a7d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.441+0000 7fcec9c00640 1 --2- 172.21.15.123:0/906155232 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcec406e960 0x7fcec40af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.441+0000 7fcec9c00640 1 --2- 172.21.15.123:0/906155232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec406dcb0 0x7fcec40999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.441+0000 7fcec9c00640 1 -- 172.21.15.123:0/906155232 >> 172.21.15.123:0/906155232 conn(0x7fcec40b51f0 msgr2=0x7fcec4073420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:17.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.441+0000 7fcec9c00640 1 -- 172.21.15.123:0/906155232 shutdown_connections 2024-04-24T05:47:17.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.442+0000 7fcec9c00640 1 -- 172.21.15.123:0/906155232 wait complete. 2024-04-24T05:47:17.463 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:47:17.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.650+0000 7fe8a4400640 1 Processor -- start 2024-04-24T05:47:17.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.650+0000 7fe8a4400640 1 -- start start 2024-04-24T05:47:17.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.652+0000 7fe8a4400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe89c071c50 0x7fe89c0ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:17.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.652+0000 7fe8a4400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe89c0cd0f0 0x7fe89c0d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:17.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.652+0000 7fe8a4400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe89c0cf170 con 0x7fe89c06c510 2024-04-24T05:47:17.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.652+0000 7fe8a4400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe89c0cd520 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.652+0000 7fe8a4400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe89c0cf450 con 0x7fe89c071c50 2024-04-24T05:47:17.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.652+0000 7fe8a1a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe89c06c510 0x7fe89c06c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56798/0 (socket says 172.21.15.123:56798) 2024-04-24T05:47:17.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.652+0000 7fe8a1a00640 1 -- 172.21.15.123:0/1681609455 learned_addr learned my addr 172.21.15.123:0/1681609455 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:17.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.652+0000 7fe8a2400640 1 --2- 172.21.15.123:0/1681609455 >> v2:172.21.15.123:3300/0 conn(0x7fe89c0cd0f0 0x7fe89c0d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:17.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.652+0000 7fe8a1000640 1 --2- 172.21.15.123:0/1681609455 >> v2:172.21.15.155:3300/0 conn(0x7fe89c071c50 0x7fe89c0ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:17.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.652+0000 7fe8a2400640 1 -- 172.21.15.123:0/1681609455 >> v2:172.21.15.155:3300/0 conn(0x7fe89c071c50 msgr2=0x7fe89c0ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:17.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.652+0000 7fe8a2400640 1 --2- 172.21.15.123:0/1681609455 >> v2:172.21.15.155:3300/0 conn(0x7fe89c071c50 0x7fe89c0ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.653+0000 7fe8a2400640 1 -- 172.21.15.123:0/1681609455 >> v1:172.21.15.123:6789/0 conn(0x7fe89c06c510 legacy=0x7fe89c06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:17.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.653+0000 7fe8a2400640 1 -- 172.21.15.123:0/1681609455 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c0cf730 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.653+0000 7fe8a1000640 1 --2- 172.21.15.123:0/1681609455 >> v2:172.21.15.155:3300/0 conn(0x7fe89c071c50 0x7fe89c0ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:17.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.654+0000 7fe8a2400640 1 --2- 172.21.15.123:0/1681609455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe89c0cd0f0 0x7fe89c0d0210 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe89c070160 tx=0x7fe89402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c56cf531d6392c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:17.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.654+0000 7fe89be00640 1 -- 172.21.15.123:0/1681609455 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894037070 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.654+0000 7fe89be00640 1 -- 172.21.15.123:0/1681609455 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe894032440 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.654+0000 7fe89be00640 1 -- 172.21.15.123:0/1681609455 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894032810 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.654+0000 7fe8a4400640 1 -- 172.21.15.123:0/1681609455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe89c0cd0f0 msgr2=0x7fe89c0d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:17.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.654+0000 7fe8a4400640 1 --2- 172.21.15.123:0/1681609455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe89c0cd0f0 0x7fe89c0d0210 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe89c070160 tx=0x7fe89402a1a0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.655+0000 7fe8a4400640 1 -- 172.21.15.123:0/1681609455 shutdown_connections 2024-04-24T05:47:17.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.655+0000 7fe8a4400640 1 --2- 172.21.15.123:0/1681609455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe89c0cd0f0 0x7fe89c0d0210 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.655+0000 7fe8a4400640 1 --2- 172.21.15.123:0/1681609455 >> v2:172.21.15.155:3300/0 conn(0x7fe89c071c50 0x7fe89c0ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.655+0000 7fe8a4400640 1 -- 172.21.15.123:0/1681609455 >> 172.21.15.123:0/1681609455 conn(0x7fe89c0b51f0 msgr2=0x7fe89c0b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:17.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.655+0000 7fe8a4400640 1 -- 172.21.15.123:0/1681609455 shutdown_connections 2024-04-24T05:47:17.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.655+0000 7fe8a4400640 1 -- 172.21.15.123:0/1681609455 wait complete. 2024-04-24T05:47:17.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.657+0000 7fe8a4400640 1 Processor -- start 2024-04-24T05:47:17.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.657+0000 7fe8a4400640 1 -- start start 2024-04-24T05:47:17.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.658+0000 7fe8a4400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe89c06c510 0x7fe89c158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:17.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.658+0000 7fe8a4400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe89c071c50 0x7fe89c0c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:17.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.658+0000 7fe8a4400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe89c0cd0f0 0x7fe89c158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:17.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.658+0000 7fe8a4400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c06d000 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.658+0000 7fe8a4400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c0cfcf0 con 0x7fe89c06c510 2024-04-24T05:47:17.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.658+0000 7fe8a4400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe89c06d180 con 0x7fe89c071c50 2024-04-24T05:47:17.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.659+0000 7fe8a2400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe89c0cd0f0 0x7fe89c158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:17.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.659+0000 7fe8a2400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe89c0cd0f0 0x7fe89c158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59860/0 (socket says 172.21.15.123:59860) 2024-04-24T05:47:17.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.659+0000 7fe8a2400640 1 -- 172.21.15.123:0/1015791379 learned_addr learned my addr 172.21.15.123:0/1015791379 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:17.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.659+0000 7fe8a2400640 1 -- 172.21.15.123:0/1015791379 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe89c071c50 msgr2=0x7fe89c0c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:17.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.659+0000 7fe8a2400640 1 --2- 172.21.15.123:0/1015791379 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe89c071c50 0x7fe89c0c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.659+0000 7fe8a2400640 1 -- 172.21.15.123:0/1015791379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe89c06c510 msgr2=0x7fe89c158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:17.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.659+0000 7fe8a2400640 1 --2- 172.21.15.123:0/1015791379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe89c06c510 0x7fe89c158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.659+0000 7fe8a2400640 1 -- 172.21.15.123:0/1015791379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c0c79a0 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.659+0000 7fe8a2400640 1 --2- 172.21.15.123:0/1015791379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe89c0cd0f0 0x7fe89c158590 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe894035040 tx=0x7fe8940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:17.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.660+0000 7fe899600640 1 -- 172.21.15.123:0/1015791379 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894037040 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.660+0000 7fe899600640 1 -- 172.21.15.123:0/1015791379 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe894042070 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.660+0000 7fe899600640 1 -- 172.21.15.123:0/1015791379 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894030b30 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.660+0000 7fe8a4400640 1 -- 172.21.15.123:0/1015791379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89c0c7c80 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.660+0000 7fe8a4400640 1 -- 172.21.15.123:0/1015791379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe89c0c8240 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.661+0000 7fe8a4400640 1 -- 172.21.15.123:0/1015791379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe89c068a20 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.662+0000 7fe899600640 1 -- 172.21.15.123:0/1015791379 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe894030cd0 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.663+0000 7fe899600640 1 --2- 172.21.15.123:0/1015791379 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe88006c8f0 0x7fe88006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.663+0000 7fe899600640 1 -- 172.21.15.123:0/1015791379 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fe8940acaf0 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.663+0000 7fe8a1a00640 1 --2- 172.21.15.123:0/1015791379 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe88006c8f0 0x7fe88006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:17.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.664+0000 7fe8a1a00640 1 --2- 172.21.15.123:0/1015791379 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe88006c8f0 0x7fe88006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe890000f60 tx=0x7fe890007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:17.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.668+0000 7fe899600640 1 -- 172.21.15.123:0/1015791379 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe894045360 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.865+0000 7fe8a4400640 1 -- 172.21.15.123:0/1015791379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe89c06f400 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.866+0000 7fe899600640 1 -- 172.21.15.123:0/1015791379 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fe89407aba0 con 0x7fe89c0cd0f0 2024-04-24T05:47:17.868 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:17.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.869+0000 7fe8a4400640 1 -- 172.21.15.123:0/1015791379 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe88006c8f0 msgr2=0x7fe88006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:17.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.869+0000 7fe8a4400640 1 --2- 172.21.15.123:0/1015791379 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe88006c8f0 0x7fe88006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe890000f60 tx=0x7fe890007a80 comp rx=0 tx=0).stop 2024-04-24T05:47:17.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.869+0000 7fe8a4400640 1 -- 172.21.15.123:0/1015791379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe89c0cd0f0 msgr2=0x7fe89c158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:17.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.869+0000 7fe8a4400640 1 --2- 172.21.15.123:0/1015791379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe89c0cd0f0 0x7fe89c158590 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe894035040 tx=0x7fe8940046c0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.869+0000 7fe8a4400640 1 -- 172.21.15.123:0/1015791379 shutdown_connections 2024-04-24T05:47:17.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.869+0000 7fe8a4400640 1 --2- 172.21.15.123:0/1015791379 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe88006c8f0 0x7fe88006edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.869+0000 7fe8a4400640 1 --2- 172.21.15.123:0/1015791379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe89c0cd0f0 0x7fe89c158590 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.869+0000 7fe8a4400640 1 --2- 172.21.15.123:0/1015791379 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe89c071c50 0x7fe89c0c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.869+0000 7fe8a4400640 1 --2- 172.21.15.123:0/1015791379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe89c06c510 0x7fe89c158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:17.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.869+0000 7fe8a4400640 1 -- 172.21.15.123:0/1015791379 >> 172.21.15.123:0/1015791379 conn(0x7fe89c0b51f0 msgr2=0x7fe89c15be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:17.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.869+0000 7fe8a4400640 1 -- 172.21.15.123:0/1015791379 shutdown_connections 2024-04-24T05:47:17.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:17.869+0000 7fe8a4400640 1 -- 172.21.15.123:0/1015791379 wait complete. 2024-04-24T05:47:17.888 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:47:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.081+0000 7f04dfe00640 1 Processor -- start 2024-04-24T05:47:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.082+0000 7f04dfe00640 1 -- start start 2024-04-24T05:47:18.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.083+0000 7f04dfe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f04e0072290 0x7f04e006c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.083+0000 7f04dfe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f04e006f800 0x7f04e006c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.083+0000 7f04dfe00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04e006eb20 con 0x7f04e0162a30 2024-04-24T05:47:18.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.083+0000 7f04dfe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f04e006ee00 con 0x7f04e006f800 2024-04-24T05:47:18.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.083+0000 7f04dfe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f04e006bc10 con 0x7f04e0072290 2024-04-24T05:47:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.084+0000 7f04df400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f04e006f800 0x7f04e006c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.084+0000 7f04df400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f04e006f800 0x7f04e006c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59870/0 (socket says 172.21.15.123:59870) 2024-04-24T05:47:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.084+0000 7f04df400640 1 -- 172.21.15.123:0/780573528 learned_addr learned my addr 172.21.15.123:0/780573528 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.084+0000 7f04de000640 1 --2- 172.21.15.123:0/780573528 >> v2:172.21.15.155:3300/0 conn(0x7f04e0072290 0x7f04e006c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.084+0000 7f04df400640 1 -- 172.21.15.123:0/780573528 >> v2:172.21.15.155:3300/0 conn(0x7f04e0072290 msgr2=0x7f04e006c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.084+0000 7f04df400640 1 --2- 172.21.15.123:0/780573528 >> v2:172.21.15.155:3300/0 conn(0x7f04e0072290 0x7f04e006c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.084+0000 7f04df400640 1 -- 172.21.15.123:0/780573528 >> v1:172.21.15.155:6789/0 conn(0x7f04e0162a30 legacy=0x7f04e0071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.084+0000 7f04df400640 1 -- 172.21.15.123:0/780573528 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e006bef0 con 0x7f04e006f800 2024-04-24T05:47:18.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.085+0000 7f04df400640 1 --2- 172.21.15.123:0/780573528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e006f800 0x7f04e006c950 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f04e0073620 tx=0x7f04d802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c103d1f5bf8ea7f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:18.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.086+0000 7f04dd600640 1 -- 172.21.15.123:0/780573528 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8036070 con 0x7f04e006f800 2024-04-24T05:47:18.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.086+0000 7f04dd600640 1 -- 172.21.15.123:0/780573528 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d8031600 con 0x7f04e006f800 2024-04-24T05:47:18.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.086+0000 7f04dd600640 1 -- 172.21.15.123:0/780573528 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d80319d0 con 0x7f04e006f800 2024-04-24T05:47:18.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.086+0000 7f04dfe00640 1 -- 172.21.15.123:0/780573528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e006f800 msgr2=0x7f04e006c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.086+0000 7f04dfe00640 1 --2- 172.21.15.123:0/780573528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e006f800 0x7f04e006c950 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f04e0073620 tx=0x7f04d802a6f0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.086+0000 7f04dfe00640 1 -- 172.21.15.123:0/780573528 shutdown_connections 2024-04-24T05:47:18.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.086+0000 7f04dfe00640 1 --2- 172.21.15.123:0/780573528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e006f800 0x7f04e006c950 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.086+0000 7f04dfe00640 1 --2- 172.21.15.123:0/780573528 >> v2:172.21.15.155:3300/0 conn(0x7f04e0072290 0x7f04e006c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.086+0000 7f04dfe00640 1 -- 172.21.15.123:0/780573528 >> 172.21.15.123:0/780573528 conn(0x7f04e00b1cf0 msgr2=0x7f04e00b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:18.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.086+0000 7f04dfe00640 1 -- 172.21.15.123:0/780573528 shutdown_connections 2024-04-24T05:47:18.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.086+0000 7f04dfe00640 1 -- 172.21.15.123:0/780573528 wait complete. 2024-04-24T05:47:18.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.088+0000 7f04dfe00640 1 Processor -- start 2024-04-24T05:47:18.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.088+0000 7f04dfe00640 1 -- start start 2024-04-24T05:47:18.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.089+0000 7f04dfe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e006f800 0x7f04e01591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.089+0000 7f04dfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e0072290 0x7f04e00ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.089+0000 7f04dfe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04e0162a30 0x7f04e00ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.089+0000 7f04dfe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04e006e8c0 con 0x7f04e0072290 2024-04-24T05:47:18.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.089+0000 7f04dfe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04e006e740 con 0x7f04e0162a30 2024-04-24T05:47:18.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.089+0000 7f04dfe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f04e00acfd0 con 0x7f04e006f800 2024-04-24T05:47:18.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.089+0000 7f04de000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e0072290 0x7f04e00ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.089+0000 7f04de000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e0072290 0x7f04e00ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59880/0 (socket says 172.21.15.123:59880) 2024-04-24T05:47:18.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.089+0000 7f04de000640 1 -- 172.21.15.123:0/94695535 learned_addr learned my addr 172.21.15.123:0/94695535 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:18.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.090+0000 7f04df400640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04e0162a30 0x7f04e00ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.090+0000 7f04de000640 1 -- 172.21.15.123:0/94695535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e006f800 msgr2=0x7f04e01591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.090+0000 7f04dea00640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e006f800 0x7f04e01591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.090+0000 7f04de000640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e006f800 0x7f04e01591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.090+0000 7f04de000640 1 -- 172.21.15.123:0/94695535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04e0162a30 msgr2=0x7f04e00ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.090+0000 7f04de000640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04e0162a30 0x7f04e00ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.090+0000 7f04de000640 1 -- 172.21.15.123:0/94695535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e0159700 con 0x7f04e0072290 2024-04-24T05:47:18.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.090+0000 7f04df400640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04e0162a30 0x7f04e00ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:18.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.090+0000 7f04dea00640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e006f800 0x7f04e01591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:18.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.090+0000 7f04de000640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e0072290 0x7f04e00ce180 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00d800 tx=0x7f04cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:18.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.091+0000 7f04d6a00640 1 -- 172.21.15.123:0/94695535 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00b6c0 con 0x7f04e0072290 2024-04-24T05:47:18.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.091+0000 7f04d6a00640 1 -- 172.21.15.123:0/94695535 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04cc019070 con 0x7f04e0072290 2024-04-24T05:47:18.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.091+0000 7f04dfe00640 1 -- 172.21.15.123:0/94695535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e00ccda0 con 0x7f04e0072290 2024-04-24T05:47:18.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.091+0000 7f04d6a00640 1 -- 172.21.15.123:0/94695535 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00ba80 con 0x7f04e0072290 2024-04-24T05:47:18.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.091+0000 7f04dfe00640 1 -- 172.21.15.123:0/94695535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e00cd360 con 0x7f04e0072290 2024-04-24T05:47:18.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.092+0000 7f04dfe00640 1 -- 172.21.15.123:0/94695535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e0068a20 con 0x7f04e0072290 2024-04-24T05:47:18.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.093+0000 7f04d6a00640 1 -- 172.21.15.123:0/94695535 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04cc00bc20 con 0x7f04e0072290 2024-04-24T05:47:18.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.094+0000 7f04d6a00640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f04c006c920 0x7f04c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.094+0000 7f04d6a00640 1 -- 172.21.15.123:0/94695535 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f04cc08c440 con 0x7f04e0072290 2024-04-24T05:47:18.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.094+0000 7f04dea00640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f04c006c920 0x7f04c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.095+0000 7f04dea00640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f04c006c920 0x7f04c006ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f04d000a830 tx=0x7f04d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:18.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.098+0000 7f04d6a00640 1 -- 172.21.15.123:0/94695535 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04cc01c360 con 0x7f04e0072290 2024-04-24T05:47:18.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.299+0000 7f04dfe00640 1 -- 172.21.15.123:0/94695535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f04e00b28e0 con 0x7f04e0072290 2024-04-24T05:47:18.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.299+0000 7f04d6a00640 1 -- 172.21.15.123:0/94695535 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f04cc05a3c0 con 0x7f04e0072290 2024-04-24T05:47:18.301 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:47:18.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.302+0000 7f04dfe00640 1 -- 172.21.15.123:0/94695535 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f04c006c920 msgr2=0x7f04c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.302+0000 7f04dfe00640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f04c006c920 0x7f04c006ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f04d000a830 tx=0x7f04d0008040 comp rx=0 tx=0).stop 2024-04-24T05:47:18.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.302+0000 7f04dfe00640 1 -- 172.21.15.123:0/94695535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e0072290 msgr2=0x7f04e00ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.302+0000 7f04dfe00640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e0072290 0x7f04e00ce180 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00d800 tx=0x7f04cc00dcc0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.302+0000 7f04dfe00640 1 -- 172.21.15.123:0/94695535 shutdown_connections 2024-04-24T05:47:18.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.302+0000 7f04dfe00640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f04c006c920 0x7f04c006ede0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.302+0000 7f04dfe00640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04e0162a30 0x7f04e00ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.302+0000 7f04dfe00640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e0072290 0x7f04e00ce180 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.302+0000 7f04dfe00640 1 --2- 172.21.15.123:0/94695535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04e006f800 0x7f04e01591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.302+0000 7f04dfe00640 1 -- 172.21.15.123:0/94695535 >> 172.21.15.123:0/94695535 conn(0x7f04e00b1cf0 msgr2=0x7f04e00ad300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:18.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.302+0000 7f04dfe00640 1 -- 172.21.15.123:0/94695535 shutdown_connections 2024-04-24T05:47:18.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.302+0000 7f04dfe00640 1 -- 172.21.15.123:0/94695535 wait complete. 2024-04-24T05:47:18.322 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:47:18.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.524+0000 7fe7f1600640 1 Processor -- start 2024-04-24T05:47:18.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.525+0000 7fe7f1600640 1 -- start start 2024-04-24T05:47:18.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.526+0000 7fe7f1600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe7ec06c510 0x7fe7ec071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.526+0000 7fe7f1600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe7ec0cd2c0 0x7fe7ec06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.526+0000 7fe7f1600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7ec0cf170 con 0x7fe7ec06c8f0 2024-04-24T05:47:18.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.526+0000 7fe7f1600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe7ec0cf450 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.526+0000 7fe7f1600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe7ec0cf730 con 0x7fe7ec06c510 2024-04-24T05:47:18.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.526+0000 7fe7eb400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe7ec0cd2c0 0x7fe7ec06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.526+0000 7fe7ea000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe7ec06c8f0 0x7fe7ec0ccbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56804/0 (socket says 172.21.15.123:56804) 2024-04-24T05:47:18.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.526+0000 7fe7ea000640 1 -- 172.21.15.123:0/1738562774 learned_addr learned my addr 172.21.15.123:0/1738562774 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:18.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.526+0000 7fe7eaa00640 1 --2- 172.21.15.123:0/1738562774 >> v2:172.21.15.155:3300/0 conn(0x7fe7ec06c510 0x7fe7ec071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.527+0000 7fe7eb400640 1 -- 172.21.15.123:0/1738562774 >> v2:172.21.15.155:3300/0 conn(0x7fe7ec06c510 msgr2=0x7fe7ec071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:18.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.527+0000 7fe7eb400640 1 --2- 172.21.15.123:0/1738562774 >> v2:172.21.15.155:3300/0 conn(0x7fe7ec06c510 0x7fe7ec071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.527+0000 7fe7eb400640 1 -- 172.21.15.123:0/1738562774 >> v1:172.21.15.123:6789/0 conn(0x7fe7ec06c8f0 legacy=0x7fe7ec0ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.527+0000 7fe7eb400640 1 -- 172.21.15.123:0/1738562774 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec0cfa10 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.527+0000 7fe7e9600640 1 -- 172.21.15.123:0/1738562774 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4081712442 0 0) 0x7fe7ec0cf170 con 0x7fe7ec06c8f0 2024-04-24T05:47:18.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.528+0000 7fe7eb400640 1 --2- 172.21.15.123:0/1738562774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7ec0cd2c0 0x7fe7ec06cd50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0d1240 tx=0x7fe7e402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47382b3c549b0ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:18.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.528+0000 7fe7e9600640 1 -- 172.21.15.123:0/1738562774 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4007740 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.528+0000 7fe7e9600640 1 -- 172.21.15.123:0/1738562774 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e40078e0 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.528+0000 7fe7e9600640 1 -- 172.21.15.123:0/1738562774 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4007cb0 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.529+0000 7fe7f1600640 1 -- 172.21.15.123:0/1738562774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7ec0cd2c0 msgr2=0x7fe7ec06cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.529+0000 7fe7f1600640 1 --2- 172.21.15.123:0/1738562774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7ec0cd2c0 0x7fe7ec06cd50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0d1240 tx=0x7fe7e402a680 comp rx=0 tx=0).stop 2024-04-24T05:47:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.529+0000 7fe7f1600640 1 -- 172.21.15.123:0/1738562774 shutdown_connections 2024-04-24T05:47:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.529+0000 7fe7f1600640 1 --2- 172.21.15.123:0/1738562774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7ec0cd2c0 0x7fe7ec06cd50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.529+0000 7fe7f1600640 1 --2- 172.21.15.123:0/1738562774 >> v2:172.21.15.155:3300/0 conn(0x7fe7ec06c510 0x7fe7ec071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.529+0000 7fe7f1600640 1 -- 172.21.15.123:0/1738562774 >> 172.21.15.123:0/1738562774 conn(0x7fe7ec0b51f0 msgr2=0x7fe7ec0b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.529+0000 7fe7f1600640 1 -- 172.21.15.123:0/1738562774 shutdown_connections 2024-04-24T05:47:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.529+0000 7fe7f1600640 1 -- 172.21.15.123:0/1738562774 wait complete. 2024-04-24T05:47:18.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.531+0000 7fe7f1600640 1 Processor -- start 2024-04-24T05:47:18.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.531+0000 7fe7f1600640 1 -- start start 2024-04-24T05:47:18.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.532+0000 7fe7f1600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7ec06c510 0x7fe7ec15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.532+0000 7fe7f1600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7ec06c8f0 0x7fe7ec15c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.532+0000 7fe7f1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7ec0cd2c0 0x7fe7ec1f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.532+0000 7fe7f1600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec0cee80 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.532+0000 7fe7f1600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec0ced00 con 0x7fe7ec06c510 2024-04-24T05:47:18.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.532+0000 7fe7f1600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe7ec0d0210 con 0x7fe7ec06c8f0 2024-04-24T05:47:18.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.532+0000 7fe7eb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7ec0cd2c0 0x7fe7ec1f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.532+0000 7fe7eb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7ec0cd2c0 0x7fe7ec1f4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59894/0 (socket says 172.21.15.123:59894) 2024-04-24T05:47:18.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.532+0000 7fe7eb400640 1 -- 172.21.15.123:0/3683629127 learned_addr learned my addr 172.21.15.123:0/3683629127 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:18.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.533+0000 7fe7ea000640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7ec06c8f0 0x7fe7ec15c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.533+0000 7fe7eaa00640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7ec06c510 0x7fe7ec15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.533+0000 7fe7eb400640 1 -- 172.21.15.123:0/3683629127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7ec06c8f0 msgr2=0x7fe7ec15c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.533+0000 7fe7eb400640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7ec06c8f0 0x7fe7ec15c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.533+0000 7fe7eb400640 1 -- 172.21.15.123:0/3683629127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7ec06c510 msgr2=0x7fe7ec15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.533+0000 7fe7eb400640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7ec06c510 0x7fe7ec15ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.533+0000 7fe7eb400640 1 -- 172.21.15.123:0/3683629127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec15f490 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.533+0000 7fe7ea000640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7ec06c8f0 0x7fe7ec15c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:18.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.533+0000 7fe7eaa00640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7ec06c510 0x7fe7ec15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:18.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.533+0000 7fe7eb400640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7ec0cd2c0 0x7fe7ec1f4610 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe7e402ac70 tx=0x7fe7e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:18.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.533+0000 7fe7e2a00640 1 -- 172.21.15.123:0/3683629127 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e403f070 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.533+0000 7fe7e2a00640 1 -- 172.21.15.123:0/3683629127 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e4031bc0 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.533+0000 7fe7e2a00640 1 -- 172.21.15.123:0/3683629127 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4044460 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.533+0000 7fe7f1600640 1 -- 172.21.15.123:0/3683629127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec15cb60 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.534+0000 7fe7f1600640 1 -- 172.21.15.123:0/3683629127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec15d0e0 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.535+0000 7fe7f1600640 1 -- 172.21.15.123:0/3683629127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ec068a20 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.536+0000 7fe7e2a00640 1 -- 172.21.15.123:0/3683629127 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7e4007790 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.536+0000 7fe7e2a00640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe7c806c8f0 0x7fe7c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.537+0000 7fe7e2a00640 1 -- 172.21.15.123:0/3683629127 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fe7e40ad860 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.537+0000 7fe7eaa00640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe7c806c8f0 0x7fe7c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.537+0000 7fe7eaa00640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe7c806c8f0 0x7fe7c806edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc006fd0 tx=0x7fe7dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:18.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.541+0000 7fe7e2a00640 1 -- 172.21.15.123:0/3683629127 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe7ec15f490 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.742+0000 7fe7f1600640 1 -- 172.21.15.123:0/3683629127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe7ec06f410 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.743+0000 7fe7e2a00640 1 -- 172.21.15.123:0/3683629127 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fe7e404c350 con 0x7fe7ec0cd2c0 2024-04-24T05:47:18.745 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:18.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.745+0000 7fe7f1600640 1 -- 172.21.15.123:0/3683629127 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe7c806c8f0 msgr2=0x7fe7c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.745+0000 7fe7f1600640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe7c806c8f0 0x7fe7c806edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc006fd0 tx=0x7fe7dc008040 comp rx=0 tx=0).stop 2024-04-24T05:47:18.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.745+0000 7fe7f1600640 1 -- 172.21.15.123:0/3683629127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7ec0cd2c0 msgr2=0x7fe7ec1f4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.745+0000 7fe7f1600640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7ec0cd2c0 0x7fe7ec1f4610 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe7e402ac70 tx=0x7fe7e4004a40 comp rx=0 tx=0).stop 2024-04-24T05:47:18.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.746+0000 7fe7f1600640 1 -- 172.21.15.123:0/3683629127 shutdown_connections 2024-04-24T05:47:18.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.746+0000 7fe7f1600640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe7c806c8f0 0x7fe7c806edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.746+0000 7fe7f1600640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7ec0cd2c0 0x7fe7ec1f4610 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.746+0000 7fe7f1600640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7ec06c8f0 0x7fe7ec15c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.746+0000 7fe7f1600640 1 --2- 172.21.15.123:0/3683629127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7ec06c510 0x7fe7ec15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.746+0000 7fe7f1600640 1 -- 172.21.15.123:0/3683629127 >> 172.21.15.123:0/3683629127 conn(0x7fe7ec0b51f0 msgr2=0x7fe7ec06e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:18.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.746+0000 7fe7f1600640 1 -- 172.21.15.123:0/3683629127 shutdown_connections 2024-04-24T05:47:18.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.746+0000 7fe7f1600640 1 -- 172.21.15.123:0/3683629127 wait complete. 2024-04-24T05:47:18.767 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T05:47:18.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.959+0000 7f190f000640 1 Processor -- start 2024-04-24T05:47:18.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.960+0000 7f190f000640 1 -- start start 2024-04-24T05:47:18.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.961+0000 7f190f000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f19080ad150 0x7f190806cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.961+0000 7f190f000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f190815d5d0 0x7f190806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.961+0000 7f190f000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f190815da60 con 0x7f190806ba00 2024-04-24T05:47:18.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.961+0000 7f190f000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f190806be90 con 0x7f19080ad150 2024-04-24T05:47:18.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.961+0000 7f190f000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f190806c170 con 0x7f190815d5d0 2024-04-24T05:47:18.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.962+0000 7f1907e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f19080ad150 0x7f190806cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.962+0000 7f190d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f190815d5d0 0x7f190806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.962+0000 7f1907e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f19080ad150 0x7f190806cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:57756/0 (socket says 172.21.15.123:57756) 2024-04-24T05:47:18.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.962+0000 7f1907e00640 1 -- 172.21.15.123:0/2845659467 learned_addr learned my addr 172.21.15.123:0/2845659467 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:18.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.962+0000 7f190d000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f190815d5d0 0x7f190806dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:44328/0 (socket says 172.21.15.123:44328) 2024-04-24T05:47:18.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.963+0000 7f190d000640 1 -- 172.21.15.123:0/2845659467 >> v2:172.21.15.155:3300/0 conn(0x7f19080ad150 msgr2=0x7f190806cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:18.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.963+0000 7f190d000640 1 --2- 172.21.15.123:0/2845659467 >> v2:172.21.15.155:3300/0 conn(0x7f19080ad150 0x7f190806cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.963+0000 7f190d000640 1 -- 172.21.15.123:0/2845659467 >> v1:172.21.15.155:6789/0 conn(0x7f190806ba00 legacy=0x7f19080ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.963+0000 7f190d000640 1 -- 172.21.15.123:0/2845659467 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190806c450 con 0x7f190815d5d0 2024-04-24T05:47:18.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.963+0000 7f1907e00640 1 --2- 172.21.15.123:0/2845659467 >> v2:172.21.15.155:3300/0 conn(0x7f19080ad150 0x7f190806cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:47:18.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.964+0000 7f190d000640 1 --2- 172.21.15.123:0/2845659467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f190815d5d0 0x7f190806dbe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f19000097a0 tx=0x7f190002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a45d616b4beb8c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:18.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.965+0000 7f1907400640 1 -- 172.21.15.123:0/2845659467 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900037070 con 0x7f190815d5d0 2024-04-24T05:47:18.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.965+0000 7f1907400640 1 -- 172.21.15.123:0/2845659467 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1900032440 con 0x7f190815d5d0 2024-04-24T05:47:18.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.965+0000 7f1907400640 1 -- 172.21.15.123:0/2845659467 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900032810 con 0x7f190815d5d0 2024-04-24T05:47:18.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.965+0000 7f190f000640 1 -- 172.21.15.123:0/2845659467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f190815d5d0 msgr2=0x7f190806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.965+0000 7f190f000640 1 --2- 172.21.15.123:0/2845659467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f190815d5d0 0x7f190806dbe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f19000097a0 tx=0x7f190002a180 comp rx=0 tx=0).stop 2024-04-24T05:47:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.966+0000 7f190f000640 1 -- 172.21.15.123:0/2845659467 shutdown_connections 2024-04-24T05:47:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.966+0000 7f190f000640 1 --2- 172.21.15.123:0/2845659467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f190815d5d0 0x7f190806dbe0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.966+0000 7f190f000640 1 --2- 172.21.15.123:0/2845659467 >> v2:172.21.15.155:3300/0 conn(0x7f19080ad150 0x7f190806cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.966+0000 7f190f000640 1 -- 172.21.15.123:0/2845659467 >> 172.21.15.123:0/2845659467 conn(0x7f19080b1e70 msgr2=0x7f19080b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.966+0000 7f190f000640 1 -- 172.21.15.123:0/2845659467 shutdown_connections 2024-04-24T05:47:18.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.966+0000 7f190f000640 1 -- 172.21.15.123:0/2845659467 wait complete. 2024-04-24T05:47:18.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.967+0000 7f190f000640 1 Processor -- start 2024-04-24T05:47:18.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.967+0000 7f190f000640 1 -- start start 2024-04-24T05:47:18.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.968+0000 7f190f000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19080ad150 0x7f190809d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.968+0000 7f190f000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f190815d5d0 0x7f190809d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.968+0000 7f190f000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f190815c3a0 0x7f190809dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.968+0000 7f190f000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19080af360 con 0x7f190815d5d0 2024-04-24T05:47:18.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.968+0000 7f190f000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f190806ca10 con 0x7f19080ad150 2024-04-24T05:47:18.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.968+0000 7f190f000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f19081f4590 con 0x7f190815c3a0 2024-04-24T05:47:18.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.969+0000 7f18ff400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f190815d5d0 0x7f190809d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.969+0000 7f18ff400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f190815d5d0 0x7f190809d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33588/0 (socket says 172.21.15.123:33588) 2024-04-24T05:47:18.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.969+0000 7f18ff400640 1 -- 172.21.15.123:0/184257740 learned_addr learned my addr 172.21.15.123:0/184257740 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:18.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.969+0000 7f1907e00640 1 --2- 172.21.15.123:0/184257740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19080ad150 0x7f190809d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.969+0000 7f18ff400640 1 -- 172.21.15.123:0/184257740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f190815c3a0 msgr2=0x7f190809dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.969+0000 7f18ff400640 1 --2- 172.21.15.123:0/184257740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f190815c3a0 0x7f190809dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.969+0000 7f18ff400640 1 -- 172.21.15.123:0/184257740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19080ad150 msgr2=0x7f190809d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:18.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.969+0000 7f18ff400640 1 --2- 172.21.15.123:0/184257740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19080ad150 0x7f190809d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:18.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.969+0000 7f18ff400640 1 -- 172.21.15.123:0/184257740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19080ca150 con 0x7f190815d5d0 2024-04-24T05:47:18.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.969+0000 7f18ff400640 1 --2- 172.21.15.123:0/184257740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f190815d5d0 0x7f190809d800 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f18f000c7c0 tx=0x7f18f000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:18.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.970+0000 7f1904c00640 1 -- 172.21.15.123:0/184257740 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0016020 con 0x7f190815d5d0 2024-04-24T05:47:18.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.970+0000 7f1904c00640 1 -- 172.21.15.123:0/184257740 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18f0006d50 con 0x7f190815d5d0 2024-04-24T05:47:18.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.970+0000 7f1904c00640 1 -- 172.21.15.123:0/184257740 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0003bb0 con 0x7f190815d5d0 2024-04-24T05:47:18.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.970+0000 7f190f000640 1 -- 172.21.15.123:0/184257740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19080ca430 con 0x7f190815d5d0 2024-04-24T05:47:18.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.970+0000 7f190f000640 1 -- 172.21.15.123:0/184257740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1908071dd0 con 0x7f190815d5d0 2024-04-24T05:47:18.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.971+0000 7f190f000640 1 -- 172.21.15.123:0/184257740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19080cc880 con 0x7f190815d5d0 2024-04-24T05:47:18.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.972+0000 7f1904c00640 1 -- 172.21.15.123:0/184257740 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f18f000a850 con 0x7f190815d5d0 2024-04-24T05:47:18.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.973+0000 7f1904c00640 1 --2- 172.21.15.123:0/184257740 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f18e406c8f0 0x7f18e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:18.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.973+0000 7f1904c00640 1 -- 172.21.15.123:0/184257740 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f18f008bc70 con 0x7f190815d5d0 2024-04-24T05:47:18.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.973+0000 7f1907e00640 1 --2- 172.21.15.123:0/184257740 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f18e406c8f0 0x7f18e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:18.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.974+0000 7f1907e00640 1 --2- 172.21.15.123:0/184257740 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f18e406c8f0 0x7f18e406edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f190815c980 tx=0x7f18f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:18.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:18.978+0000 7f1904c00640 1 -- 172.21.15.123:0/184257740 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19080cc880 con 0x7f190815d5d0 2024-04-24T05:47:19.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.191+0000 7f190f000640 1 -- 172.21.15.123:0/184257740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1908073e40 con 0x7f190815d5d0 2024-04-24T05:47:19.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.192+0000 7f1904c00640 1 -- 172.21.15.123:0/184257740 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f1908073e40 con 0x7f190815d5d0 2024-04-24T05:47:19.193 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-24T05:47:19.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.194+0000 7f190f000640 1 -- 172.21.15.123:0/184257740 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f18e406c8f0 msgr2=0x7f18e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:19.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.194+0000 7f190f000640 1 --2- 172.21.15.123:0/184257740 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f18e406c8f0 0x7f18e406edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f190815c980 tx=0x7f18f8008040 comp rx=0 tx=0).stop 2024-04-24T05:47:19.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.194+0000 7f190f000640 1 -- 172.21.15.123:0/184257740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f190815d5d0 msgr2=0x7f190809d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.194+0000 7f190f000640 1 --2- 172.21.15.123:0/184257740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f190815d5d0 0x7f190809d800 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f18f000c7c0 tx=0x7f18f000cc80 comp rx=0 tx=0).stop 2024-04-24T05:47:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.195+0000 7f190f000640 1 -- 172.21.15.123:0/184257740 shutdown_connections 2024-04-24T05:47:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.195+0000 7f190f000640 1 --2- 172.21.15.123:0/184257740 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f18e406c8f0 0x7f18e406edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.195+0000 7f190f000640 1 --2- 172.21.15.123:0/184257740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f190815c3a0 0x7f190809dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.195+0000 7f190f000640 1 --2- 172.21.15.123:0/184257740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f190815d5d0 0x7f190809d800 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.195+0000 7f190f000640 1 --2- 172.21.15.123:0/184257740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19080ad150 0x7f190809d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.195+0000 7f190f000640 1 -- 172.21.15.123:0/184257740 >> 172.21.15.123:0/184257740 conn(0x7f19080b1e70 msgr2=0x7f19081527e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.195+0000 7f190f000640 1 -- 172.21.15.123:0/184257740 shutdown_connections 2024-04-24T05:47:19.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.195+0000 7f190f000640 1 -- 172.21.15.123:0/184257740 wait complete. 2024-04-24T05:47:19.217 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2024-04-24T05:47:19.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:19.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-24T05:47:19.302 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T05:47:19.302 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-04-24T05:47:19.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:19.303 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-24T05:47:19.356 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:47:19.356 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T05:47:19.415 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:47:19.415 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T05:47:19.467 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:19.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.672+0000 7f0659c00640 1 Processor -- start 2024-04-24T05:47:19.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.672+0000 7f0659c00640 1 -- start start 2024-04-24T05:47:19.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.674+0000 7f0659c00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f065415bf80 0x7f06541588f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:19.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.674+0000 7f0659c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f065406ba00 0x7f0654071c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:19.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.674+0000 7f0659c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f065406c330 con 0x7f065406be90 2024-04-24T05:47:19.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.674+0000 7f0659c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f065406cf40 con 0x7f065406ba00 2024-04-24T05:47:19.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.674+0000 7f0659c00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f065406c610 con 0x7f065415bf80 2024-04-24T05:47:19.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.674+0000 7f0652000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f065406ba00 0x7f0654071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:19.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.674+0000 7f0652000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f065406ba00 0x7f0654071c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33600/0 (socket says 172.21.15.123:33600) 2024-04-24T05:47:19.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.674+0000 7f0652000640 1 -- 172.21.15.123:0/2191327536 learned_addr learned my addr 172.21.15.123:0/2191327536 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:19.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.675+0000 7f0652000640 1 -- 172.21.15.123:0/2191327536 >> v2:172.21.15.155:3300/0 conn(0x7f065415bf80 msgr2=0x7f06541588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:19.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.675+0000 7f0652000640 1 --2- 172.21.15.123:0/2191327536 >> v2:172.21.15.155:3300/0 conn(0x7f065415bf80 0x7f06541588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:19.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.675+0000 7f0652000640 1 -- 172.21.15.123:0/2191327536 >> v1:172.21.15.155:6789/0 conn(0x7f065406be90 legacy=0x7f0654072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:19.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.675+0000 7f0652000640 1 -- 172.21.15.123:0/2191327536 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065406c8f0 con 0x7f065406ba00 2024-04-24T05:47:19.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.676+0000 7f0652000640 1 --2- 172.21.15.123:0/2191327536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065406ba00 0x7f0654071c30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f06540d12c0 tx=0x7f063c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdcdc6352a16d27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:19.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.676+0000 7f0651600640 1 -- 172.21.15.123:0/2191327536 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c036070 con 0x7f065406ba00 2024-04-24T05:47:19.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.676+0000 7f0651600640 1 -- 172.21.15.123:0/2191327536 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f063c031600 con 0x7f065406ba00 2024-04-24T05:47:19.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.676+0000 7f0651600640 1 -- 172.21.15.123:0/2191327536 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c0319d0 con 0x7f065406ba00 2024-04-24T05:47:19.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.677+0000 7f0659c00640 1 -- 172.21.15.123:0/2191327536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065406ba00 msgr2=0x7f0654071c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:19.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.677+0000 7f0659c00640 1 --2- 172.21.15.123:0/2191327536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065406ba00 0x7f0654071c30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f06540d12c0 tx=0x7f063c02a660 comp rx=0 tx=0).stop 2024-04-24T05:47:19.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.677+0000 7f0659c00640 1 -- 172.21.15.123:0/2191327536 shutdown_connections 2024-04-24T05:47:19.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.677+0000 7f0659c00640 1 --2- 172.21.15.123:0/2191327536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065406ba00 0x7f0654071c30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:19.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.677+0000 7f0659c00640 1 --2- 172.21.15.123:0/2191327536 >> v2:172.21.15.155:3300/0 conn(0x7f065415bf80 0x7f06541588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:19.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.677+0000 7f0659c00640 1 -- 172.21.15.123:0/2191327536 >> 172.21.15.123:0/2191327536 conn(0x7f06540b1e70 msgr2=0x7f06540b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:19.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.677+0000 7f0659c00640 1 -- 172.21.15.123:0/2191327536 shutdown_connections 2024-04-24T05:47:19.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.677+0000 7f0659c00640 1 -- 172.21.15.123:0/2191327536 wait complete. 2024-04-24T05:47:19.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.679+0000 7f0659c00640 1 Processor -- start 2024-04-24T05:47:19.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.679+0000 7f0659c00640 1 -- start start 2024-04-24T05:47:19.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0659c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f065406ba00 0x7f065414f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:19.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0659c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065406be90 0x7f06540ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:19.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0659c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f065415bf80 0x7f065406dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:19.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0659c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06540722f0 con 0x7f065406be90 2024-04-24T05:47:19.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0659c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0654072170 con 0x7f065406ba00 2024-04-24T05:47:19.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0659c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0654072d90 con 0x7f065415bf80 2024-04-24T05:47:19.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0652000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065406be90 0x7f06540ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:19.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0652000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065406be90 0x7f06540ae0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33612/0 (socket says 172.21.15.123:33612) 2024-04-24T05:47:19.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0652000640 1 -- 172.21.15.123:0/1190567783 learned_addr learned my addr 172.21.15.123:0/1190567783 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:19.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0652000640 1 -- 172.21.15.123:0/1190567783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f065415bf80 msgr2=0x7f065406dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:19.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0652000640 1 --2- 172.21.15.123:0/1190567783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f065415bf80 0x7f065406dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:19.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0652a00640 1 --2- 172.21.15.123:0/1190567783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f065406ba00 0x7f065414f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:19.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0652000640 1 -- 172.21.15.123:0/1190567783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f065406ba00 msgr2=0x7f065414f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:19.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0652000640 1 --2- 172.21.15.123:0/1190567783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f065406ba00 0x7f065414f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:19.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0652000640 1 -- 172.21.15.123:0/1190567783 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06540b0b50 con 0x7f065406be90 2024-04-24T05:47:19.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.680+0000 7f0652a00640 1 --2- 172.21.15.123:0/1190567783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f065406ba00 0x7f065414f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:19.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.681+0000 7f0652000640 1 --2- 172.21.15.123:0/1190567783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065406be90 0x7f06540ae0c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f063c0093e0 tx=0x7f063c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:19.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.681+0000 7f064aa00640 1 -- 172.21.15.123:0/1190567783 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c041020 con 0x7f065406be90 2024-04-24T05:47:19.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.681+0000 7f064aa00640 1 -- 172.21.15.123:0/1190567783 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f063c007820 con 0x7f065406be90 2024-04-24T05:47:19.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.681+0000 7f064aa00640 1 -- 172.21.15.123:0/1190567783 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c036040 con 0x7f065406be90 2024-04-24T05:47:19.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.681+0000 7f0659c00640 1 -- 172.21.15.123:0/1190567783 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06540b0e30 con 0x7f065406be90 2024-04-24T05:47:19.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.681+0000 7f0659c00640 1 -- 172.21.15.123:0/1190567783 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06540b13f0 con 0x7f065406be90 2024-04-24T05:47:19.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.682+0000 7f0659c00640 1 -- 172.21.15.123:0/1190567783 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0654068a20 con 0x7f065406be90 2024-04-24T05:47:19.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.683+0000 7f064aa00640 1 -- 172.21.15.123:0/1190567783 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f063c044020 con 0x7f065406be90 2024-04-24T05:47:19.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.684+0000 7f064aa00640 1 --2- 172.21.15.123:0/1190567783 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f063006c920 0x7f063006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:19.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.684+0000 7f064aa00640 1 -- 172.21.15.123:0/1190567783 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f063c0acda0 con 0x7f065406be90 2024-04-24T05:47:19.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.684+0000 7f0652a00640 1 --2- 172.21.15.123:0/1190567783 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f063006c920 0x7f063006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:19.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.685+0000 7f0652a00640 1 --2- 172.21.15.123:0/1190567783 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f063006c920 0x7f063006ede0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f06541f5f60 tx=0x7f0644008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:19.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.689+0000 7f064aa00640 1 -- 172.21.15.123:0/1190567783 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f063c04b030 con 0x7f065406be90 2024-04-24T05:47:19.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.889+0000 7f0659c00640 1 -- 172.21.15.123:0/1190567783 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06541505a0 con 0x7f065406be90 2024-04-24T05:47:19.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.891+0000 7f064aa00640 1 -- 172.21.15.123:0/1190567783 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7f063c07adb0 con 0x7f065406be90 2024-04-24T05:47:19.893 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:19.893 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:19.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.894+0000 7f0659c00640 1 -- 172.21.15.123:0/1190567783 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f063006c920 msgr2=0x7f063006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:19.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.894+0000 7f0659c00640 1 --2- 172.21.15.123:0/1190567783 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f063006c920 0x7f063006ede0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f06541f5f60 tx=0x7f0644008040 comp rx=0 tx=0).stop 2024-04-24T05:47:19.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.894+0000 7f0659c00640 1 -- 172.21.15.123:0/1190567783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065406be90 msgr2=0x7f06540ae0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:19.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.894+0000 7f0659c00640 1 --2- 172.21.15.123:0/1190567783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065406be90 0x7f06540ae0c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f063c0093e0 tx=0x7f063c004a40 comp rx=0 tx=0).stop 2024-04-24T05:47:19.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.894+0000 7f0659c00640 1 -- 172.21.15.123:0/1190567783 shutdown_connections 2024-04-24T05:47:19.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.894+0000 7f0659c00640 1 --2- 172.21.15.123:0/1190567783 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f063006c920 0x7f063006ede0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:19.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.894+0000 7f0659c00640 1 --2- 172.21.15.123:0/1190567783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f065415bf80 0x7f065406dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:19.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.894+0000 7f0659c00640 1 --2- 172.21.15.123:0/1190567783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065406be90 0x7f06540ae0c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:19.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.894+0000 7f0659c00640 1 --2- 172.21.15.123:0/1190567783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f065406ba00 0x7f065414f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:19.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.894+0000 7f0659c00640 1 -- 172.21.15.123:0/1190567783 >> 172.21.15.123:0/1190567783 conn(0x7f06540b1e70 msgr2=0x7f0654156630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:19.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.895+0000 7f0659c00640 1 -- 172.21.15.123:0/1190567783 shutdown_connections 2024-04-24T05:47:19.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:19.895+0000 7f0659c00640 1 -- 172.21.15.123:0/1190567783 wait complete. 2024-04-24T05:47:19.914 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:47:19.914 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:47:20.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.107+0000 7fbdc6200640 1 Processor -- start 2024-04-24T05:47:20.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.108+0000 7fbdc6200640 1 -- start start 2024-04-24T05:47:20.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.109+0000 7fbdc6200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc00ad6e0 0x7fbdc0167130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.109+0000 7fbdc6200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdc006ba00 con 0x7fbdc00cc5d0 2024-04-24T05:47:20.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.109+0000 7fbdc6200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdc0142530 con 0x7fbdc015c2f0 2024-04-24T05:47:20.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.109+0000 7fbdc6200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc006bd30 con 0x7fbdc00ad6e0 2024-04-24T05:47:20.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.109+0000 7fbdbfe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbdc00cc5d0 0x7fbdc00acfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43166/0 (socket says 172.21.15.123:43166) 2024-04-24T05:47:20.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.109+0000 7fbdbfe00640 1 -- 172.21.15.123:0/230552214 learned_addr learned my addr 172.21.15.123:0/230552214 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:20.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 -- 172.21.15.123:0/230552214 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1631638085 0 0) 0x7fbdc006ba00 con 0x7fbdc00cc5d0 2024-04-24T05:47:20.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 -- 172.21.15.123:0/230552214 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbda8003520 con 0x7fbdc00cc5d0 2024-04-24T05:47:20.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdc5800640 1 --2- 172.21.15.123:0/230552214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc00ad6e0 0x7fbdc0167130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 -- 172.21.15.123:0/230552214 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 369570437 0 0) 0x7fbda8003520 con 0x7fbdc00cc5d0 2024-04-24T05:47:20.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 -- 172.21.15.123:0/230552214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc00ad6e0 msgr2=0x7fbdc0167130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:20.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 --2- 172.21.15.123:0/230552214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc00ad6e0 0x7fbdc0167130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 -- 172.21.15.123:0/230552214 >> v1:172.21.15.155:6789/0 conn(0x7fbdc015c2f0 legacy=0x7fbdc0141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:20.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 -- 172.21.15.123:0/230552214 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc0142810 con 0x7fbdc00cc5d0 2024-04-24T05:47:20.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 -- 172.21.15.123:0/230552214 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7fbdac001670 con 0x7fbdc00cc5d0 2024-04-24T05:47:20.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 -- 172.21.15.123:0/230552214 >> v1:172.21.15.123:6789/0 conn(0x7fbdc00cc5d0 legacy=0x7fbdc00acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:20.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 --2- 172.21.15.123:0/230552214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbda80039e0 0x7fbda8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 --2- 172.21.15.123:0/230552214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc00ad6e0 0x7fbda8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 --2- 172.21.15.123:0/230552214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbda8004d40 0x7fbda80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 -- 172.21.15.123:0/230552214 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc006bd30 con 0x7fbda8004d40 2024-04-24T05:47:20.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 -- 172.21.15.123:0/230552214 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbda800bc70 con 0x7fbda80039e0 2024-04-24T05:47:20.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.110+0000 7fbdbf400640 1 -- 172.21.15.123:0/230552214 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbda800be70 con 0x7fbdc00ad6e0 2024-04-24T05:47:20.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.111+0000 7fbdc5800640 1 --2- 172.21.15.123:0/230552214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbda8004d40 0x7fbda80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.111+0000 7fbdbfe00640 1 --2- 172.21.15.123:0/230552214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc00ad6e0 0x7fbda8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.111+0000 7fbdc4e00640 1 --2- 172.21.15.123:0/230552214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbda80039e0 0x7fbda8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.111+0000 7fbdc5800640 1 -- 172.21.15.123:0/230552214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc00ad6e0 msgr2=0x7fbda8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:20.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.111+0000 7fbdc5800640 1 --2- 172.21.15.123:0/230552214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc00ad6e0 0x7fbda8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.111+0000 7fbdc5800640 1 -- 172.21.15.123:0/230552214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbda80039e0 msgr2=0x7fbda8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:20.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.111+0000 7fbdc5800640 1 --2- 172.21.15.123:0/230552214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbda80039e0 0x7fbda8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.111+0000 7fbdc5800640 1 -- 172.21.15.123:0/230552214 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda8003520 con 0x7fbda8004d40 2024-04-24T05:47:20.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.112+0000 7fbdc5800640 1 --2- 172.21.15.123:0/230552214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbda8004d40 0x7fbda80051c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00afb00 tx=0x7fbda8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cf91fa83b40f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:20.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.113+0000 7fbdbf400640 1 -- 172.21.15.123:0/230552214 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8033070 con 0x7fbda8004d40 2024-04-24T05:47:20.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.113+0000 7fbdbf400640 1 -- 172.21.15.123:0/230552214 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdb802f630 con 0x7fbda8004d40 2024-04-24T05:47:20.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.113+0000 7fbdbf400640 1 -- 172.21.15.123:0/230552214 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8009be0 con 0x7fbda8004d40 2024-04-24T05:47:20.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.113+0000 7fbdc6200640 1 -- 172.21.15.123:0/230552214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbda8004d40 msgr2=0x7fbda80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:20.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.113+0000 7fbdc6200640 1 --2- 172.21.15.123:0/230552214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbda8004d40 0x7fbda80051c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00afb00 tx=0x7fbda8005830 comp rx=0 tx=0).stop 2024-04-24T05:47:20.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.113+0000 7fbdc5800640 1 -- 172.21.15.123:0/230552214 reap_dead start 2024-04-24T05:47:20.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.113+0000 7fbdc6200640 1 -- 172.21.15.123:0/230552214 shutdown_connections 2024-04-24T05:47:20.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.113+0000 7fbdc6200640 1 -- 172.21.15.123:0/230552214 >> 172.21.15.123:0/230552214 conn(0x7fbdc00b1cf0 msgr2=0x7fbdc00b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:20.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.114+0000 7fbdc6200640 1 -- 172.21.15.123:0/230552214 shutdown_connections 2024-04-24T05:47:20.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.114+0000 7fbdc6200640 1 -- 172.21.15.123:0/230552214 wait complete. 2024-04-24T05:47:20.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.115+0000 7fbdc6200640 1 Processor -- start 2024-04-24T05:47:20.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.115+0000 7fbdc6200640 1 -- start start 2024-04-24T05:47:20.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.116+0000 7fbdc6200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbda8004d40 0x7fbdc01f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.116+0000 7fbdc6200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdc006f5b0 0x7fbdc00cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.116+0000 7fbdc6200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc01ee180 0x7fbdc015e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.116+0000 7fbdc6200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc006d350 con 0x7fbdc006f5b0 2024-04-24T05:47:20.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.116+0000 7fbdc6200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc006d1d0 con 0x7fbda8004d40 2024-04-24T05:47:20.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.116+0000 7fbdc6200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc015c6d0 con 0x7fbdc01ee180 2024-04-24T05:47:20.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.117+0000 7fbdbfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdc006f5b0 0x7fbdc00cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.117+0000 7fbdbfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdc006f5b0 0x7fbdc00cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33630/0 (socket says 172.21.15.123:33630) 2024-04-24T05:47:20.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.117+0000 7fbdbfe00640 1 -- 172.21.15.123:0/3024980522 learned_addr learned my addr 172.21.15.123:0/3024980522 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:20.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.117+0000 7fbdc4e00640 1 --2- 172.21.15.123:0/3024980522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbda8004d40 0x7fbdc01f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.117+0000 7fbdc5800640 1 --2- 172.21.15.123:0/3024980522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc01ee180 0x7fbdc015e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.117+0000 7fbdbfe00640 1 -- 172.21.15.123:0/3024980522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc01ee180 msgr2=0x7fbdc015e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.117+0000 7fbdbfe00640 1 --2- 172.21.15.123:0/3024980522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc01ee180 0x7fbdc015e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.117+0000 7fbdbfe00640 1 -- 172.21.15.123:0/3024980522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbda8004d40 msgr2=0x7fbdc01f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.117+0000 7fbdbfe00640 1 --2- 172.21.15.123:0/3024980522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbda8004d40 0x7fbdc01f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.117+0000 7fbdbfe00640 1 -- 172.21.15.123:0/3024980522 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc00cbc60 con 0x7fbdc006f5b0 2024-04-24T05:47:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.118+0000 7fbdbfe00640 1 --2- 172.21.15.123:0/3024980522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdc006f5b0 0x7fbdc00cb5a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00e860 tx=0x7fbdac00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.118+0000 7fbdbcc00640 1 -- 172.21.15.123:0/3024980522 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00c8d0 con 0x7fbdc006f5b0 2024-04-24T05:47:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.118+0000 7fbdbcc00640 1 -- 172.21.15.123:0/3024980522 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdac018070 con 0x7fbdc006f5b0 2024-04-24T05:47:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.118+0000 7fbdbcc00640 1 -- 172.21.15.123:0/3024980522 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00ccd0 con 0x7fbdc006f5b0 2024-04-24T05:47:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.118+0000 7fbdc6200640 1 -- 172.21.15.123:0/3024980522 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc00cbf40 con 0x7fbdc006f5b0 2024-04-24T05:47:20.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.118+0000 7fbdc6200640 1 -- 172.21.15.123:0/3024980522 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc01f46c0 con 0x7fbdc006f5b0 2024-04-24T05:47:20.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.119+0000 7fbdc6200640 1 -- 172.21.15.123:0/3024980522 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc0068a20 con 0x7fbdc006f5b0 2024-04-24T05:47:20.122 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.120+0000 7fbdbcc00640 1 -- 172.21.15.123:0/3024980522 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbdac013420 con 0x7fbdc006f5b0 2024-04-24T05:47:20.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.121+0000 7fbdbcc00640 1 --2- 172.21.15.123:0/3024980522 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbda006c920 0x7fbda006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.121+0000 7fbdbcc00640 1 -- 172.21.15.123:0/3024980522 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fbdac08bbd0 con 0x7fbdc006f5b0 2024-04-24T05:47:20.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.121+0000 7fbdc4e00640 1 --2- 172.21.15.123:0/3024980522 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbda006c920 0x7fbda006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.123 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.122+0000 7fbdc4e00640 1 --2- 172.21.15.123:0/3024980522 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbda006c920 0x7fbda006ede0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4000c00 tx=0x7fbdb4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:20.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.125+0000 7fbdbcc00640 1 -- 172.21.15.123:0/3024980522 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbdac059b50 con 0x7fbdc006f5b0 2024-04-24T05:47:20.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.329+0000 7fbdc6200640 1 -- 172.21.15.123:0/3024980522 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbdc00b0400 con 0x7fbdc006f5b0 2024-04-24T05:47:20.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.330+0000 7fbdbcc00640 1 -- 172.21.15.123:0/3024980522 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fbdac059450 con 0x7fbdc006f5b0 2024-04-24T05:47:20.332 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:47:20.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.333+0000 7fbdc6200640 1 -- 172.21.15.123:0/3024980522 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbda006c920 msgr2=0x7fbda006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:20.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.333+0000 7fbdc6200640 1 --2- 172.21.15.123:0/3024980522 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbda006c920 0x7fbda006ede0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4000c00 tx=0x7fbdb4009040 comp rx=0 tx=0).stop 2024-04-24T05:47:20.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.333+0000 7fbdc6200640 1 -- 172.21.15.123:0/3024980522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdc006f5b0 msgr2=0x7fbdc00cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:20.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.333+0000 7fbdc6200640 1 --2- 172.21.15.123:0/3024980522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdc006f5b0 0x7fbdc00cb5a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00e860 tx=0x7fbdac00ed20 comp rx=0 tx=0).stop 2024-04-24T05:47:20.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.333+0000 7fbdc6200640 1 -- 172.21.15.123:0/3024980522 shutdown_connections 2024-04-24T05:47:20.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.333+0000 7fbdc6200640 1 --2- 172.21.15.123:0/3024980522 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbda006c920 0x7fbda006ede0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.333+0000 7fbdc6200640 1 --2- 172.21.15.123:0/3024980522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdc01ee180 0x7fbdc015e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.333+0000 7fbdc6200640 1 --2- 172.21.15.123:0/3024980522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdc006f5b0 0x7fbdc00cb5a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.333+0000 7fbdc6200640 1 --2- 172.21.15.123:0/3024980522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbda8004d40 0x7fbdc01f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.333+0000 7fbdc6200640 1 -- 172.21.15.123:0/3024980522 >> 172.21.15.123:0/3024980522 conn(0x7fbdc00b1cf0 msgr2=0x7fbdc006db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:20.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.334+0000 7fbdc6200640 1 -- 172.21.15.123:0/3024980522 shutdown_connections 2024-04-24T05:47:20.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.334+0000 7fbdc6200640 1 -- 172.21.15.123:0/3024980522 wait complete. 2024-04-24T05:47:20.355 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:47:20.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.558+0000 7f2917e00640 1 Processor -- start 2024-04-24T05:47:20.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.559+0000 7f2917e00640 1 -- start start 2024-04-24T05:47:20.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.560+0000 7f2917e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f291815e5f0 0x7f291815f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.560+0000 7f2917e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29180d15f0 0x7f29180cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.560+0000 7f2917e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2918162a60 con 0x7f2918162250 2024-04-24T05:47:20.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.560+0000 7f2917e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f29180ceb50 con 0x7f291815e5f0 2024-04-24T05:47:20.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.560+0000 7f2917e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f29180cee30 con 0x7f29180d15f0 2024-04-24T05:47:20.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.561+0000 7f2916000640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f2918162250 0x7f29180d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.123:34522/0 (socket says 172.21.15.123:34522) 2024-04-24T05:47:20.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.561+0000 7f2916a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f291815e5f0 0x7f291815f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.561+0000 7f2916000640 1 -- 172.21.15.123:0/370539562 learned_addr learned my addr 172.21.15.123:0/370539562 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:20.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.561+0000 7f2917400640 1 --2- 172.21.15.123:0/370539562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29180d15f0 0x7f29180cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.561+0000 7f2916a00640 1 -- 172.21.15.123:0/370539562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29180d15f0 msgr2=0x7f29180cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:20.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.561+0000 7f2916a00640 1 --2- 172.21.15.123:0/370539562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29180d15f0 0x7f29180cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.561+0000 7f2916a00640 1 -- 172.21.15.123:0/370539562 >> v1:172.21.15.155:6789/0 conn(0x7f2918162250 legacy=0x7f29180d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:20.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.561+0000 7f2916a00640 1 -- 172.21.15.123:0/370539562 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2918162d40 con 0x7f291815e5f0 2024-04-24T05:47:20.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.563+0000 7f2916a00640 1 --2- 172.21.15.123:0/370539562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291815e5f0 0x7f291815f870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2908009d90 tx=0x7f290802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d7db47079ac6bfb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:20.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.563+0000 7f2915600640 1 -- 172.21.15.123:0/370539562 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908007740 con 0x7f291815e5f0 2024-04-24T05:47:20.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.563+0000 7f2915600640 1 -- 172.21.15.123:0/370539562 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29080078e0 con 0x7f291815e5f0 2024-04-24T05:47:20.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.563+0000 7f2915600640 1 -- 172.21.15.123:0/370539562 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908007cb0 con 0x7f291815e5f0 2024-04-24T05:47:20.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.564+0000 7f2917e00640 1 -- 172.21.15.123:0/370539562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291815e5f0 msgr2=0x7f291815f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:20.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.564+0000 7f2917e00640 1 --2- 172.21.15.123:0/370539562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291815e5f0 0x7f291815f870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2908009d90 tx=0x7f290802a660 comp rx=0 tx=0).stop 2024-04-24T05:47:20.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.564+0000 7f2917e00640 1 -- 172.21.15.123:0/370539562 shutdown_connections 2024-04-24T05:47:20.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.564+0000 7f2917e00640 1 --2- 172.21.15.123:0/370539562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29180d15f0 0x7f29180cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.564+0000 7f2917e00640 1 --2- 172.21.15.123:0/370539562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f291815e5f0 0x7f291815f870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.564+0000 7f2917e00640 1 -- 172.21.15.123:0/370539562 >> 172.21.15.123:0/370539562 conn(0x7f29180b9db0 msgr2=0x7f291809d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:20.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.564+0000 7f2917e00640 1 -- 172.21.15.123:0/370539562 shutdown_connections 2024-04-24T05:47:20.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.564+0000 7f2917e00640 1 -- 172.21.15.123:0/370539562 wait complete. 2024-04-24T05:47:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.566+0000 7f2917e00640 1 Processor -- start 2024-04-24T05:47:20.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.566+0000 7f2917e00640 1 -- start start 2024-04-24T05:47:20.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.567+0000 7f2917e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29180d15f0 0x7f2918157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.567+0000 7f2917e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f291815e5f0 0x7f2918159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.567+0000 7f2917e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918162250 0x7f29180b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.567+0000 7f2917e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f291815e9d0 con 0x7f291815e5f0 2024-04-24T05:47:20.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.567+0000 7f2917e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29181626d0 con 0x7f2918162250 2024-04-24T05:47:20.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.567+0000 7f2917e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f291815eb50 con 0x7f29180d15f0 2024-04-24T05:47:20.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.567+0000 7f2916000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f291815e5f0 0x7f2918159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.567+0000 7f2916000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f291815e5f0 0x7f2918159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33642/0 (socket says 172.21.15.123:33642) 2024-04-24T05:47:20.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.567+0000 7f2916000640 1 -- 172.21.15.123:0/3382207874 learned_addr learned my addr 172.21.15.123:0/3382207874 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:20.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.567+0000 7f2916a00640 1 --2- 172.21.15.123:0/3382207874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29180d15f0 0x7f2918157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.567+0000 7f2917400640 1 --2- 172.21.15.123:0/3382207874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918162250 0x7f29180b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.568+0000 7f2916000640 1 -- 172.21.15.123:0/3382207874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29180d15f0 msgr2=0x7f2918157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:20.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.568+0000 7f2916000640 1 --2- 172.21.15.123:0/3382207874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29180d15f0 0x7f2918157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.568+0000 7f2916000640 1 -- 172.21.15.123:0/3382207874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918162250 msgr2=0x7f29180b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:20.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.568+0000 7f2916000640 1 --2- 172.21.15.123:0/3382207874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918162250 0x7f29180b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.568+0000 7f2916000640 1 -- 172.21.15.123:0/3382207874 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29180b5790 con 0x7f291815e5f0 2024-04-24T05:47:20.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.568+0000 7f2916000640 1 --2- 172.21.15.123:0/3382207874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f291815e5f0 0x7f2918159030 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f290000c440 tx=0x7f290000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:20.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.568+0000 7f290ea00640 1 -- 172.21.15.123:0/3382207874 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900017070 con 0x7f291815e5f0 2024-04-24T05:47:20.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.568+0000 7f290ea00640 1 -- 172.21.15.123:0/3382207874 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2900005ce0 con 0x7f291815e5f0 2024-04-24T05:47:20.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.568+0000 7f2917e00640 1 -- 172.21.15.123:0/3382207874 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29180cca60 con 0x7f291815e5f0 2024-04-24T05:47:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.568+0000 7f290ea00640 1 -- 172.21.15.123:0/3382207874 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29000072f0 con 0x7f291815e5f0 2024-04-24T05:47:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.569+0000 7f2917e00640 1 -- 172.21.15.123:0/3382207874 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29180cd020 con 0x7f291815e5f0 2024-04-24T05:47:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.570+0000 7f2917e00640 1 -- 172.21.15.123:0/3382207874 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2918068a20 con 0x7f291815e5f0 2024-04-24T05:47:20.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.570+0000 7f290ea00640 1 -- 172.21.15.123:0/3382207874 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f290001a070 con 0x7f291815e5f0 2024-04-24T05:47:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.571+0000 7f290ea00640 1 --2- 172.21.15.123:0/3382207874 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f28f406c8f0 0x7f28f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.571+0000 7f290ea00640 1 -- 172.21.15.123:0/3382207874 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f290008b3f0 con 0x7f291815e5f0 2024-04-24T05:47:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.572+0000 7f2916a00640 1 --2- 172.21.15.123:0/3382207874 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f28f406c8f0 0x7f28f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.572+0000 7f2916a00640 1 --2- 172.21.15.123:0/3382207874 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f28f406c8f0 0x7f28f406edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2908034000 tx=0x7f290802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:20.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.576+0000 7f290ea00640 1 -- 172.21.15.123:0/3382207874 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29000592e0 con 0x7f291815e5f0 2024-04-24T05:47:20.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.775+0000 7f2917e00640 1 -- 172.21.15.123:0/3382207874 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f29180d5100 con 0x7f291815e5f0 2024-04-24T05:47:20.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.776+0000 7f290ea00640 1 -- 172.21.15.123:0/3382207874 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f2900058be0 con 0x7f291815e5f0 2024-04-24T05:47:20.777 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:20.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.778+0000 7f2917e00640 1 -- 172.21.15.123:0/3382207874 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f28f406c8f0 msgr2=0x7f28f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:20.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.778+0000 7f2917e00640 1 --2- 172.21.15.123:0/3382207874 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f28f406c8f0 0x7f28f406edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2908034000 tx=0x7f290802f000 comp rx=0 tx=0).stop 2024-04-24T05:47:20.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.778+0000 7f2917e00640 1 -- 172.21.15.123:0/3382207874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f291815e5f0 msgr2=0x7f2918159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:20.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.778+0000 7f2917e00640 1 --2- 172.21.15.123:0/3382207874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f291815e5f0 0x7f2918159030 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f290000c440 tx=0x7f290000c900 comp rx=0 tx=0).stop 2024-04-24T05:47:20.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.778+0000 7f2917e00640 1 -- 172.21.15.123:0/3382207874 shutdown_connections 2024-04-24T05:47:20.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.778+0000 7f2917e00640 1 --2- 172.21.15.123:0/3382207874 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f28f406c8f0 0x7f28f406edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.778+0000 7f2917e00640 1 --2- 172.21.15.123:0/3382207874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2918162250 0x7f29180b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.778+0000 7f2917e00640 1 --2- 172.21.15.123:0/3382207874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f291815e5f0 0x7f2918159030 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.778+0000 7f2917e00640 1 --2- 172.21.15.123:0/3382207874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29180d15f0 0x7f2918157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:20.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.778+0000 7f2917e00640 1 -- 172.21.15.123:0/3382207874 >> 172.21.15.123:0/3382207874 conn(0x7f29180b9db0 msgr2=0x7f29181610a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:20.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.778+0000 7f2917e00640 1 -- 172.21.15.123:0/3382207874 shutdown_connections 2024-04-24T05:47:20.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.779+0000 7f2917e00640 1 -- 172.21.15.123:0/3382207874 wait complete. 2024-04-24T05:47:20.798 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:47:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.995+0000 7f96d3e00640 1 Processor -- start 2024-04-24T05:47:20.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.995+0000 7f96d3e00640 1 -- start start 2024-04-24T05:47:20.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.997+0000 7f96d3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f96cc15f590 0x7f96cc071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.997+0000 7f96d3e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96cc15fb40 con 0x7f96cc06dc60 2024-04-24T05:47:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.997+0000 7f96d3e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96cc0ccbc0 con 0x7f96cc071840 2024-04-24T05:47:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.997+0000 7f96d3e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f96cc0ccea0 con 0x7f96cc15f590 2024-04-24T05:47:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.997+0000 7f96d1e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f96cc15f590 0x7f96cc071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.997+0000 7f96d1400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f96cc06dc60 0x7f96cc0cd6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43172/0 (socket says 172.21.15.123:43172) 2024-04-24T05:47:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.997+0000 7f96d1400640 1 -- 172.21.15.123:0/1901962664 learned_addr learned my addr 172.21.15.123:0/1901962664 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:20.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.998+0000 7f96d1e00640 1 -- 172.21.15.123:0/1901962664 >> v1:172.21.15.155:6789/0 conn(0x7f96cc071840 legacy=0x7f96cc15ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:21.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.998+0000 7f96d1e00640 1 -- 172.21.15.123:0/1901962664 >> v1:172.21.15.123:6789/0 conn(0x7f96cc06dc60 legacy=0x7f96cc0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.998+0000 7f96d1e00640 1 -- 172.21.15.123:0/1901962664 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc0cd180 con 0x7f96cc15f590 2024-04-24T05:47:21.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.998+0000 7f96cbe00640 1 -- 172.21.15.123:0/1901962664 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4155634758 0 0) 0x7f96cc15fb40 con 0x7f96cc06dc60 2024-04-24T05:47:21.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.999+0000 7f96d1e00640 1 --2- 172.21.15.123:0/1901962664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc15f590 0x7f96cc071cc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009d90 tx=0x7f96c402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe61eaf8238eec67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:21.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.999+0000 7f96cbe00640 1 -- 172.21.15.123:0/1901962664 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4031600 con 0x7f96cc15f590 2024-04-24T05:47:21.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.999+0000 7f96cbe00640 1 -- 172.21.15.123:0/1901962664 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c40317a0 con 0x7f96cc15f590 2024-04-24T05:47:21.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:20.999+0000 7f96cbe00640 1 -- 172.21.15.123:0/1901962664 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4031b70 con 0x7f96cc15f590 2024-04-24T05:47:21.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.000+0000 7f96d3e00640 1 -- 172.21.15.123:0/1901962664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc15f590 msgr2=0x7f96cc071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.000+0000 7f96d3e00640 1 --2- 172.21.15.123:0/1901962664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc15f590 0x7f96cc071cc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009d90 tx=0x7f96c402a610 comp rx=0 tx=0).stop 2024-04-24T05:47:21.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.000+0000 7f96d3e00640 1 -- 172.21.15.123:0/1901962664 shutdown_connections 2024-04-24T05:47:21.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.000+0000 7f96d3e00640 1 --2- 172.21.15.123:0/1901962664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc15f590 0x7f96cc071cc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.000+0000 7f96d3e00640 1 -- 172.21.15.123:0/1901962664 >> 172.21.15.123:0/1901962664 conn(0x7f96cc0d23f0 msgr2=0x7f96cc0d27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:21.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.000+0000 7f96d3e00640 1 -- 172.21.15.123:0/1901962664 shutdown_connections 2024-04-24T05:47:21.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.000+0000 7f96d3e00640 1 -- 172.21.15.123:0/1901962664 wait complete. 2024-04-24T05:47:21.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.002+0000 7f96d3e00640 1 Processor -- start 2024-04-24T05:47:21.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.002+0000 7f96d3e00640 1 -- start start 2024-04-24T05:47:21.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.003+0000 7f96d3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96cc06dc60 0x7f96cc0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.003+0000 7f96d3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc071840 0x7f96cc0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.003+0000 7f96d3e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96cc15f590 0x7f96cc0c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.003+0000 7f96d3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc0d0700 con 0x7f96cc071840 2024-04-24T05:47:21.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.003+0000 7f96d3e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc0d0580 con 0x7f96cc06dc60 2024-04-24T05:47:21.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.003+0000 7f96d3e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f96cc0d0880 con 0x7f96cc15f590 2024-04-24T05:47:21.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.003+0000 7f96d0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc071840 0x7f96cc0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.003+0000 7f96d0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc071840 0x7f96cc0c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33650/0 (socket says 172.21.15.123:33650) 2024-04-24T05:47:21.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.003+0000 7f96d0a00640 1 -- 172.21.15.123:0/2197805806 learned_addr learned my addr 172.21.15.123:0/2197805806 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:21.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.003+0000 7f96d0a00640 1 -- 172.21.15.123:0/2197805806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96cc15f590 msgr2=0x7f96cc0c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.003+0000 7f96d0a00640 1 --2- 172.21.15.123:0/2197805806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96cc15f590 0x7f96cc0c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.004+0000 7f96d1400640 1 --2- 172.21.15.123:0/2197805806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96cc06dc60 0x7f96cc0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.004+0000 7f96d0a00640 1 -- 172.21.15.123:0/2197805806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96cc06dc60 msgr2=0x7f96cc0cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.004+0000 7f96d0a00640 1 --2- 172.21.15.123:0/2197805806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96cc06dc60 0x7f96cc0cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.004+0000 7f96d0a00640 1 -- 172.21.15.123:0/2197805806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc0c7a40 con 0x7f96cc071840 2024-04-24T05:47:21.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.004+0000 7f96d0a00640 1 --2- 172.21.15.123:0/2197805806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc071840 0x7f96cc0c7210 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f96b800c430 tx=0x7f96b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:21.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.004+0000 7f96c9600640 1 -- 172.21.15.123:0/2197805806 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8011790 con 0x7f96cc071840 2024-04-24T05:47:21.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.004+0000 7f96c9600640 1 -- 172.21.15.123:0/2197805806 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b8017070 con 0x7f96cc071840 2024-04-24T05:47:21.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.004+0000 7f96d3e00640 1 -- 172.21.15.123:0/2197805806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96cc0c7d20 con 0x7f96cc071840 2024-04-24T05:47:21.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.004+0000 7f96c9600640 1 -- 172.21.15.123:0/2197805806 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8011b70 con 0x7f96cc071840 2024-04-24T05:47:21.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.004+0000 7f96d3e00640 1 -- 172.21.15.123:0/2197805806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96cc0c82e0 con 0x7f96cc071840 2024-04-24T05:47:21.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.006+0000 7f96d3e00640 1 -- 172.21.15.123:0/2197805806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96cc068a20 con 0x7f96cc071840 2024-04-24T05:47:21.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.007+0000 7f96c9600640 1 -- 172.21.15.123:0/2197805806 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96b8011d10 con 0x7f96cc071840 2024-04-24T05:47:21.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.007+0000 7f96c9600640 1 --2- 172.21.15.123:0/2197805806 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f96ac06c8f0 0x7f96ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.008+0000 7f96c9600640 1 -- 172.21.15.123:0/2197805806 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f96b808be30 con 0x7f96cc071840 2024-04-24T05:47:21.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.008+0000 7f96d1400640 1 --2- 172.21.15.123:0/2197805806 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f96ac06c8f0 0x7f96ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.008+0000 7f96d1400640 1 --2- 172.21.15.123:0/2197805806 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f96ac06c8f0 0x7f96ac06edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f96bc000f60 tx=0x7f96bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:21.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.012+0000 7f96c9600640 1 -- 172.21.15.123:0/2197805806 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96b8059e50 con 0x7f96cc071840 2024-04-24T05:47:21.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.213+0000 7f96d3e00640 1 -- 172.21.15.123:0/2197805806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f96cc161b00 con 0x7f96cc071840 2024-04-24T05:47:21.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.214+0000 7f96c9600640 1 -- 172.21.15.123:0/2197805806 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f96b8059750 con 0x7f96cc071840 2024-04-24T05:47:21.215 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:47:21.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.217+0000 7f96d3e00640 1 -- 172.21.15.123:0/2197805806 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f96ac06c8f0 msgr2=0x7f96ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.217+0000 7f96d3e00640 1 --2- 172.21.15.123:0/2197805806 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f96ac06c8f0 0x7f96ac06edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f96bc000f60 tx=0x7f96bc009040 comp rx=0 tx=0).stop 2024-04-24T05:47:21.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.217+0000 7f96d3e00640 1 -- 172.21.15.123:0/2197805806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc071840 msgr2=0x7f96cc0c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.217+0000 7f96d3e00640 1 --2- 172.21.15.123:0/2197805806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc071840 0x7f96cc0c7210 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f96b800c430 tx=0x7f96b800c8f0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.217+0000 7f96d3e00640 1 -- 172.21.15.123:0/2197805806 shutdown_connections 2024-04-24T05:47:21.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.217+0000 7f96d3e00640 1 --2- 172.21.15.123:0/2197805806 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f96ac06c8f0 0x7f96ac06edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.217+0000 7f96d3e00640 1 --2- 172.21.15.123:0/2197805806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96cc15f590 0x7f96cc0c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.217+0000 7f96d3e00640 1 --2- 172.21.15.123:0/2197805806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc071840 0x7f96cc0c7210 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.217+0000 7f96d3e00640 1 --2- 172.21.15.123:0/2197805806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96cc06dc60 0x7f96cc0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.217+0000 7f96d3e00640 1 -- 172.21.15.123:0/2197805806 >> 172.21.15.123:0/2197805806 conn(0x7f96cc0d23f0 msgr2=0x7f96cc0cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:21.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.217+0000 7f96d3e00640 1 -- 172.21.15.123:0/2197805806 shutdown_connections 2024-04-24T05:47:21.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.217+0000 7f96d3e00640 1 -- 172.21.15.123:0/2197805806 wait complete. 2024-04-24T05:47:21.238 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:47:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.439+0000 7f5cc3e00640 1 Processor -- start 2024-04-24T05:47:21.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.440+0000 7f5cc3e00640 1 -- start start 2024-04-24T05:47:21.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.441+0000 7f5cc3e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5cc415e790 0x7f5cc40d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.441+0000 7f5cc3e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc415eb70 con 0x7f5cc40b5f10 2024-04-24T05:47:21.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.441+0000 7f5cc3e00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc40d2e70 con 0x7f5cc40d2120 2024-04-24T05:47:21.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.441+0000 7f5cc3e00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5cc40d3a80 con 0x7f5cc415e790 2024-04-24T05:47:21.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.441+0000 7f5cc2000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f5cc40b5f10 0x7f5cc40b6330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43180/0 (socket says 172.21.15.123:43180) 2024-04-24T05:47:21.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.441+0000 7f5cc2000640 1 -- 172.21.15.123:0/4103104168 learned_addr learned my addr 172.21.15.123:0/4103104168 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.442+0000 7f5cc1600640 1 -- 172.21.15.123:0/4103104168 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2999845720 0 0) 0x7f5cc415eb70 con 0x7f5cc40b5f10 2024-04-24T05:47:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.442+0000 7f5cc1600640 1 -- 172.21.15.123:0/4103104168 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ca8003520 con 0x7f5cc40b5f10 2024-04-24T05:47:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.442+0000 7f5cc2a00640 1 --2- 172.21.15.123:0/4103104168 >> v2:172.21.15.155:3300/0 conn(0x7f5cc415e790 0x7f5cc40d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.442+0000 7f5cc1600640 1 -- 172.21.15.123:0/4103104168 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2163534700 0 0) 0x7f5ca8003520 con 0x7f5cc40b5f10 2024-04-24T05:47:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.442+0000 7f5cc1600640 1 -- 172.21.15.123:0/4103104168 >> v2:172.21.15.155:3300/0 conn(0x7f5cc415e790 msgr2=0x7f5cc40d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.442+0000 7f5cc1600640 1 --2- 172.21.15.123:0/4103104168 >> v2:172.21.15.155:3300/0 conn(0x7f5cc415e790 0x7f5cc40d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.442+0000 7f5cc1600640 1 -- 172.21.15.123:0/4103104168 >> v1:172.21.15.155:6789/0 conn(0x7f5cc40d2120 legacy=0x7f5cc40d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.442+0000 7f5cc1600640 1 -- 172.21.15.123:0/4103104168 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc40d3150 con 0x7f5cc40b5f10 2024-04-24T05:47:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.442+0000 7f5cc1600640 1 -- 172.21.15.123:0/4103104168 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f5cb0001670 con 0x7f5cc40b5f10 2024-04-24T05:47:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.442+0000 7f5cc1600640 1 -- 172.21.15.123:0/4103104168 >> v1:172.21.15.123:6789/0 conn(0x7f5cc40b5f10 legacy=0x7f5cc40b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.442+0000 7f5cc1600640 1 --2- 172.21.15.123:0/4103104168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca8003a00 0x7f5ca8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.442+0000 7f5cc1600640 1 --2- 172.21.15.123:0/4103104168 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca8004800 0x7f5ca8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.443+0000 7f5cc1600640 1 --2- 172.21.15.123:0/4103104168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca8005210 0x7f5ca800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.443+0000 7f5cc1600640 1 -- 172.21.15.123:0/4103104168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc40d3a80 con 0x7f5ca8005210 2024-04-24T05:47:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.443+0000 7f5cc1600640 1 -- 172.21.15.123:0/4103104168 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca800c160 con 0x7f5ca8003a00 2024-04-24T05:47:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.443+0000 7f5cc1600640 1 -- 172.21.15.123:0/4103104168 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca800c360 con 0x7f5ca8004800 2024-04-24T05:47:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.443+0000 7f5cc3400640 1 --2- 172.21.15.123:0/4103104168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca8005210 0x7f5ca800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.443+0000 7f5cc2a00640 1 --2- 172.21.15.123:0/4103104168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca8003a00 0x7f5ca8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.443+0000 7f5cc3400640 1 -- 172.21.15.123:0/4103104168 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca8004800 msgr2=0x7f5ca8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.443+0000 7f5cc2000640 1 --2- 172.21.15.123:0/4103104168 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca8004800 0x7f5ca8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.443+0000 7f5cc3400640 1 --2- 172.21.15.123:0/4103104168 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ca8004800 0x7f5ca8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.443+0000 7f5cc3400640 1 -- 172.21.15.123:0/4103104168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca8003a00 msgr2=0x7f5ca8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.443+0000 7f5cc3400640 1 --2- 172.21.15.123:0/4103104168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca8003a00 0x7f5ca8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.443+0000 7f5cc3400640 1 -- 172.21.15.123:0/4103104168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca8003520 con 0x7f5ca8005210 2024-04-24T05:47:21.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.443+0000 7f5cc2a00640 1 --2- 172.21.15.123:0/4103104168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca8003a00 0x7f5ca8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.444+0000 7f5cc3400640 1 -- 172.21.15.123:0/4103104168 reap_dead start 2024-04-24T05:47:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.444+0000 7f5cc3400640 1 --2- 172.21.15.123:0/4103104168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca8005210 0x7f5ca800b9c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc001e50 tx=0x7f5ca8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1412978a50ec03c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.445+0000 7f5cc1600640 1 -- 172.21.15.123:0/4103104168 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc001a70 con 0x7f5ca8005210 2024-04-24T05:47:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.445+0000 7f5cc1600640 1 -- 172.21.15.123:0/4103104168 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cbc003970 con 0x7f5ca8005210 2024-04-24T05:47:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.445+0000 7f5cc1600640 1 -- 172.21.15.123:0/4103104168 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc003d60 con 0x7f5ca8005210 2024-04-24T05:47:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.445+0000 7f5cc3e00640 1 -- 172.21.15.123:0/4103104168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca8005210 msgr2=0x7f5ca800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.445+0000 7f5cc3e00640 1 --2- 172.21.15.123:0/4103104168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca8005210 0x7f5ca800b9c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc001e50 tx=0x7f5ca8005830 comp rx=0 tx=0).stop 2024-04-24T05:47:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.445+0000 7f5cc3e00640 1 -- 172.21.15.123:0/4103104168 shutdown_connections 2024-04-24T05:47:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.445+0000 7f5cc3e00640 1 --2- 172.21.15.123:0/4103104168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca8005210 0x7f5ca800b9c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.445+0000 7f5cc3e00640 1 -- 172.21.15.123:0/4103104168 >> 172.21.15.123:0/4103104168 conn(0x7f5cc409d830 msgr2=0x7f5cc40b73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:21.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.445+0000 7f5cc3e00640 1 -- 172.21.15.123:0/4103104168 shutdown_connections 2024-04-24T05:47:21.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.445+0000 7f5cc3e00640 1 -- 172.21.15.123:0/4103104168 wait complete. 2024-04-24T05:47:21.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.447+0000 7f5cc3e00640 1 Processor -- start 2024-04-24T05:47:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.447+0000 7f5cc3e00640 1 -- start start 2024-04-24T05:47:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.448+0000 7f5cc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca8005210 0x7f5cc41535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.448+0000 7f5cc3e00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cc406d540 0x7f5cc406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.448+0000 7f5cc3e00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cc406ba00 0x7f5cc406be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.448+0000 7f5cc3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc40b6330 con 0x7f5ca8005210 2024-04-24T05:47:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.448+0000 7f5cc3e00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc40d3710 con 0x7f5cc406ba00 2024-04-24T05:47:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.448+0000 7f5cc3e00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc40b64b0 con 0x7f5cc406d540 2024-04-24T05:47:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.449+0000 7f5cc2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca8005210 0x7f5cc41535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.449+0000 7f5cc2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca8005210 0x7f5cc41535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33676/0 (socket says 172.21.15.123:33676) 2024-04-24T05:47:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.449+0000 7f5cc2a00640 1 -- 172.21.15.123:0/655778208 learned_addr learned my addr 172.21.15.123:0/655778208 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.449+0000 7f5cc2000640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cc406d540 0x7f5cc406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.449+0000 7f5cc3400640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cc406ba00 0x7f5cc406be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.449+0000 7f5cc2a00640 1 -- 172.21.15.123:0/655778208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cc406d540 msgr2=0x7f5cc406d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.449+0000 7f5cc2a00640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cc406d540 0x7f5cc406d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.449+0000 7f5cc2a00640 1 -- 172.21.15.123:0/655778208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cc406ba00 msgr2=0x7f5cc406be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.449+0000 7f5cc2a00640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cc406ba00 0x7f5cc406be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.449+0000 7f5cc2a00640 1 -- 172.21.15.123:0/655778208 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc4154090 con 0x7f5ca8005210 2024-04-24T05:47:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.449+0000 7f5cc3400640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cc406ba00 0x7f5cc406be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:21.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.449+0000 7f5cc2a00640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca8005210 0x7f5cc41535c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400e840 tx=0x7f5cb400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.450+0000 7f5cbaa00640 1 -- 172.21.15.123:0/655778208 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4017070 con 0x7f5ca8005210 2024-04-24T05:47:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.450+0000 7f5cbaa00640 1 -- 172.21.15.123:0/655778208 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb400c6d0 con 0x7f5ca8005210 2024-04-24T05:47:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.450+0000 7f5cbaa00640 1 -- 172.21.15.123:0/655778208 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb400f040 con 0x7f5ca8005210 2024-04-24T05:47:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.450+0000 7f5cc3e00640 1 -- 172.21.15.123:0/655778208 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc406c5e0 con 0x7f5ca8005210 2024-04-24T05:47:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.450+0000 7f5cc3e00640 1 -- 172.21.15.123:0/655778208 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc406cba0 con 0x7f5ca8005210 2024-04-24T05:47:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.451+0000 7f5cc3e00640 1 -- 172.21.15.123:0/655778208 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc4069130 con 0x7f5ca8005210 2024-04-24T05:47:21.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.452+0000 7f5cbaa00640 1 -- 172.21.15.123:0/655778208 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5cb401a070 con 0x7f5ca8005210 2024-04-24T05:47:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.453+0000 7f5cbaa00640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5ca406c8f0 0x7f5ca406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.453+0000 7f5cbaa00640 1 -- 172.21.15.123:0/655778208 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f5cb408bba0 con 0x7f5ca8005210 2024-04-24T05:47:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.453+0000 7f5cc2000640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5ca406c8f0 0x7f5ca406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.454+0000 7f5cc2000640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5ca406c8f0 0x7f5ca406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00017e0 tx=0x7f5cb0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:21.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.457+0000 7f5cbaa00640 1 -- 172.21.15.123:0/655778208 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5cc4154090 con 0x7f5ca8005210 2024-04-24T05:47:21.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.658+0000 7f5cc3e00640 1 -- 172.21.15.123:0/655778208 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5cc406cd70 con 0x7f5ca8005210 2024-04-24T05:47:21.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.658+0000 7f5cbaa00640 1 -- 172.21.15.123:0/655778208 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f5cb4059b20 con 0x7f5ca8005210 2024-04-24T05:47:21.660 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.661+0000 7f5cc3e00640 1 -- 172.21.15.123:0/655778208 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5ca406c8f0 msgr2=0x7f5ca406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.661+0000 7f5cc3e00640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5ca406c8f0 0x7f5ca406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00017e0 tx=0x7f5cb0009040 comp rx=0 tx=0).stop 2024-04-24T05:47:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.661+0000 7f5cc3e00640 1 -- 172.21.15.123:0/655778208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca8005210 msgr2=0x7f5cc41535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.661+0000 7f5cc3e00640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca8005210 0x7f5cc41535c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400e840 tx=0x7f5cb400ed00 comp rx=0 tx=0).stop 2024-04-24T05:47:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.661+0000 7f5cc3e00640 1 -- 172.21.15.123:0/655778208 shutdown_connections 2024-04-24T05:47:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.661+0000 7f5cc3e00640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5ca406c8f0 0x7f5ca406edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.661+0000 7f5cc3e00640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cc406ba00 0x7f5cc406be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.661+0000 7f5cc3e00640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cc406d540 0x7f5cc406d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.661+0000 7f5cc3e00640 1 --2- 172.21.15.123:0/655778208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca8005210 0x7f5cc41535c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.661+0000 7f5cc3e00640 1 -- 172.21.15.123:0/655778208 >> 172.21.15.123:0/655778208 conn(0x7f5cc409d830 msgr2=0x7f5cc40cf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.661+0000 7f5cc3e00640 1 -- 172.21.15.123:0/655778208 shutdown_connections 2024-04-24T05:47:21.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.661+0000 7f5cc3e00640 1 -- 172.21.15.123:0/655778208 wait complete. 2024-04-24T05:47:21.681 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T05:47:21.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.868+0000 7f3200a00640 1 Processor -- start 2024-04-24T05:47:21.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.868+0000 7f3200a00640 1 -- start start 2024-04-24T05:47:21.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.869+0000 7f3200a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f31fc0ad150 0x7f31fc06cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.869+0000 7f3200a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f31fc06ba00 0x7f31fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.869+0000 7f3200a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31fc06be90 con 0x7f31fc0ad530 2024-04-24T05:47:21.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.869+0000 7f3200a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f31fc06c170 con 0x7f31fc0ad150 2024-04-24T05:47:21.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.869+0000 7f3200a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f31fc06c450 con 0x7f31fc06ba00 2024-04-24T05:47:21.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.870+0000 7f31f9e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f31fc0ad150 0x7f31fc06cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.870+0000 7f31f9e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f31fc0ad150 0x7f31fc06cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33682/0 (socket says 172.21.15.123:33682) 2024-04-24T05:47:21.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.870+0000 7f31f9e00640 1 -- 172.21.15.123:0/2999644810 learned_addr learned my addr 172.21.15.123:0/2999644810 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:21.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.870+0000 7f31f3e00640 1 --2- 172.21.15.123:0/2999644810 >> v2:172.21.15.155:3300/0 conn(0x7f31fc06ba00 0x7f31fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.870+0000 7f31f9e00640 1 -- 172.21.15.123:0/2999644810 >> v2:172.21.15.155:3300/0 conn(0x7f31fc06ba00 msgr2=0x7f31fc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:21.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.870+0000 7f31f9e00640 1 --2- 172.21.15.123:0/2999644810 >> v2:172.21.15.155:3300/0 conn(0x7f31fc06ba00 0x7f31fc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.870+0000 7f31f9e00640 1 -- 172.21.15.123:0/2999644810 >> v1:172.21.15.123:6789/0 conn(0x7f31fc0ad530 legacy=0x7f31fc06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.870+0000 7f31f9e00640 1 -- 172.21.15.123:0/2999644810 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc06c730 con 0x7f31fc0ad150 2024-04-24T05:47:21.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.870+0000 7f31f9400640 1 -- 172.21.15.123:0/2999644810 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3318946055 0 0) 0x7f31fc06be90 con 0x7f31fc0ad530 2024-04-24T05:47:21.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.871+0000 7f31f9e00640 1 --2- 172.21.15.123:0/2999644810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31fc0ad150 0x7f31fc06cf30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0afd90 tx=0x7f31ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c962ce41c6f9c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:21.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.872+0000 7f31f9400640 1 -- 172.21.15.123:0/2999644810 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec037070 con 0x7f31fc0ad150 2024-04-24T05:47:21.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.872+0000 7f31f9400640 1 -- 172.21.15.123:0/2999644810 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ec032440 con 0x7f31fc0ad150 2024-04-24T05:47:21.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.872+0000 7f31f9400640 1 -- 172.21.15.123:0/2999644810 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec032810 con 0x7f31fc0ad150 2024-04-24T05:47:21.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.872+0000 7f3200a00640 1 -- 172.21.15.123:0/2999644810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31fc0ad150 msgr2=0x7f31fc06cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.872+0000 7f3200a00640 1 --2- 172.21.15.123:0/2999644810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31fc0ad150 0x7f31fc06cf30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0afd90 tx=0x7f31ec02a270 comp rx=0 tx=0).stop 2024-04-24T05:47:21.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.872+0000 7f3200a00640 1 -- 172.21.15.123:0/2999644810 shutdown_connections 2024-04-24T05:47:21.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.872+0000 7f3200a00640 1 --2- 172.21.15.123:0/2999644810 >> v2:172.21.15.155:3300/0 conn(0x7f31fc06ba00 0x7f31fc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.872+0000 7f3200a00640 1 --2- 172.21.15.123:0/2999644810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31fc0ad150 0x7f31fc06cf30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.872+0000 7f3200a00640 1 -- 172.21.15.123:0/2999644810 >> 172.21.15.123:0/2999644810 conn(0x7f31fc0b1e70 msgr2=0x7f31fc0b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:21.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.872+0000 7f3200a00640 1 -- 172.21.15.123:0/2999644810 shutdown_connections 2024-04-24T05:47:21.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.872+0000 7f3200a00640 1 -- 172.21.15.123:0/2999644810 wait complete. 2024-04-24T05:47:21.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.874+0000 7f3200a00640 1 Processor -- start 2024-04-24T05:47:21.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.874+0000 7f3200a00640 1 -- start start 2024-04-24T05:47:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.875+0000 7f3200a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31fc0ad150 0x7f31fc08f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.875+0000 7f3200a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31fc0ad530 0x7f31fc152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.875+0000 7f3200a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31fc153050 0x7f31fc15b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.875+0000 7f3200a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc0af360 con 0x7f31fc153050 2024-04-24T05:47:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.875+0000 7f3200a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc0ad960 con 0x7f31fc0ad150 2024-04-24T05:47:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.875+0000 7f3200a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f31fc1f4ca0 con 0x7f31fc0ad530 2024-04-24T05:47:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.875+0000 7f31fa800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31fc153050 0x7f31fc15b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.875+0000 7f31fa800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31fc153050 0x7f31fc15b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33694/0 (socket says 172.21.15.123:33694) 2024-04-24T05:47:21.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.875+0000 7f31fa800640 1 -- 172.21.15.123:0/1282213727 learned_addr learned my addr 172.21.15.123:0/1282213727 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:21.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.876+0000 7f31fa800640 1 -- 172.21.15.123:0/1282213727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31fc0ad530 msgr2=0x7f31fc152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.876+0000 7f31fa800640 1 --2- 172.21.15.123:0/1282213727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31fc0ad530 0x7f31fc152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.876+0000 7f31fa800640 1 -- 172.21.15.123:0/1282213727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31fc0ad150 msgr2=0x7f31fc08f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:21.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.876+0000 7f31fa800640 1 --2- 172.21.15.123:0/1282213727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31fc0ad150 0x7f31fc08f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:21.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.876+0000 7f31fa800640 1 -- 172.21.15.123:0/1282213727 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc150060 con 0x7f31fc153050 2024-04-24T05:47:21.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.876+0000 7f31fa800640 1 --2- 172.21.15.123:0/1282213727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31fc153050 0x7f31fc15b560 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f31f400c780 tx=0x7f31f400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:21.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.876+0000 7f31f2000640 1 -- 172.21.15.123:0/1282213727 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4003b70 con 0x7f31fc153050 2024-04-24T05:47:21.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.876+0000 7f31f2000640 1 -- 172.21.15.123:0/1282213727 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f4003d10 con 0x7f31fc153050 2024-04-24T05:47:21.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.876+0000 7f31f2000640 1 -- 172.21.15.123:0/1282213727 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4005290 con 0x7f31fc153050 2024-04-24T05:47:21.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.876+0000 7f3200a00640 1 -- 172.21.15.123:0/1282213727 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31fc150340 con 0x7f31fc153050 2024-04-24T05:47:21.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.877+0000 7f3200a00640 1 -- 172.21.15.123:0/1282213727 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31fc150900 con 0x7f31fc153050 2024-04-24T05:47:21.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.878+0000 7f3200a00640 1 -- 172.21.15.123:0/1282213727 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31fc0cc880 con 0x7f31fc153050 2024-04-24T05:47:21.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.879+0000 7f31f2000640 1 -- 172.21.15.123:0/1282213727 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31f4005430 con 0x7f31fc153050 2024-04-24T05:47:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.880+0000 7f31f2000640 1 --2- 172.21.15.123:0/1282213727 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f31d806c8f0 0x7f31d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.880+0000 7f31f2000640 1 -- 172.21.15.123:0/1282213727 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f31f408bd50 con 0x7f31fc153050 2024-04-24T05:47:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.880+0000 7f31f9e00640 1 --2- 172.21.15.123:0/1282213727 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f31d806c8f0 0x7f31d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:21.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.881+0000 7f31f9e00640 1 --2- 172.21.15.123:0/1282213727 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f31d806c8f0 0x7f31d806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f31ec034000 tx=0x7f31ec02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:21.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:21.885+0000 7f31f2000640 1 -- 172.21.15.123:0/1282213727 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31fc0cc880 con 0x7f31fc153050 2024-04-24T05:47:22.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.096+0000 7f3200a00640 1 -- 172.21.15.123:0/1282213727 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f31fc068a20 con 0x7f31fc153050 2024-04-24T05:47:22.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.097+0000 7f31f2000640 1 -- 172.21.15.123:0/1282213727 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f31f408f030 con 0x7f31fc153050 2024-04-24T05:47:22.099 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-24T05:47:22.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.100+0000 7f3200a00640 1 -- 172.21.15.123:0/1282213727 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f31d806c8f0 msgr2=0x7f31d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:22.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.100+0000 7f3200a00640 1 --2- 172.21.15.123:0/1282213727 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f31d806c8f0 0x7f31d806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f31ec034000 tx=0x7f31ec02c000 comp rx=0 tx=0).stop 2024-04-24T05:47:22.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.100+0000 7f3200a00640 1 -- 172.21.15.123:0/1282213727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31fc153050 msgr2=0x7f31fc15b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:22.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.100+0000 7f3200a00640 1 --2- 172.21.15.123:0/1282213727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31fc153050 0x7f31fc15b560 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f31f400c780 tx=0x7f31f400cc40 comp rx=0 tx=0).stop 2024-04-24T05:47:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.100+0000 7f3200a00640 1 -- 172.21.15.123:0/1282213727 shutdown_connections 2024-04-24T05:47:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.100+0000 7f3200a00640 1 --2- 172.21.15.123:0/1282213727 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f31d806c8f0 0x7f31d806edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.100+0000 7f3200a00640 1 --2- 172.21.15.123:0/1282213727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31fc153050 0x7f31fc15b560 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.100+0000 7f3200a00640 1 --2- 172.21.15.123:0/1282213727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31fc0ad530 0x7f31fc152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.100+0000 7f3200a00640 1 --2- 172.21.15.123:0/1282213727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31fc0ad150 0x7f31fc08f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.100+0000 7f3200a00640 1 -- 172.21.15.123:0/1282213727 >> 172.21.15.123:0/1282213727 conn(0x7f31fc0b1e70 msgr2=0x7f31fc1e37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.100+0000 7f3200a00640 1 -- 172.21.15.123:0/1282213727 shutdown_connections 2024-04-24T05:47:22.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.101+0000 7f3200a00640 1 -- 172.21.15.123:0/1282213727 wait complete. 2024-04-24T05:47:22.121 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-04-24T05:47:22.121 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T05:47:22.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:22.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.381+0000 7f732fe00640 1 Processor -- start 2024-04-24T05:47:22.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.381+0000 7f732fe00640 1 -- start start 2024-04-24T05:47:22.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.383+0000 7f732fe00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f73280ad860 0x7f73280af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:22.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.383+0000 7f732fe00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f732815cdf0 con 0x7f732815a2a0 2024-04-24T05:47:22.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.383+0000 7f732fe00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f732815d0d0 con 0x7f73280af780 2024-04-24T05:47:22.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.383+0000 7f732fe00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f732815a680 con 0x7f73280ad860 2024-04-24T05:47:22.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.383+0000 7f732d400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f732815a2a0 0x7f73280ad150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43204/0 (socket says 172.21.15.123:43204) 2024-04-24T05:47:22.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.383+0000 7f732d400640 1 -- 172.21.15.123:0/2472009853 learned_addr learned my addr 172.21.15.123:0/2472009853 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:22.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 -- 172.21.15.123:0/2472009853 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1513527043 0 0) 0x7f732815cdf0 con 0x7f732815a2a0 2024-04-24T05:47:22.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 -- 172.21.15.123:0/2472009853 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7310003520 con 0x7f732815a2a0 2024-04-24T05:47:22.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f732ca00640 1 --2- 172.21.15.123:0/2472009853 >> v2:172.21.15.155:3300/0 conn(0x7f73280ad860 0x7f73280af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 -- 172.21.15.123:0/2472009853 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 678581278 0 0) 0x7f7310003520 con 0x7f732815a2a0 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 -- 172.21.15.123:0/2472009853 >> v2:172.21.15.155:3300/0 conn(0x7f73280ad860 msgr2=0x7f73280af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 --2- 172.21.15.123:0/2472009853 >> v2:172.21.15.155:3300/0 conn(0x7f73280ad860 0x7f73280af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 -- 172.21.15.123:0/2472009853 >> v1:172.21.15.155:6789/0 conn(0x7f73280af780 legacy=0x7f732806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 -- 172.21.15.123:0/2472009853 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732815d3b0 con 0x7f732815a2a0 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 -- 172.21.15.123:0/2472009853 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f731c001670 con 0x7f732815a2a0 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 -- 172.21.15.123:0/2472009853 >> v1:172.21.15.123:6789/0 conn(0x7f732815a2a0 legacy=0x7f73280ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 --2- 172.21.15.123:0/2472009853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73100039b0 0x7f7310003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 --2- 172.21.15.123:0/2472009853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7310004800 0x7f7310004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 --2- 172.21.15.123:0/2472009853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73100052e0 0x7f731000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 -- 172.21.15.123:0/2472009853 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f732815a680 con 0x7f7310004800 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 -- 172.21.15.123:0/2472009853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f731000c1b0 con 0x7f73100052e0 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.384+0000 7f7327e00640 1 -- 172.21.15.123:0/2472009853 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f731000c3b0 con 0x7f73100039b0 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.385+0000 7f732ca00640 1 --2- 172.21.15.123:0/2472009853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7310004800 0x7f7310004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:22.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.385+0000 7f732ca00640 1 -- 172.21.15.123:0/2472009853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73100039b0 msgr2=0x7f7310003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:22.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.385+0000 7f732ca00640 1 --2- 172.21.15.123:0/2472009853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73100039b0 0x7f7310003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.385+0000 7f732ca00640 1 -- 172.21.15.123:0/2472009853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73100052e0 msgr2=0x7f731000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:22.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.385+0000 7f732ca00640 1 --2- 172.21.15.123:0/2472009853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73100052e0 0x7f731000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.385+0000 7f732ca00640 1 -- 172.21.15.123:0/2472009853 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7310003520 con 0x7f7310004800 2024-04-24T05:47:22.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.385+0000 7f732de00640 1 -- 172.21.15.123:0/2472009853 reap_dead start 2024-04-24T05:47:22.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.386+0000 7f732ca00640 1 --2- 172.21.15.123:0/2472009853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7310004800 0x7f7310004c80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f731400ba00 tx=0x7f7310005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a4aec97b0f0005 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:22.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.386+0000 7f7327e00640 1 -- 172.21.15.123:0/2472009853 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314030070 con 0x7f7310004800 2024-04-24T05:47:22.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.386+0000 7f7327e00640 1 -- 172.21.15.123:0/2472009853 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731402adb0 con 0x7f7310004800 2024-04-24T05:47:22.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.386+0000 7f7327e00640 1 -- 172.21.15.123:0/2472009853 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731400b600 con 0x7f7310004800 2024-04-24T05:47:22.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.387+0000 7f732fe00640 1 -- 172.21.15.123:0/2472009853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7310004800 msgr2=0x7f7310004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:22.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.387+0000 7f732fe00640 1 --2- 172.21.15.123:0/2472009853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7310004800 0x7f7310004c80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f731400ba00 tx=0x7f7310005830 comp rx=0 tx=0).stop 2024-04-24T05:47:22.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.387+0000 7f732fe00640 1 -- 172.21.15.123:0/2472009853 shutdown_connections 2024-04-24T05:47:22.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.387+0000 7f732fe00640 1 --2- 172.21.15.123:0/2472009853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7310004800 0x7f7310004c80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.387+0000 7f732fe00640 1 -- 172.21.15.123:0/2472009853 >> 172.21.15.123:0/2472009853 conn(0x7f73280b1e70 msgr2=0x7f73280b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:22.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.387+0000 7f732fe00640 1 -- 172.21.15.123:0/2472009853 shutdown_connections 2024-04-24T05:47:22.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.387+0000 7f732fe00640 1 -- 172.21.15.123:0/2472009853 wait complete. 2024-04-24T05:47:22.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.389+0000 7f732fe00640 1 Processor -- start 2024-04-24T05:47:22.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.389+0000 7f732fe00640 1 -- start start 2024-04-24T05:47:22.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.389+0000 7f732fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7310004800 0x7f73280cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:22.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.389+0000 7f732fe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f732806f0f0 0x7f73281f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:22.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.389+0000 7f732fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7328099690 0x7f732815ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:22.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.389+0000 7f732fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73280afb90 con 0x7f7328099690 2024-04-24T05:47:22.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.389+0000 7f732fe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f732815da00 con 0x7f732806f0f0 2024-04-24T05:47:22.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.389+0000 7f732fe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f73280af240 con 0x7f7310004800 2024-04-24T05:47:22.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.390+0000 7f732de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7328099690 0x7f732815ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:22.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.390+0000 7f732de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7328099690 0x7f732815ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33714/0 (socket says 172.21.15.123:33714) 2024-04-24T05:47:22.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.390+0000 7f732de00640 1 -- 172.21.15.123:0/4009109780 learned_addr learned my addr 172.21.15.123:0/4009109780 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:22.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.390+0000 7f732d400640 1 --2- 172.21.15.123:0/4009109780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7310004800 0x7f73280cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:22.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.390+0000 7f732de00640 1 -- 172.21.15.123:0/4009109780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7310004800 msgr2=0x7f73280cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:22.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.390+0000 7f732de00640 1 --2- 172.21.15.123:0/4009109780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7310004800 0x7f73280cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.390+0000 7f732de00640 1 -- 172.21.15.123:0/4009109780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f732806f0f0 msgr2=0x7f73281f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:22.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.390+0000 7f732de00640 1 --2- 172.21.15.123:0/4009109780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f732806f0f0 0x7f73281f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.390+0000 7f732de00640 1 -- 172.21.15.123:0/4009109780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7328072d20 con 0x7f7328099690 2024-04-24T05:47:22.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.390+0000 7f732de00640 1 --2- 172.21.15.123:0/4009109780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7328099690 0x7f732815ef20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7328075290 tx=0x7f7320010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:22.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.391+0000 7f7325600640 1 -- 172.21.15.123:0/4009109780 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732001b070 con 0x7f7328099690 2024-04-24T05:47:22.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.391+0000 7f7325600640 1 -- 172.21.15.123:0/4009109780 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f732000ea50 con 0x7f7328099690 2024-04-24T05:47:22.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.391+0000 7f7325600640 1 -- 172.21.15.123:0/4009109780 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732000ee40 con 0x7f7328099690 2024-04-24T05:47:22.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.391+0000 7f732fe00640 1 -- 172.21.15.123:0/4009109780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7328073000 con 0x7f7328099690 2024-04-24T05:47:22.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.391+0000 7f732fe00640 1 -- 172.21.15.123:0/4009109780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73280735c0 con 0x7f7328099690 2024-04-24T05:47:22.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.392+0000 7f732fe00640 1 -- 172.21.15.123:0/4009109780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7328068a20 con 0x7f7328099690 2024-04-24T05:47:22.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.393+0000 7f7325600640 1 -- 172.21.15.123:0/4009109780 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f732001e070 con 0x7f7328099690 2024-04-24T05:47:22.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.394+0000 7f7325600640 1 --2- 172.21.15.123:0/4009109780 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f730806c920 0x7f730806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:22.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.394+0000 7f7325600640 1 -- 172.21.15.123:0/4009109780 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f732008f4c0 con 0x7f7328099690 2024-04-24T05:47:22.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.394+0000 7f732d400640 1 --2- 172.21.15.123:0/4009109780 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f730806c920 0x7f730806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:22.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.395+0000 7f732d400640 1 --2- 172.21.15.123:0/4009109780 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f730806c920 0x7f730806ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f731c007a70 tx=0x7f731c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:22.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.399+0000 7f7325600640 1 -- 172.21.15.123:0/4009109780 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f732005ce80 con 0x7f7328099690 2024-04-24T05:47:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.600+0000 7f732fe00640 1 -- 172.21.15.123:0/4009109780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f732806fdf0 con 0x7f7328099690 2024-04-24T05:47:22.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.602+0000 7f7325600640 1 -- 172.21.15.123:0/4009109780 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7f7320061180 con 0x7f7328099690 2024-04-24T05:47:22.604 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:22.604 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.604+0000 7f732fe00640 1 -- 172.21.15.123:0/4009109780 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f730806c920 msgr2=0x7f730806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.604+0000 7f732fe00640 1 --2- 172.21.15.123:0/4009109780 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f730806c920 0x7f730806ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f731c007a70 tx=0x7f731c009040 comp rx=0 tx=0).stop 2024-04-24T05:47:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.604+0000 7f732fe00640 1 -- 172.21.15.123:0/4009109780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7328099690 msgr2=0x7f732815ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.604+0000 7f732fe00640 1 --2- 172.21.15.123:0/4009109780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7328099690 0x7f732815ef20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7328075290 tx=0x7f7320010cd0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.605+0000 7f732fe00640 1 -- 172.21.15.123:0/4009109780 shutdown_connections 2024-04-24T05:47:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.605+0000 7f732fe00640 1 --2- 172.21.15.123:0/4009109780 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f730806c920 0x7f730806ede0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.605+0000 7f732fe00640 1 --2- 172.21.15.123:0/4009109780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7328099690 0x7f732815ef20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.605+0000 7f732fe00640 1 --2- 172.21.15.123:0/4009109780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f732806f0f0 0x7f73281f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.605+0000 7f732fe00640 1 --2- 172.21.15.123:0/4009109780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7310004800 0x7f73280cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.605+0000 7f732fe00640 1 -- 172.21.15.123:0/4009109780 >> 172.21.15.123:0/4009109780 conn(0x7f73280b1e70 msgr2=0x7f732815af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.605+0000 7f732fe00640 1 -- 172.21.15.123:0/4009109780 shutdown_connections 2024-04-24T05:47:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.605+0000 7f732fe00640 1 -- 172.21.15.123:0/4009109780 wait complete. 2024-04-24T05:47:22.623 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:47:22.624 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:47:22.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.811+0000 7f6d61200640 1 Processor -- start 2024-04-24T05:47:22.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.811+0000 7f6d61200640 1 -- start start 2024-04-24T05:47:22.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.813+0000 7f6d61200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6d5c15b980 0x7f6d5c15bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:22.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.813+0000 7f6d61200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6d5c15c2a0 0x7f6d5c0aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:22.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.813+0000 7f6d61200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5c06dbc0 con 0x7f6d5c0af040 2024-04-24T05:47:22.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.813+0000 7f6d61200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6d5c071b80 con 0x7f6d5c15c2a0 2024-04-24T05:47:22.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.813+0000 7f6d61200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6d5c06de30 con 0x7f6d5c15b980 2024-04-24T05:47:22.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.813+0000 7f6d5aa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6d5c15c2a0 0x7f6d5c0aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:22.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.813+0000 7f6d5be00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6d5c0af040 0x7f6d5c06e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43208/0 (socket says 172.21.15.123:43208) 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.813+0000 7f6d5be00640 1 -- 172.21.15.123:0/3501299930 learned_addr learned my addr 172.21.15.123:0/3501299930 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.813+0000 7f6d5b400640 1 --2- 172.21.15.123:0/3501299930 >> v2:172.21.15.155:3300/0 conn(0x7f6d5c15b980 0x7f6d5c15bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.813+0000 7f6d5aa00640 1 -- 172.21.15.123:0/3501299930 >> v2:172.21.15.155:3300/0 conn(0x7f6d5c15b980 msgr2=0x7f6d5c15bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.813+0000 7f6d5aa00640 1 --2- 172.21.15.123:0/3501299930 >> v2:172.21.15.155:3300/0 conn(0x7f6d5c15b980 0x7f6d5c15bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.813+0000 7f6d5aa00640 1 -- 172.21.15.123:0/3501299930 >> v1:172.21.15.123:6789/0 conn(0x7f6d5c0af040 legacy=0x7f6d5c06e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.813+0000 7f6d5aa00640 1 -- 172.21.15.123:0/3501299930 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c06e110 con 0x7f6d5c15c2a0 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.813+0000 7f6d5a000640 1 -- 172.21.15.123:0/3501299930 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2982333590 0 0) 0x7f6d5c06dbc0 con 0x7f6d5c0af040 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.814+0000 7f6d5aa00640 1 --2- 172.21.15.123:0/3501299930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d5c15c2a0 0x7f6d5c0aeb00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c06c260 tx=0x7f6d4802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2c2fa3cb277374 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.815+0000 7f6d5a000640 1 -- 172.21.15.123:0/3501299930 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48037070 con 0x7f6d5c15c2a0 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.815+0000 7f6d5a000640 1 -- 172.21.15.123:0/3501299930 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d48032440 con 0x7f6d5c15c2a0 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.815+0000 7f6d5a000640 1 -- 172.21.15.123:0/3501299930 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48032810 con 0x7f6d5c15c2a0 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.815+0000 7f6d61200640 1 -- 172.21.15.123:0/3501299930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d5c15c2a0 msgr2=0x7f6d5c0aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.815+0000 7f6d61200640 1 --2- 172.21.15.123:0/3501299930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d5c15c2a0 0x7f6d5c0aeb00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c06c260 tx=0x7f6d4802a1a0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.815+0000 7f6d61200640 1 -- 172.21.15.123:0/3501299930 shutdown_connections 2024-04-24T05:47:22.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.815+0000 7f6d61200640 1 --2- 172.21.15.123:0/3501299930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d5c15c2a0 0x7f6d5c0aeb00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.815+0000 7f6d61200640 1 --2- 172.21.15.123:0/3501299930 >> v2:172.21.15.155:3300/0 conn(0x7f6d5c15b980 0x7f6d5c15bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.815+0000 7f6d61200640 1 -- 172.21.15.123:0/3501299930 >> 172.21.15.123:0/3501299930 conn(0x7f6d5c0b1cf0 msgr2=0x7f6d5c0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.815+0000 7f6d61200640 1 -- 172.21.15.123:0/3501299930 shutdown_connections 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.815+0000 7f6d61200640 1 -- 172.21.15.123:0/3501299930 wait complete. 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.817+0000 7f6d61200640 1 Processor -- start 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.817+0000 7f6d61200640 1 -- start start 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.818+0000 7f6d61200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d5c0af040 0x7f6d5c14c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.818+0000 7f6d61200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d5c15b980 0x7f6d5c09c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.818+0000 7f6d61200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d5c15c2a0 0x7f6d5c09cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.818+0000 7f6d61200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c0af830 con 0x7f6d5c0af040 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.818+0000 7f6d61200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c15c730 con 0x7f6d5c15c2a0 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.818+0000 7f6d61200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5c0af4d0 con 0x7f6d5c15b980 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.818+0000 7f6d5b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d5c0af040 0x7f6d5c14c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.818+0000 7f6d5b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d5c0af040 0x7f6d5c14c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33732/0 (socket says 172.21.15.123:33732) 2024-04-24T05:47:22.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.818+0000 7f6d5b400640 1 -- 172.21.15.123:0/2476143276 learned_addr learned my addr 172.21.15.123:0/2476143276 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.819+0000 7f6d5b400640 1 -- 172.21.15.123:0/2476143276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d5c15b980 msgr2=0x7f6d5c09c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.819+0000 7f6d5b400640 1 --2- 172.21.15.123:0/2476143276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d5c15b980 0x7f6d5c09c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.819+0000 7f6d5b400640 1 -- 172.21.15.123:0/2476143276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d5c15c2a0 msgr2=0x7f6d5c09cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.819+0000 7f6d5b400640 1 --2- 172.21.15.123:0/2476143276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d5c15c2a0 0x7f6d5c09cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.819+0000 7f6d5b400640 1 -- 172.21.15.123:0/2476143276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c09a3a0 con 0x7f6d5c0af040 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.819+0000 7f6d5b400640 1 --2- 172.21.15.123:0/2476143276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d5c0af040 0x7f6d5c14c640 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00c820 tx=0x7f6d4c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.819+0000 7f6d53400640 1 -- 172.21.15.123:0/2476143276 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c015070 con 0x7f6d5c0af040 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.819+0000 7f6d53400640 1 -- 172.21.15.123:0/2476143276 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d4c004240 con 0x7f6d5c0af040 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.819+0000 7f6d61200640 1 -- 172.21.15.123:0/2476143276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5c09a680 con 0x7f6d5c0af040 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.819+0000 7f6d53400640 1 -- 172.21.15.123:0/2476143276 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c00ce10 con 0x7f6d5c0af040 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.819+0000 7f6d61200640 1 -- 172.21.15.123:0/2476143276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5c09ac40 con 0x7f6d5c0af040 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.820+0000 7f6d61200640 1 -- 172.21.15.123:0/2476143276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5c068a20 con 0x7f6d5c0af040 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.821+0000 7f6d53400640 1 -- 172.21.15.123:0/2476143276 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d4c007830 con 0x7f6d5c0af040 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.822+0000 7f6d53400640 1 --2- 172.21.15.123:0/2476143276 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6d3c06c8f0 0x7f6d3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:22.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.822+0000 7f6d53400640 1 -- 172.21.15.123:0/2476143276 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f6d4c08be80 con 0x7f6d5c0af040 2024-04-24T05:47:22.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.823+0000 7f6d5aa00640 1 --2- 172.21.15.123:0/2476143276 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6d3c06c8f0 0x7f6d3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:22.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.823+0000 7f6d5aa00640 1 --2- 172.21.15.123:0/2476143276 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6d3c06c8f0 0x7f6d3c06edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d48034000 tx=0x7f6d4802c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:22.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:22.827+0000 7f6d53400640 1 -- 172.21.15.123:0/2476143276 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d4c059e90 con 0x7f6d5c0af040 2024-04-24T05:47:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.016+0000 7f6d61200640 1 -- 172.21.15.123:0/2476143276 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6d5c0addf0 con 0x7f6d5c0af040 2024-04-24T05:47:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.017+0000 7f6d53400640 1 -- 172.21.15.123:0/2476143276 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f6d4c059790 con 0x7f6d5c0af040 2024-04-24T05:47:23.018 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:47:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.019+0000 7f6d61200640 1 -- 172.21.15.123:0/2476143276 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6d3c06c8f0 msgr2=0x7f6d3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.019+0000 7f6d61200640 1 --2- 172.21.15.123:0/2476143276 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6d3c06c8f0 0x7f6d3c06edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d48034000 tx=0x7f6d4802c000 comp rx=0 tx=0).stop 2024-04-24T05:47:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.019+0000 7f6d61200640 1 -- 172.21.15.123:0/2476143276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d5c0af040 msgr2=0x7f6d5c14c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.019+0000 7f6d61200640 1 --2- 172.21.15.123:0/2476143276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d5c0af040 0x7f6d5c14c640 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00c820 tx=0x7f6d4c00cce0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.019+0000 7f6d61200640 1 -- 172.21.15.123:0/2476143276 shutdown_connections 2024-04-24T05:47:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.019+0000 7f6d61200640 1 --2- 172.21.15.123:0/2476143276 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6d3c06c8f0 0x7f6d3c06edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.019+0000 7f6d61200640 1 --2- 172.21.15.123:0/2476143276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d5c15c2a0 0x7f6d5c09cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.019+0000 7f6d61200640 1 --2- 172.21.15.123:0/2476143276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d5c15b980 0x7f6d5c09c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.019+0000 7f6d61200640 1 --2- 172.21.15.123:0/2476143276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d5c0af040 0x7f6d5c14c640 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.019+0000 7f6d61200640 1 -- 172.21.15.123:0/2476143276 >> 172.21.15.123:0/2476143276 conn(0x7f6d5c0b1cf0 msgr2=0x7f6d5c06e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.019+0000 7f6d61200640 1 -- 172.21.15.123:0/2476143276 shutdown_connections 2024-04-24T05:47:23.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.019+0000 7f6d61200640 1 -- 172.21.15.123:0/2476143276 wait complete. 2024-04-24T05:47:23.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:47:23.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.231+0000 7fbf03200640 1 Processor -- start 2024-04-24T05:47:23.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.232+0000 7fbf03200640 1 -- start start 2024-04-24T05:47:23.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.233+0000 7fbf03200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbefc15c2f0 0x7fbefc141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.233+0000 7fbf03200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbefc0cc5d0 0x7fbefc0acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.233+0000 7fbf03200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbefc0ad510 0x7fbefc15ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.233+0000 7fbf03200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbefc06bd30 con 0x7fbefc0ad510 2024-04-24T05:47:23.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.233+0000 7fbf03200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbefc06ba00 con 0x7fbefc0cc5d0 2024-04-24T05:47:23.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.233+0000 7fbf03200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbefc142530 con 0x7fbefc15c2f0 2024-04-24T05:47:23.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.233+0000 7fbf02800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbefc0ad510 0x7fbefc15ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:23.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.233+0000 7fbf02800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbefc0ad510 0x7fbefc15ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33748/0 (socket says 172.21.15.123:33748) 2024-04-24T05:47:23.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.234+0000 7fbf02800640 1 -- 172.21.15.123:0/1720555287 learned_addr learned my addr 172.21.15.123:0/1720555287 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:23.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.234+0000 7fbf01400640 1 --2- 172.21.15.123:0/1720555287 >> v2:172.21.15.155:3300/0 conn(0x7fbefc0cc5d0 0x7fbefc0acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:23.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.234+0000 7fbf02800640 1 -- 172.21.15.123:0/1720555287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbefc15c2f0 msgr2=0x7fbefc141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:23.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.234+0000 7fbf01e00640 1 --2- 172.21.15.123:0/1720555287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbefc15c2f0 0x7fbefc141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:23.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.234+0000 7fbf02800640 1 --2- 172.21.15.123:0/1720555287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbefc15c2f0 0x7fbefc141ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.234+0000 7fbf02800640 1 -- 172.21.15.123:0/1720555287 >> v2:172.21.15.155:3300/0 conn(0x7fbefc0cc5d0 msgr2=0x7fbefc0acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:23.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.234+0000 7fbf02800640 1 --2- 172.21.15.123:0/1720555287 >> v2:172.21.15.155:3300/0 conn(0x7fbefc0cc5d0 0x7fbefc0acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.234+0000 7fbf02800640 1 -- 172.21.15.123:0/1720555287 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbefc142810 con 0x7fbefc0ad510 2024-04-24T05:47:23.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.234+0000 7fbf01400640 1 --2- 172.21.15.123:0/1720555287 >> v2:172.21.15.155:3300/0 conn(0x7fbefc0cc5d0 0x7fbefc0acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:23.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.235+0000 7fbf02800640 1 --2- 172.21.15.123:0/1720555287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbefc0ad510 0x7fbefc15ffd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0d11e0 tx=0x7fbef802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cdbe9b2005329 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:23.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.235+0000 7fbf00a00640 1 -- 172.21.15.123:0/1720555287 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8036070 con 0x7fbefc0ad510 2024-04-24T05:47:23.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.235+0000 7fbf00a00640 1 -- 172.21.15.123:0/1720555287 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef8031600 con 0x7fbefc0ad510 2024-04-24T05:47:23.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.235+0000 7fbf00a00640 1 -- 172.21.15.123:0/1720555287 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef80319d0 con 0x7fbefc0ad510 2024-04-24T05:47:23.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.236+0000 7fbf03200640 1 -- 172.21.15.123:0/1720555287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbefc0ad510 msgr2=0x7fbefc15ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:23.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.236+0000 7fbf03200640 1 --2- 172.21.15.123:0/1720555287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbefc0ad510 0x7fbefc15ffd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0d11e0 tx=0x7fbef802a610 comp rx=0 tx=0).stop 2024-04-24T05:47:23.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.236+0000 7fbf03200640 1 -- 172.21.15.123:0/1720555287 shutdown_connections 2024-04-24T05:47:23.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.236+0000 7fbf03200640 1 --2- 172.21.15.123:0/1720555287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbefc0ad510 0x7fbefc15ffd0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.236+0000 7fbf03200640 1 --2- 172.21.15.123:0/1720555287 >> v2:172.21.15.155:3300/0 conn(0x7fbefc0cc5d0 0x7fbefc0acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.236+0000 7fbf03200640 1 --2- 172.21.15.123:0/1720555287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbefc15c2f0 0x7fbefc141ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.236+0000 7fbf03200640 1 -- 172.21.15.123:0/1720555287 >> 172.21.15.123:0/1720555287 conn(0x7fbefc0b1cf0 msgr2=0x7fbefc0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:23.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.236+0000 7fbf03200640 1 -- 172.21.15.123:0/1720555287 shutdown_connections 2024-04-24T05:47:23.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.236+0000 7fbf03200640 1 -- 172.21.15.123:0/1720555287 wait complete. 2024-04-24T05:47:23.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.238+0000 7fbf03200640 1 Processor -- start 2024-04-24T05:47:23.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.238+0000 7fbf03200640 1 -- start start 2024-04-24T05:47:23.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.239+0000 7fbf03200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbefc0ad510 0x7fbefc099ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.239+0000 7fbf03200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbefc0cc5d0 0x7fbefc0cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.239+0000 7fbf03200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbefc15c2f0 0x7fbefc0ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.239+0000 7fbf03200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc15c6d0 con 0x7fbefc0cc5d0 2024-04-24T05:47:23.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.239+0000 7fbf03200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc142220 con 0x7fbefc0ad510 2024-04-24T05:47:23.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.239+0000 7fbf03200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbefc15c850 con 0x7fbefc15c2f0 2024-04-24T05:47:23.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.239+0000 7fbf01400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbefc0cc5d0 0x7fbefc0cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:23.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.239+0000 7fbf01400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbefc0cc5d0 0x7fbefc0cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33758/0 (socket says 172.21.15.123:33758) 2024-04-24T05:47:23.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.239+0000 7fbf01400640 1 -- 172.21.15.123:0/927604808 learned_addr learned my addr 172.21.15.123:0/927604808 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:23.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.240+0000 7fbf01e00640 1 --2- 172.21.15.123:0/927604808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbefc0ad510 0x7fbefc099ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:23.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.240+0000 7fbf02800640 1 --2- 172.21.15.123:0/927604808 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbefc15c2f0 0x7fbefc0ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:23.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.240+0000 7fbf01400640 1 -- 172.21.15.123:0/927604808 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbefc15c2f0 msgr2=0x7fbefc0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:23.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.240+0000 7fbf01400640 1 --2- 172.21.15.123:0/927604808 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbefc15c2f0 0x7fbefc0ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.240+0000 7fbf01400640 1 -- 172.21.15.123:0/927604808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbefc0ad510 msgr2=0x7fbefc099ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:23.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.240+0000 7fbf01400640 1 --2- 172.21.15.123:0/927604808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbefc0ad510 0x7fbefc099ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.240+0000 7fbf01400640 1 -- 172.21.15.123:0/927604808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbefc144420 con 0x7fbefc0cc5d0 2024-04-24T05:47:23.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.240+0000 7fbf01400640 1 --2- 172.21.15.123:0/927604808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbefc0cc5d0 0x7fbefc0cca80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbee800c820 tx=0x7fbee800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:23.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.240+0000 7fbef6000640 1 -- 172.21.15.123:0/927604808 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8015070 con 0x7fbefc0cc5d0 2024-04-24T05:47:23.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.240+0000 7fbef6000640 1 -- 172.21.15.123:0/927604808 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee8007a30 con 0x7fbefc0cc5d0 2024-04-24T05:47:23.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.240+0000 7fbf03200640 1 -- 172.21.15.123:0/927604808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbefc144700 con 0x7fbefc0cc5d0 2024-04-24T05:47:23.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.241+0000 7fbef6000640 1 -- 172.21.15.123:0/927604808 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8007de0 con 0x7fbefc0cc5d0 2024-04-24T05:47:23.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.241+0000 7fbf03200640 1 -- 172.21.15.123:0/927604808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbefc144cc0 con 0x7fbefc0cc5d0 2024-04-24T05:47:23.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.242+0000 7fbf03200640 1 -- 172.21.15.123:0/927604808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbefc068a20 con 0x7fbefc0cc5d0 2024-04-24T05:47:23.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.243+0000 7fbef6000640 1 -- 172.21.15.123:0/927604808 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbee8018070 con 0x7fbefc0cc5d0 2024-04-24T05:47:23.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.244+0000 7fbef6000640 1 --2- 172.21.15.123:0/927604808 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbedc06c8f0 0x7fbedc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.244+0000 7fbef6000640 1 -- 172.21.15.123:0/927604808 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fbee808be70 con 0x7fbefc0cc5d0 2024-04-24T05:47:23.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.244+0000 7fbf01e00640 1 --2- 172.21.15.123:0/927604808 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbedc06c8f0 0x7fbedc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:23.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.245+0000 7fbf01e00640 1 --2- 172.21.15.123:0/927604808 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbedc06c8f0 0x7fbedc06edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbef000a8f0 tx=0x7fbef0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:23.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.248+0000 7fbef6000640 1 -- 172.21.15.123:0/927604808 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbee8059e80 con 0x7fbefc0cc5d0 2024-04-24T05:47:23.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.448+0000 7fbf03200640 1 -- 172.21.15.123:0/927604808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbefc0d0d60 con 0x7fbefc0cc5d0 2024-04-24T05:47:23.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.449+0000 7fbef6000640 1 -- 172.21.15.123:0/927604808 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fbee8059780 con 0x7fbefc0cc5d0 2024-04-24T05:47:23.451 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:23.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.452+0000 7fbf03200640 1 -- 172.21.15.123:0/927604808 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbedc06c8f0 msgr2=0x7fbedc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:23.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.452+0000 7fbf03200640 1 --2- 172.21.15.123:0/927604808 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbedc06c8f0 0x7fbedc06edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbef000a8f0 tx=0x7fbef0008040 comp rx=0 tx=0).stop 2024-04-24T05:47:23.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.452+0000 7fbf03200640 1 -- 172.21.15.123:0/927604808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbefc0cc5d0 msgr2=0x7fbefc0cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:23.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.452+0000 7fbf03200640 1 --2- 172.21.15.123:0/927604808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbefc0cc5d0 0x7fbefc0cca80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbee800c820 tx=0x7fbee800cce0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.452+0000 7fbf03200640 1 -- 172.21.15.123:0/927604808 shutdown_connections 2024-04-24T05:47:23.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.452+0000 7fbf03200640 1 --2- 172.21.15.123:0/927604808 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbedc06c8f0 0x7fbedc06edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.452+0000 7fbf03200640 1 --2- 172.21.15.123:0/927604808 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbefc15c2f0 0x7fbefc0ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.452+0000 7fbf03200640 1 --2- 172.21.15.123:0/927604808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbefc0cc5d0 0x7fbefc0cca80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.452+0000 7fbf03200640 1 --2- 172.21.15.123:0/927604808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbefc0ad510 0x7fbefc099ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.452+0000 7fbf03200640 1 -- 172.21.15.123:0/927604808 >> 172.21.15.123:0/927604808 conn(0x7fbefc0b1cf0 msgr2=0x7fbefc0b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:23.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.452+0000 7fbf03200640 1 -- 172.21.15.123:0/927604808 shutdown_connections 2024-04-24T05:47:23.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.452+0000 7fbf03200640 1 -- 172.21.15.123:0/927604808 wait complete. 2024-04-24T05:47:23.474 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:47:23.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.676+0000 7fd917800640 1 Processor -- start 2024-04-24T05:47:23.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.677+0000 7fd917800640 1 -- start start 2024-04-24T05:47:23.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.678+0000 7fd917800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd91015c6e0 0x7fd9100b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.678+0000 7fd917800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd91015cac0 con 0x7fd9100b6530 2024-04-24T05:47:23.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.678+0000 7fd917800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9100cbdf0 con 0x7fd91006c340 2024-04-24T05:47:23.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.678+0000 7fd917800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd9100cbba0 con 0x7fd91015c6e0 2024-04-24T05:47:23.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.678+0000 7fd90fe00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd9100b6530 0x7fd91006bc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43224/0 (socket says 172.21.15.123:43224) 2024-04-24T05:47:23.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.678+0000 7fd90fe00640 1 -- 172.21.15.123:0/2333294983 learned_addr learned my addr 172.21.15.123:0/2333294983 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:23.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 -- 172.21.15.123:0/2333294983 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 199400516 0 0) 0x7fd91015cac0 con 0x7fd9100b6530 2024-04-24T05:47:23.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 -- 172.21.15.123:0/2333294983 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8f4003520 con 0x7fd9100b6530 2024-04-24T05:47:23.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd914e00640 1 --2- 172.21.15.123:0/2333294983 >> v2:172.21.15.155:3300/0 conn(0x7fd91015c6e0 0x7fd9100b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:23.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 -- 172.21.15.123:0/2333294983 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 656381068 0 0) 0x7fd8f4003520 con 0x7fd9100b6530 2024-04-24T05:47:23.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 -- 172.21.15.123:0/2333294983 >> v2:172.21.15.155:3300/0 conn(0x7fd91015c6e0 msgr2=0x7fd9100b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:23.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 --2- 172.21.15.123:0/2333294983 >> v2:172.21.15.155:3300/0 conn(0x7fd91015c6e0 0x7fd9100b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 -- 172.21.15.123:0/2333294983 >> v1:172.21.15.155:6789/0 conn(0x7fd91006c340 legacy=0x7fd91006e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:23.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 -- 172.21.15.123:0/2333294983 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9100cc0d0 con 0x7fd9100b6530 2024-04-24T05:47:23.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 -- 172.21.15.123:0/2333294983 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7fd8fc001670 con 0x7fd9100b6530 2024-04-24T05:47:23.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 -- 172.21.15.123:0/2333294983 >> v1:172.21.15.123:6789/0 conn(0x7fd9100b6530 legacy=0x7fd91006bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:23.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 --2- 172.21.15.123:0/2333294983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8f40039b0 0x7fd8f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 --2- 172.21.15.123:0/2333294983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f4004800 0x7fd8f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 --2- 172.21.15.123:0/2333294983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f40051c0 0x7fd8f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 -- 172.21.15.123:0/2333294983 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9100cbba0 con 0x7fd8f40051c0 2024-04-24T05:47:23.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 -- 172.21.15.123:0/2333294983 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f400c110 con 0x7fd8f4004800 2024-04-24T05:47:23.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd90f400640 1 -- 172.21.15.123:0/2333294983 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f400c310 con 0x7fd8f40039b0 2024-04-24T05:47:23.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.679+0000 7fd915800640 1 --2- 172.21.15.123:0/2333294983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f40051c0 0x7fd8f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:23.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.680+0000 7fd914e00640 1 --2- 172.21.15.123:0/2333294983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8f40039b0 0x7fd8f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:23.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.680+0000 7fd90fe00640 1 --2- 172.21.15.123:0/2333294983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f4004800 0x7fd8f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:23.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.680+0000 7fd915800640 1 -- 172.21.15.123:0/2333294983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8f40039b0 msgr2=0x7fd8f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:23.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.680+0000 7fd915800640 1 --2- 172.21.15.123:0/2333294983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8f40039b0 0x7fd8f4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.680+0000 7fd915800640 1 -- 172.21.15.123:0/2333294983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f4004800 msgr2=0x7fd8f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:23.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.680+0000 7fd915800640 1 --2- 172.21.15.123:0/2333294983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f4004800 0x7fd8f4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.680+0000 7fd915800640 1 -- 172.21.15.123:0/2333294983 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f4003520 con 0x7fd8f40051c0 2024-04-24T05:47:23.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.681+0000 7fd915800640 1 -- 172.21.15.123:0/2333294983 reap_dead start 2024-04-24T05:47:23.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.681+0000 7fd915800640 1 --2- 172.21.15.123:0/2333294983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f40051c0 0x7fd8f400b9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd908001e50 tx=0x7fd8f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e032e994df938f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:23.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.681+0000 7fd90f400640 1 -- 172.21.15.123:0/2333294983 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908001a70 con 0x7fd8f40051c0 2024-04-24T05:47:23.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.681+0000 7fd90f400640 1 -- 172.21.15.123:0/2333294983 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd908003970 con 0x7fd8f40051c0 2024-04-24T05:47:23.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.681+0000 7fd90f400640 1 -- 172.21.15.123:0/2333294983 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908003d60 con 0x7fd8f40051c0 2024-04-24T05:47:23.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.682+0000 7fd917800640 1 -- 172.21.15.123:0/2333294983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f40051c0 msgr2=0x7fd8f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:23.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.682+0000 7fd917800640 1 --2- 172.21.15.123:0/2333294983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f40051c0 0x7fd8f400b9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd908001e50 tx=0x7fd8f4005830 comp rx=0 tx=0).stop 2024-04-24T05:47:23.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.682+0000 7fd917800640 1 -- 172.21.15.123:0/2333294983 shutdown_connections 2024-04-24T05:47:23.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.682+0000 7fd917800640 1 --2- 172.21.15.123:0/2333294983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f40051c0 0x7fd8f400b9c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.682+0000 7fd917800640 1 -- 172.21.15.123:0/2333294983 >> 172.21.15.123:0/2333294983 conn(0x7fd9100cd5c0 msgr2=0x7fd9100cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:23.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.682+0000 7fd917800640 1 -- 172.21.15.123:0/2333294983 shutdown_connections 2024-04-24T05:47:23.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.682+0000 7fd917800640 1 -- 172.21.15.123:0/2333294983 wait complete. 2024-04-24T05:47:23.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.684+0000 7fd917800640 1 Processor -- start 2024-04-24T05:47:23.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.684+0000 7fd917800640 1 -- start start 2024-04-24T05:47:23.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.685+0000 7fd917800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f40051c0 0x7fd910154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.685+0000 7fd917800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd91015a9c0 0x7fd9100b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.685+0000 7fd917800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd91015ae40 0x7fd9100c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.685+0000 7fd917800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9100cc690 con 0x7fd8f40051c0 2024-04-24T05:47:23.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.685+0000 7fd917800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd9100cca00 con 0x7fd91015ae40 2024-04-24T05:47:23.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.685+0000 7fd917800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd9100b5ff0 con 0x7fd91015a9c0 2024-04-24T05:47:23.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.685+0000 7fd914e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f40051c0 0x7fd910154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:23.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.685+0000 7fd914e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f40051c0 0x7fd910154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33776/0 (socket says 172.21.15.123:33776) 2024-04-24T05:47:23.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.685+0000 7fd914e00640 1 -- 172.21.15.123:0/153952553 learned_addr learned my addr 172.21.15.123:0/153952553 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:23.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.685+0000 7fd914e00640 1 -- 172.21.15.123:0/153952553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd91015a9c0 msgr2=0x7fd9100b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:23.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.685+0000 7fd914e00640 1 --2- 172.21.15.123:0/153952553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd91015a9c0 0x7fd9100b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.685+0000 7fd914e00640 1 -- 172.21.15.123:0/153952553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd91015ae40 msgr2=0x7fd9100c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:23.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.685+0000 7fd914e00640 1 --2- 172.21.15.123:0/153952553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd91015ae40 0x7fd9100c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.685+0000 7fd914e00640 1 -- 172.21.15.123:0/153952553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd910159920 con 0x7fd8f40051c0 2024-04-24T05:47:23.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.686+0000 7fd914e00640 1 --2- 172.21.15.123:0/153952553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f40051c0 0x7fd910154fb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd90000e810 tx=0x7fd90000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:23.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.686+0000 7fd90cc00640 1 -- 172.21.15.123:0/153952553 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900017070 con 0x7fd8f40051c0 2024-04-24T05:47:23.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.686+0000 7fd90cc00640 1 -- 172.21.15.123:0/153952553 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd90000c6d0 con 0x7fd8f40051c0 2024-04-24T05:47:23.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.686+0000 7fd90cc00640 1 -- 172.21.15.123:0/153952553 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90000f040 con 0x7fd8f40051c0 2024-04-24T05:47:23.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.686+0000 7fd917800640 1 -- 172.21.15.123:0/153952553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd910159c00 con 0x7fd8f40051c0 2024-04-24T05:47:23.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.686+0000 7fd917800640 1 -- 172.21.15.123:0/153952553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd91015a1c0 con 0x7fd8f40051c0 2024-04-24T05:47:23.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.687+0000 7fd917800640 1 -- 172.21.15.123:0/153952553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd910069130 con 0x7fd8f40051c0 2024-04-24T05:47:23.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.688+0000 7fd90cc00640 1 -- 172.21.15.123:0/153952553 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd90001a070 con 0x7fd8f40051c0 2024-04-24T05:47:23.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.689+0000 7fd90cc00640 1 --2- 172.21.15.123:0/153952553 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd8f006c8f0 0x7fd8f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:23.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.689+0000 7fd90cc00640 1 -- 172.21.15.123:0/153952553 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fd90008bbd0 con 0x7fd8f40051c0 2024-04-24T05:47:23.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.689+0000 7fd90fe00640 1 --2- 172.21.15.123:0/153952553 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd8f006c8f0 0x7fd8f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:23.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.690+0000 7fd90fe00640 1 --2- 172.21.15.123:0/153952553 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd8f006c8f0 0x7fd8f006edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc001eb0 tx=0x7fd8fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:23.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.693+0000 7fd90cc00640 1 -- 172.21.15.123:0/153952553 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd900059bf0 con 0x7fd8f40051c0 2024-04-24T05:47:23.808 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:47:23.814 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:47:23.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.896+0000 7fd917800640 1 -- 172.21.15.123:0/153952553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd910070010 con 0x7fd8f40051c0 2024-04-24T05:47:23.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.897+0000 7fd90cc00640 1 -- 172.21.15.123:0/153952553 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fd9000594f0 con 0x7fd8f40051c0 2024-04-24T05:47:23.899 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:47:23.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.899+0000 7fd917800640 1 -- 172.21.15.123:0/153952553 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd8f006c8f0 msgr2=0x7fd8f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:23.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.899+0000 7fd917800640 1 --2- 172.21.15.123:0/153952553 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd8f006c8f0 0x7fd8f006edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc001eb0 tx=0x7fd8fc009040 comp rx=0 tx=0).stop 2024-04-24T05:47:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.899+0000 7fd917800640 1 -- 172.21.15.123:0/153952553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f40051c0 msgr2=0x7fd910154fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.899+0000 7fd917800640 1 --2- 172.21.15.123:0/153952553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f40051c0 0x7fd910154fb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd90000e810 tx=0x7fd90000ecd0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.899+0000 7fd917800640 1 -- 172.21.15.123:0/153952553 shutdown_connections 2024-04-24T05:47:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.899+0000 7fd917800640 1 --2- 172.21.15.123:0/153952553 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd8f006c8f0 0x7fd8f006edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.899+0000 7fd917800640 1 --2- 172.21.15.123:0/153952553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd91015ae40 0x7fd9100c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.899+0000 7fd917800640 1 --2- 172.21.15.123:0/153952553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd91015a9c0 0x7fd9100b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.899+0000 7fd917800640 1 --2- 172.21.15.123:0/153952553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f40051c0 0x7fd910154fb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.899+0000 7fd917800640 1 -- 172.21.15.123:0/153952553 >> 172.21.15.123:0/153952553 conn(0x7fd9100cd5c0 msgr2=0x7fd910156ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.900+0000 7fd917800640 1 -- 172.21.15.123:0/153952553 shutdown_connections 2024-04-24T05:47:23.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:23.900+0000 7fd917800640 1 -- 172.21.15.123:0/153952553 wait complete. 2024-04-24T05:47:23.920 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:47:24.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.124+0000 7ff503000640 1 Processor -- start 2024-04-24T05:47:24.125 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.124+0000 7ff503000640 1 -- start start 2024-04-24T05:47:24.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.125+0000 7ff503000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4fc0cc770 0x7ff4fc0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:24.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.125+0000 7ff503000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff4fc0cfc60 0x7ff4fc0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:24.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.125+0000 7ff503000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4fc06c4b0 con 0x7ff4fc15aff0 2024-04-24T05:47:24.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.125+0000 7ff503000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff4fc0d08e0 con 0x7ff4fc0cfc60 2024-04-24T05:47:24.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.125+0000 7ff503000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4fc06c790 con 0x7ff4fc0cc770 2024-04-24T05:47:24.127 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.126+0000 7ff501000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff4fc0cfc60 0x7ff4fc0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:24.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.126+0000 7ff4f3400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4fc0cc770 0x7ff4fc0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:24.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.126+0000 7ff4fbe00640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7ff4fc15aff0 0x7ff4fc0d01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.123:34576/0 (socket says 172.21.15.123:34576) 2024-04-24T05:47:24.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.126+0000 7ff4fbe00640 1 -- 172.21.15.123:0/804733946 learned_addr learned my addr 172.21.15.123:0/804733946 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:24.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.127+0000 7ff501000640 1 -- 172.21.15.123:0/804733946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4fc0cc770 msgr2=0x7ff4fc0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:24.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.127+0000 7ff501000640 1 --2- 172.21.15.123:0/804733946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4fc0cc770 0x7ff4fc0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.127+0000 7ff501000640 1 -- 172.21.15.123:0/804733946 >> v1:172.21.15.155:6789/0 conn(0x7ff4fc15aff0 legacy=0x7ff4fc0d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:24.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.127+0000 7ff501000640 1 -- 172.21.15.123:0/804733946 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4fc06ca70 con 0x7ff4fc0cfc60 2024-04-24T05:47:24.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.127+0000 7ff4f3400640 1 --2- 172.21.15.123:0/804733946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4fc0cc770 0x7ff4fc0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:47:24.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.128+0000 7ff501000640 1 --2- 172.21.15.123:0/804733946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4fc0cfc60 0x7ff4fc0ccbc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009d90 tx=0x7ff4f402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7045306f5bcaca23 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:24.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.129+0000 7ff4fb400640 1 -- 172.21.15.123:0/804733946 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4031600 con 0x7ff4fc0cfc60 2024-04-24T05:47:24.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.129+0000 7ff4fb400640 1 -- 172.21.15.123:0/804733946 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f40317a0 con 0x7ff4fc0cfc60 2024-04-24T05:47:24.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.129+0000 7ff4fb400640 1 -- 172.21.15.123:0/804733946 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4031b70 con 0x7ff4fc0cfc60 2024-04-24T05:47:24.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.130+0000 7ff503000640 1 -- 172.21.15.123:0/804733946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4fc0cfc60 msgr2=0x7ff4fc0ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:24.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.130+0000 7ff503000640 1 --2- 172.21.15.123:0/804733946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4fc0cfc60 0x7ff4fc0ccbc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009d90 tx=0x7ff4f402a5d0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.130+0000 7ff503000640 1 -- 172.21.15.123:0/804733946 shutdown_connections 2024-04-24T05:47:24.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.130+0000 7ff503000640 1 --2- 172.21.15.123:0/804733946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4fc0cfc60 0x7ff4fc0ccbc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.130+0000 7ff503000640 1 --2- 172.21.15.123:0/804733946 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4fc0cc770 0x7ff4fc0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.130+0000 7ff503000640 1 -- 172.21.15.123:0/804733946 >> 172.21.15.123:0/804733946 conn(0x7ff4fc0b51f0 msgr2=0x7ff4fc0b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:24.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.130+0000 7ff503000640 1 -- 172.21.15.123:0/804733946 shutdown_connections 2024-04-24T05:47:24.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.130+0000 7ff503000640 1 -- 172.21.15.123:0/804733946 wait complete. 2024-04-24T05:47:24.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.132+0000 7ff503000640 1 Processor -- start 2024-04-24T05:47:24.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.132+0000 7ff503000640 1 -- start start 2024-04-24T05:47:24.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.134+0000 7ff503000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4fc0cc770 0x7ff4fc0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:24.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.134+0000 7ff503000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4fc0cfc60 0x7ff4fc09db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:24.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.134+0000 7ff503000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4fc15aff0 0x7ff4fc15bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:24.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.134+0000 7ff503000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc0cd280 con 0x7ff4fc15aff0 2024-04-24T05:47:24.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.134+0000 7ff503000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc0cd100 con 0x7ff4fc0cc770 2024-04-24T05:47:24.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.134+0000 7ff503000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4fc0cd400 con 0x7ff4fc0cfc60 2024-04-24T05:47:24.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.134+0000 7ff501000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4fc15aff0 0x7ff4fc15bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:24.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.134+0000 7ff501000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4fc15aff0 0x7ff4fc15bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33792/0 (socket says 172.21.15.123:33792) 2024-04-24T05:47:24.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.134+0000 7ff501000640 1 -- 172.21.15.123:0/3706001186 learned_addr learned my addr 172.21.15.123:0/3706001186 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:24.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.134+0000 7ff4f3400640 1 --2- 172.21.15.123:0/3706001186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4fc0cfc60 0x7ff4fc09db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:24.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.134+0000 7ff4fbe00640 1 --2- 172.21.15.123:0/3706001186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4fc0cc770 0x7ff4fc0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:24.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.134+0000 7ff501000640 1 -- 172.21.15.123:0/3706001186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4fc0cfc60 msgr2=0x7ff4fc09db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:24.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.135+0000 7ff501000640 1 --2- 172.21.15.123:0/3706001186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4fc0cfc60 0x7ff4fc09db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.135+0000 7ff501000640 1 -- 172.21.15.123:0/3706001186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4fc0cc770 msgr2=0x7ff4fc0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:24.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.135+0000 7ff501000640 1 --2- 172.21.15.123:0/3706001186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4fc0cc770 0x7ff4fc0c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.135+0000 7ff501000640 1 -- 172.21.15.123:0/3706001186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4fc15c680 con 0x7ff4fc15aff0 2024-04-24T05:47:24.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.135+0000 7ff501000640 1 --2- 172.21.15.123:0/3706001186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4fc15aff0 0x7ff4fc15bf00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff4f402abc0 tx=0x7ff4f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:24.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.135+0000 7ff4f8c00640 1 -- 172.21.15.123:0/3706001186 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4040070 con 0x7ff4fc15aff0 2024-04-24T05:47:24.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.135+0000 7ff4f8c00640 1 -- 172.21.15.123:0/3706001186 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f4030b60 con 0x7ff4fc15aff0 2024-04-24T05:47:24.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.135+0000 7ff4f8c00640 1 -- 172.21.15.123:0/3706001186 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4009450 con 0x7ff4fc15aff0 2024-04-24T05:47:24.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.135+0000 7ff503000640 1 -- 172.21.15.123:0/3706001186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4fc15c960 con 0x7ff4fc15aff0 2024-04-24T05:47:24.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.135+0000 7ff503000640 1 -- 172.21.15.123:0/3706001186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4fc15cf20 con 0x7ff4fc15aff0 2024-04-24T05:47:24.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.136+0000 7ff503000640 1 -- 172.21.15.123:0/3706001186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4fc068a20 con 0x7ff4fc15aff0 2024-04-24T05:47:24.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.137+0000 7ff4f8c00640 1 -- 172.21.15.123:0/3706001186 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4f4039040 con 0x7ff4fc15aff0 2024-04-24T05:47:24.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.138+0000 7ff4f8c00640 1 --2- 172.21.15.123:0/3706001186 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff4d806c8f0 0x7ff4d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:24.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.138+0000 7ff4f8c00640 1 -- 172.21.15.123:0/3706001186 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7ff4f40ad6a0 con 0x7ff4fc15aff0 2024-04-24T05:47:24.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.139+0000 7ff4fbe00640 1 --2- 172.21.15.123:0/3706001186 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff4d806c8f0 0x7ff4d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:24.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.139+0000 7ff4fbe00640 1 --2- 172.21.15.123:0/3706001186 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff4d806c8f0 0x7ff4d806edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec005880 tx=0x7ff4ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:24.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.143+0000 7ff4f8c00640 1 -- 172.21.15.123:0/3706001186 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4f407b6b0 con 0x7ff4fc15aff0 2024-04-24T05:47:24.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.350+0000 7ff503000640 1 -- 172.21.15.123:0/3706001186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff4fc0d10e0 con 0x7ff4fc15aff0 2024-04-24T05:47:24.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.351+0000 7ff4f8c00640 1 -- 172.21.15.123:0/3706001186 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7ff4f407afb0 con 0x7ff4fc15aff0 2024-04-24T05:47:24.353 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:24.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.354+0000 7ff503000640 1 -- 172.21.15.123:0/3706001186 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff4d806c8f0 msgr2=0x7ff4d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:24.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.354+0000 7ff503000640 1 --2- 172.21.15.123:0/3706001186 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff4d806c8f0 0x7ff4d806edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec005880 tx=0x7ff4ec009040 comp rx=0 tx=0).stop 2024-04-24T05:47:24.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.354+0000 7ff503000640 1 -- 172.21.15.123:0/3706001186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4fc15aff0 msgr2=0x7ff4fc15bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:24.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.354+0000 7ff503000640 1 --2- 172.21.15.123:0/3706001186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4fc15aff0 0x7ff4fc15bf00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff4f402abc0 tx=0x7ff4f4004a40 comp rx=0 tx=0).stop 2024-04-24T05:47:24.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.354+0000 7ff503000640 1 -- 172.21.15.123:0/3706001186 shutdown_connections 2024-04-24T05:47:24.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.354+0000 7ff503000640 1 --2- 172.21.15.123:0/3706001186 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff4d806c8f0 0x7ff4d806edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.354+0000 7ff503000640 1 --2- 172.21.15.123:0/3706001186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4fc15aff0 0x7ff4fc15bf00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.354+0000 7ff503000640 1 --2- 172.21.15.123:0/3706001186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4fc0cfc60 0x7ff4fc09db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.354+0000 7ff503000640 1 --2- 172.21.15.123:0/3706001186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4fc0cc770 0x7ff4fc0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.354+0000 7ff503000640 1 -- 172.21.15.123:0/3706001186 >> 172.21.15.123:0/3706001186 conn(0x7ff4fc0b51f0 msgr2=0x7ff4fc06eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:24.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.354+0000 7ff503000640 1 -- 172.21.15.123:0/3706001186 shutdown_connections 2024-04-24T05:47:24.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.354+0000 7ff503000640 1 -- 172.21.15.123:0/3706001186 wait complete. 2024-04-24T05:47:24.373 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T05:47:24.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.570+0000 7fd6dbe00640 1 Processor -- start 2024-04-24T05:47:24.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.570+0000 7fd6dbe00640 1 -- start start 2024-04-24T05:47:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.571+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6dc06ca10 0x7fd6dc0cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.571+0000 7fd6dbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd6dc06cdf0 0x7fd6dc15eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.572+0000 7fd6dbe00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6dc15f900 con 0x7fd6dc15f0b0 2024-04-24T05:47:24.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.572+0000 7fd6dbe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd6dc0cba10 con 0x7fd6dc06cdf0 2024-04-24T05:47:24.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.572+0000 7fd6dbe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd6dc0cbcf0 con 0x7fd6dc06ca10 2024-04-24T05:47:24.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.572+0000 7fd6da000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd6dc06cdf0 0x7fd6dc15eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:24.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.572+0000 7fd6da000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd6dc06cdf0 0x7fd6dc15eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33806/0 (socket says 172.21.15.123:33806) 2024-04-24T05:47:24.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.572+0000 7fd6da000640 1 -- 172.21.15.123:0/1212681081 learned_addr learned my addr 172.21.15.123:0/1212681081 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:24.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.572+0000 7fd6daa00640 1 --2- 172.21.15.123:0/1212681081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6dc06ca10 0x7fd6dc0cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:24.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.573+0000 7fd6da000640 1 -- 172.21.15.123:0/1212681081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6dc06ca10 msgr2=0x7fd6dc0cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:24.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.573+0000 7fd6da000640 1 --2- 172.21.15.123:0/1212681081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6dc06ca10 0x7fd6dc0cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.573+0000 7fd6da000640 1 -- 172.21.15.123:0/1212681081 >> v1:172.21.15.155:6789/0 conn(0x7fd6dc15f0b0 legacy=0x7fd6dc0ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:24.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.573+0000 7fd6da000640 1 -- 172.21.15.123:0/1212681081 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc0cbfd0 con 0x7fd6dc06cdf0 2024-04-24T05:47:24.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.574+0000 7fd6da000640 1 --2- 172.21.15.123:0/1212681081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc06cdf0 0x7fd6dc15eb70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009870 tx=0x7fd6c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb1836d95946aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:24.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.574+0000 7fd6d9600640 1 -- 172.21.15.123:0/1212681081 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8037070 con 0x7fd6dc06cdf0 2024-04-24T05:47:24.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.574+0000 7fd6d9600640 1 -- 172.21.15.123:0/1212681081 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c8032440 con 0x7fd6dc06cdf0 2024-04-24T05:47:24.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.574+0000 7fd6d9600640 1 -- 172.21.15.123:0/1212681081 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8032810 con 0x7fd6dc06cdf0 2024-04-24T05:47:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.575+0000 7fd6dbe00640 1 -- 172.21.15.123:0/1212681081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc06cdf0 msgr2=0x7fd6dc15eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.575+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/1212681081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc06cdf0 0x7fd6dc15eb70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009870 tx=0x7fd6c802a270 comp rx=0 tx=0).stop 2024-04-24T05:47:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.575+0000 7fd6dbe00640 1 -- 172.21.15.123:0/1212681081 shutdown_connections 2024-04-24T05:47:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.575+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/1212681081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc06cdf0 0x7fd6dc15eb70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.575+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/1212681081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6dc06ca10 0x7fd6dc0cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.575+0000 7fd6dbe00640 1 -- 172.21.15.123:0/1212681081 >> 172.21.15.123:0/1212681081 conn(0x7fd6dc0b1b10 msgr2=0x7fd6dc0b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.575+0000 7fd6dbe00640 1 -- 172.21.15.123:0/1212681081 shutdown_connections 2024-04-24T05:47:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.575+0000 7fd6dbe00640 1 -- 172.21.15.123:0/1212681081 wait complete. 2024-04-24T05:47:24.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.577+0000 7fd6dbe00640 1 Processor -- start 2024-04-24T05:47:24.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.577+0000 7fd6dbe00640 1 -- start start 2024-04-24T05:47:24.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc06ca10 0x7fd6dc09a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:24.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6dc06cdf0 0x7fd6dc0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:24.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6dbe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6dc15f0b0 0x7fd6dc0c2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:24.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6dbe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc06d250 con 0x7fd6dc06ca10 2024-04-24T05:47:24.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6dbe00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc15fbe0 con 0x7fd6dc06cdf0 2024-04-24T05:47:24.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6dbe00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd6dc15f540 con 0x7fd6dc15f0b0 2024-04-24T05:47:24.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6daa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc06ca10 0x7fd6dc09a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:24.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6daa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc06ca10 0x7fd6dc09a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33812/0 (socket says 172.21.15.123:33812) 2024-04-24T05:47:24.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6daa00640 1 -- 172.21.15.123:0/227540935 learned_addr learned my addr 172.21.15.123:0/227540935 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:24.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6da000640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6dc06cdf0 0x7fd6dc0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:24.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6db400640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6dc15f0b0 0x7fd6dc0c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:24.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6daa00640 1 -- 172.21.15.123:0/227540935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6dc15f0b0 msgr2=0x7fd6dc0c2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:24.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6daa00640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6dc15f0b0 0x7fd6dc0c2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6daa00640 1 -- 172.21.15.123:0/227540935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6dc06cdf0 msgr2=0x7fd6dc0cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:24.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.578+0000 7fd6daa00640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6dc06cdf0 0x7fd6dc0cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.579+0000 7fd6daa00640 1 -- 172.21.15.123:0/227540935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc09b230 con 0x7fd6dc06ca10 2024-04-24T05:47:24.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.579+0000 7fd6da000640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6dc06cdf0 0x7fd6dc0cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:24.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.579+0000 7fd6daa00640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc06ca10 0x7fd6dc09a9e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00d5d0 tx=0x7fd6cc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:24.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.579+0000 7fd6d2a00640 1 -- 172.21.15.123:0/227540935 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc0125f0 con 0x7fd6dc06ca10 2024-04-24T05:47:24.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.579+0000 7fd6d2a00640 1 -- 172.21.15.123:0/227540935 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6cc019070 con 0x7fd6dc06ca10 2024-04-24T05:47:24.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.579+0000 7fd6dbe00640 1 -- 172.21.15.123:0/227540935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6dc09b510 con 0x7fd6dc06ca10 2024-04-24T05:47:24.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.579+0000 7fd6d2a00640 1 -- 172.21.15.123:0/227540935 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc0129f0 con 0x7fd6dc06ca10 2024-04-24T05:47:24.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.579+0000 7fd6dbe00640 1 -- 172.21.15.123:0/227540935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6dc09bad0 con 0x7fd6dc06ca10 2024-04-24T05:47:24.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.580+0000 7fd6dbe00640 1 -- 172.21.15.123:0/227540935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6dc075b70 con 0x7fd6dc06ca10 2024-04-24T05:47:24.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.581+0000 7fd6d2a00640 1 -- 172.21.15.123:0/227540935 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6cc012b90 con 0x7fd6dc06ca10 2024-04-24T05:47:24.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.582+0000 7fd6d2a00640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd6bc06c8f0 0x7fd6bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:24.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.582+0000 7fd6d2a00640 1 -- 172.21.15.123:0/227540935 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fd6cc08c460 con 0x7fd6dc06ca10 2024-04-24T05:47:24.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.583+0000 7fd6da000640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd6bc06c8f0 0x7fd6bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:24.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.583+0000 7fd6da000640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd6bc06c8f0 0x7fd6bc06edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8034000 tx=0x7fd6c802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:24.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.587+0000 7fd6d2a00640 1 -- 172.21.15.123:0/227540935 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6cc05a3e0 con 0x7fd6dc06ca10 2024-04-24T05:47:24.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.799+0000 7fd6dbe00640 1 -- 172.21.15.123:0/227540935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd6dc0b5aa0 con 0x7fd6dc06ca10 2024-04-24T05:47:24.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.799+0000 7fd6d2a00640 1 -- 172.21.15.123:0/227540935 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fd6cc059ce0 con 0x7fd6dc06ca10 2024-04-24T05:47:24.801 INFO:teuthology.orchestra.run.smithi123.stderr:listed 0 entries 2024-04-24T05:47:24.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.802+0000 7fd6dbe00640 1 -- 172.21.15.123:0/227540935 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd6bc06c8f0 msgr2=0x7fd6bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:24.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.802+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd6bc06c8f0 0x7fd6bc06edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8034000 tx=0x7fd6c802e000 comp rx=0 tx=0).stop 2024-04-24T05:47:24.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.802+0000 7fd6dbe00640 1 -- 172.21.15.123:0/227540935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc06ca10 msgr2=0x7fd6dc09a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:24.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.802+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc06ca10 0x7fd6dc09a9e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00d5d0 tx=0x7fd6cc00da90 comp rx=0 tx=0).stop 2024-04-24T05:47:24.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.802+0000 7fd6dbe00640 1 -- 172.21.15.123:0/227540935 shutdown_connections 2024-04-24T05:47:24.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.802+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd6bc06c8f0 0x7fd6bc06edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.802+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6dc15f0b0 0x7fd6dc0c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.802+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6dc06cdf0 0x7fd6dc0cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.802+0000 7fd6dbe00640 1 --2- 172.21.15.123:0/227540935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6dc06ca10 0x7fd6dc09a9e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.802+0000 7fd6dbe00640 1 -- 172.21.15.123:0/227540935 >> 172.21.15.123:0/227540935 conn(0x7fd6dc0b1b10 msgr2=0x7fd6dc06e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.803+0000 7fd6dbe00640 1 -- 172.21.15.123:0/227540935 shutdown_connections 2024-04-24T05:47:24.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:24.803+0000 7fd6dbe00640 1 -- 172.21.15.123:0/227540935 wait complete. 2024-04-24T05:47:24.824 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2024-04-24T05:47:24.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:24.824 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-24T05:47:24.906 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T05:47:24.907 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-04-24T05:47:24.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:24.907 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-24T05:47:24.948 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-24T05:47:24.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T05:47:25.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.154+0000 7f51d7a00640 1 Processor -- start 2024-04-24T05:47:25.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.155+0000 7f51d7a00640 1 -- start start 2024-04-24T05:47:25.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.156+0000 7f51d7a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f51d00ce5d0 0x7f51d00cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:25.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.156+0000 7f51d7a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d015cc90 0x7f51d015d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:25.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.156+0000 7f51d7a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51d00cf300 con 0x7f51d00b6880 2024-04-24T05:47:25.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.156+0000 7f51d7a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f51d00cf5e0 con 0x7f51d00ce5d0 2024-04-24T05:47:25.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.156+0000 7f51d7a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f51d00cf8c0 con 0x7f51d015cc90 2024-04-24T05:47:25.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.157+0000 7f51cfe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f51d00ce5d0 0x7f51d00cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:25.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.157+0000 7f51cfe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f51d00ce5d0 0x7f51d00cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33818/0 (socket says 172.21.15.123:33818) 2024-04-24T05:47:25.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.157+0000 7f51cfe00640 1 -- 172.21.15.123:0/4053604296 learned_addr learned my addr 172.21.15.123:0/4053604296 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:25.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.157+0000 7f51d5a00640 1 --2- 172.21.15.123:0/4053604296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d015cc90 0x7f51d015d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:25.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.157+0000 7f51cfe00640 1 -- 172.21.15.123:0/4053604296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d015cc90 msgr2=0x7f51d015d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:25.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.157+0000 7f51cfe00640 1 --2- 172.21.15.123:0/4053604296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d015cc90 0x7f51d015d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.157+0000 7f51cfe00640 1 -- 172.21.15.123:0/4053604296 >> v1:172.21.15.155:6789/0 conn(0x7f51d00b6880 legacy=0x7f51d00b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:25.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.157+0000 7f51cfe00640 1 -- 172.21.15.123:0/4053604296 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d00cfba0 con 0x7f51d00ce5d0 2024-04-24T05:47:25.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.157+0000 7f51d5a00640 1 --2- 172.21.15.123:0/4053604296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d015cc90 0x7f51d015d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:25.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.158+0000 7f51cfe00640 1 --2- 172.21.15.123:0/4053604296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51d00ce5d0 0x7f51d00cea80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009870 tx=0x7f51bc02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e3e6dba9057237 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:25.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.158+0000 7f51cf400640 1 -- 172.21.15.123:0/4053604296 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc037070 con 0x7f51d00ce5d0 2024-04-24T05:47:25.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.159+0000 7f51cf400640 1 -- 172.21.15.123:0/4053604296 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51bc032440 con 0x7f51d00ce5d0 2024-04-24T05:47:25.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.159+0000 7f51cf400640 1 -- 172.21.15.123:0/4053604296 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc032810 con 0x7f51d00ce5d0 2024-04-24T05:47:25.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.159+0000 7f51d7a00640 1 -- 172.21.15.123:0/4053604296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51d00ce5d0 msgr2=0x7f51d00cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:25.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.159+0000 7f51d7a00640 1 --2- 172.21.15.123:0/4053604296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51d00ce5d0 0x7f51d00cea80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009870 tx=0x7f51bc02a200 comp rx=0 tx=0).stop 2024-04-24T05:47:25.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.159+0000 7f51d7a00640 1 -- 172.21.15.123:0/4053604296 shutdown_connections 2024-04-24T05:47:25.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.159+0000 7f51d7a00640 1 --2- 172.21.15.123:0/4053604296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d015cc90 0x7f51d015d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.159+0000 7f51d7a00640 1 --2- 172.21.15.123:0/4053604296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51d00ce5d0 0x7f51d00cea80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.159+0000 7f51d7a00640 1 -- 172.21.15.123:0/4053604296 >> 172.21.15.123:0/4053604296 conn(0x7f51d00b61c0 msgr2=0x7f51d009a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:25.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.159+0000 7f51d7a00640 1 -- 172.21.15.123:0/4053604296 shutdown_connections 2024-04-24T05:47:25.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.160+0000 7f51d7a00640 1 -- 172.21.15.123:0/4053604296 wait complete. 2024-04-24T05:47:25.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.161+0000 7f51d7a00640 1 Processor -- start 2024-04-24T05:47:25.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.161+0000 7f51d7a00640 1 -- start start 2024-04-24T05:47:25.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.162+0000 7f51d7a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51d00b6880 0x7f51d009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:25.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.162+0000 7f51d7a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51d00ce5d0 0x7f51d014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:25.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.162+0000 7f51d7a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d015cc90 0x7f51d01f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:25.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.162+0000 7f51d7a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51d015c8c0 con 0x7f51d00b6880 2024-04-24T05:47:25.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.162+0000 7f51d7a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f51d00cefc0 con 0x7f51d00ce5d0 2024-04-24T05:47:25.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.162+0000 7f51d7a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f51d015ca40 con 0x7f51d015cc90 2024-04-24T05:47:25.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.163+0000 7f51d5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51d00b6880 0x7f51d009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:25.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.163+0000 7f51d5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51d00b6880 0x7f51d009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33830/0 (socket says 172.21.15.123:33830) 2024-04-24T05:47:25.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.163+0000 7f51d5000640 1 -- 172.21.15.123:0/1709764219 learned_addr learned my addr 172.21.15.123:0/1709764219 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:25.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.163+0000 7f51cfe00640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51d00ce5d0 0x7f51d014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:25.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.163+0000 7f51d5a00640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d015cc90 0x7f51d01f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:25.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.163+0000 7f51d5000640 1 -- 172.21.15.123:0/1709764219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d015cc90 msgr2=0x7f51d01f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:25.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.163+0000 7f51d5000640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d015cc90 0x7f51d01f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.163+0000 7f51d5000640 1 -- 172.21.15.123:0/1709764219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51d00ce5d0 msgr2=0x7f51d014eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:25.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.163+0000 7f51d5000640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51d00ce5d0 0x7f51d014eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.163+0000 7f51d5000640 1 -- 172.21.15.123:0/1709764219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d009b8e0 con 0x7f51d00b6880 2024-04-24T05:47:25.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.163+0000 7f51d5a00640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d015cc90 0x7f51d01f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:25.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.163+0000 7f51d5000640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51d00b6880 0x7f51d009dac0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f51c400c430 tx=0x7f51c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:25.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.164+0000 7f51ccc00640 1 -- 172.21.15.123:0/1709764219 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4011790 con 0x7f51d00b6880 2024-04-24T05:47:25.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.164+0000 7f51ccc00640 1 -- 172.21.15.123:0/1709764219 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c4017070 con 0x7f51d00b6880 2024-04-24T05:47:25.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.164+0000 7f51ccc00640 1 -- 172.21.15.123:0/1709764219 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4011bb0 con 0x7f51d00b6880 2024-04-24T05:47:25.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.164+0000 7f51d7a00640 1 -- 172.21.15.123:0/1709764219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d01f4a10 con 0x7f51d00b6880 2024-04-24T05:47:25.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.164+0000 7f51d7a00640 1 -- 172.21.15.123:0/1709764219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d01f4fd0 con 0x7f51d00b6880 2024-04-24T05:47:25.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.165+0000 7f51d7a00640 1 -- 172.21.15.123:0/1709764219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d009bab0 con 0x7f51d00b6880 2024-04-24T05:47:25.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.166+0000 7f51ccc00640 1 -- 172.21.15.123:0/1709764219 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51c401c460 con 0x7f51d00b6880 2024-04-24T05:47:25.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.167+0000 7f51ccc00640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f51b006c8f0 0x7f51b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:25.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.167+0000 7f51ccc00640 1 -- 172.21.15.123:0/1709764219 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f51c408be80 con 0x7f51d00b6880 2024-04-24T05:47:25.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.167+0000 7f51cfe00640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f51b006c8f0 0x7f51b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:25.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.168+0000 7f51cfe00640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f51b006c8f0 0x7f51b006edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f51bc034000 tx=0x7f51bc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:25.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.172+0000 7f51ccc00640 1 -- 172.21.15.123:0/1709764219 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f51c4059ea0 con 0x7f51d00b6880 2024-04-24T05:47:25.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.407+0000 7f51d7a00640 1 -- 172.21.15.123:0/1709764219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51d0069130 con 0x7f51d00b6880 2024-04-24T05:47:25.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.408+0000 7f51ccc00640 1 -- 172.21.15.123:0/1709764219 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f51c40597a0 con 0x7f51d00b6880 2024-04-24T05:47:25.410 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:25.410 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2743935658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2743935658},{"type":"v1","addr":"172.21.15.155:6834","nonce":2743935658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1148561953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1148561953},{"type":"v1","addr":"172.21.15.123:6836","nonce":1148561953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2992052647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2992052647},{"type":"v1","addr":"172.21.15.155:6835","nonce":2992052647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T05:45:59.639548+0000","modified":"2024-04-24T05:46:19.813408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/3370584410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3370584410},{"type":"v1","addr":"172.21.15.123:6837","nonce":3370584410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T05:47:25.411 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-24T05:47:25.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.411+0000 7f51d7a00640 1 -- 172.21.15.123:0/1709764219 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f51b006c8f0 msgr2=0x7f51b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:25.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.411+0000 7f51d7a00640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f51b006c8f0 0x7f51b006edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f51bc034000 tx=0x7f51bc02e000 comp rx=0 tx=0).stop 2024-04-24T05:47:25.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.411+0000 7f51d7a00640 1 -- 172.21.15.123:0/1709764219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51d00b6880 msgr2=0x7f51d009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:25.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.411+0000 7f51d7a00640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51d00b6880 0x7f51d009dac0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f51c400c430 tx=0x7f51c400c8f0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.411+0000 7f51d7a00640 1 -- 172.21.15.123:0/1709764219 shutdown_connections 2024-04-24T05:47:25.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.411+0000 7f51d7a00640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f51b006c8f0 0x7f51b006edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.411+0000 7f51d7a00640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d015cc90 0x7f51d01f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.411+0000 7f51d7a00640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51d00ce5d0 0x7f51d014eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.411+0000 7f51d7a00640 1 --2- 172.21.15.123:0/1709764219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51d00b6880 0x7f51d009dac0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.411+0000 7f51d7a00640 1 -- 172.21.15.123:0/1709764219 >> 172.21.15.123:0/1709764219 conn(0x7f51d00b61c0 msgr2=0x7f51d00ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:25.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.411+0000 7f51d7a00640 1 -- 172.21.15.123:0/1709764219 shutdown_connections 2024-04-24T05:47:25.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.411+0000 7f51d7a00640 1 -- 172.21.15.123:0/1709764219 wait complete. 2024-04-24T05:47:25.431 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:25.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.623+0000 7f1b7b200640 1 Processor -- start 2024-04-24T05:47:25.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.623+0000 7f1b7b200640 1 -- start start 2024-04-24T05:47:25.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.625+0000 7f1b7b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b7c15c5d0 0x7f1b7c160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:25.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.625+0000 7f1b7b200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1b7c0adee0 0x7f1b7c0acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:25.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.625+0000 7f1b7b200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1b7c0ae2e0 0x7f1b7c0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.625+0000 7f1b7b200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1b7c160470 con 0x7f1b7c0adee0 2024-04-24T05:47:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.625+0000 7f1b7b200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1b7c160120 con 0x7f1b7c0ae2e0 2024-04-24T05:47:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.625+0000 7f1b7b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1b7c06dd80 con 0x7f1b7c15c5d0 2024-04-24T05:47:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.625+0000 7f1b73e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1b7c0adee0 0x7f1b7c0acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.625+0000 7f1b73e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1b7c0adee0 0x7f1b7c0acf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33836/0 (socket says 172.21.15.123:33836) 2024-04-24T05:47:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.625+0000 7f1b73e00640 1 -- 172.21.15.123:0/3841024683 learned_addr learned my addr 172.21.15.123:0/3841024683 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.625+0000 7f1b79e00640 1 --2- 172.21.15.123:0/3841024683 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b7c15c5d0 0x7f1b7c160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.626+0000 7f1b73e00640 1 -- 172.21.15.123:0/3841024683 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b7c15c5d0 msgr2=0x7f1b7c160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.626+0000 7f1b73e00640 1 --2- 172.21.15.123:0/3841024683 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b7c15c5d0 0x7f1b7c160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.626+0000 7f1b73e00640 1 -- 172.21.15.123:0/3841024683 >> v2:172.21.15.155:3300/0 conn(0x7f1b7c0ae2e0 msgr2=0x7f1b7c0ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.626+0000 7f1b7a800640 1 --2- 172.21.15.123:0/3841024683 >> v2:172.21.15.155:3300/0 conn(0x7f1b7c0ae2e0 0x7f1b7c0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.626+0000 7f1b73e00640 1 --2- 172.21.15.123:0/3841024683 >> v2:172.21.15.155:3300/0 conn(0x7f1b7c0ae2e0 0x7f1b7c0ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.626+0000 7f1b73e00640 1 -- 172.21.15.123:0/3841024683 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c06e060 con 0x7f1b7c0adee0 2024-04-24T05:47:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.627+0000 7f1b73e00640 1 --2- 172.21.15.123:0/3841024683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0adee0 0x7f1b7c0acf40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c071850 tx=0x7f1b6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3854530b6edfa0e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.627+0000 7f1b79400640 1 -- 172.21.15.123:0/3841024683 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64037070 con 0x7f1b7c0adee0 2024-04-24T05:47:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.627+0000 7f1b79400640 1 -- 172.21.15.123:0/3841024683 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b64032440 con 0x7f1b7c0adee0 2024-04-24T05:47:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.627+0000 7f1b79400640 1 -- 172.21.15.123:0/3841024683 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64032810 con 0x7f1b7c0adee0 2024-04-24T05:47:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.628+0000 7f1b7b200640 1 -- 172.21.15.123:0/3841024683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0adee0 msgr2=0x7f1b7c0acf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.628+0000 7f1b7b200640 1 --2- 172.21.15.123:0/3841024683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0adee0 0x7f1b7c0acf40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c071850 tx=0x7f1b6402a1e0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.628+0000 7f1b7b200640 1 -- 172.21.15.123:0/3841024683 shutdown_connections 2024-04-24T05:47:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.628+0000 7f1b7b200640 1 --2- 172.21.15.123:0/3841024683 >> v2:172.21.15.155:3300/0 conn(0x7f1b7c0ae2e0 0x7f1b7c0ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.628+0000 7f1b7b200640 1 --2- 172.21.15.123:0/3841024683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0adee0 0x7f1b7c0acf40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.628+0000 7f1b7b200640 1 --2- 172.21.15.123:0/3841024683 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b7c15c5d0 0x7f1b7c160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.628+0000 7f1b7b200640 1 -- 172.21.15.123:0/3841024683 >> 172.21.15.123:0/3841024683 conn(0x7f1b7c0d0070 msgr2=0x7f1b7c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:25.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.628+0000 7f1b7b200640 1 -- 172.21.15.123:0/3841024683 shutdown_connections 2024-04-24T05:47:25.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.628+0000 7f1b7b200640 1 -- 172.21.15.123:0/3841024683 wait complete. 2024-04-24T05:47:25.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.630+0000 7f1b7b200640 1 Processor -- start 2024-04-24T05:47:25.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.630+0000 7f1b7b200640 1 -- start start 2024-04-24T05:47:25.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.631+0000 7f1b7b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c0adee0 0x7f1b7c1f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:25.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.631+0000 7f1b7b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0ae2e0 0x7f1b7c1f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:25.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.631+0000 7f1b7b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b7c15c5d0 0x7f1b7c1f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:25.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.631+0000 7f1b7b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c160fc0 con 0x7f1b7c0ae2e0 2024-04-24T05:47:25.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.631+0000 7f1b7b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c160e40 con 0x7f1b7c0adee0 2024-04-24T05:47:25.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.631+0000 7f1b7b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1b7c0d0b60 con 0x7f1b7c15c5d0 2024-04-24T05:47:25.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.631+0000 7f1b73e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0ae2e0 0x7f1b7c1f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:25.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.631+0000 7f1b73e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0ae2e0 0x7f1b7c1f2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33846/0 (socket says 172.21.15.123:33846) 2024-04-24T05:47:25.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.631+0000 7f1b73e00640 1 -- 172.21.15.123:0/145181097 learned_addr learned my addr 172.21.15.123:0/145181097 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:25.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.631+0000 7f1b79e00640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c0adee0 0x7f1b7c1f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:25.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.632+0000 7f1b7a800640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b7c15c5d0 0x7f1b7c1f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:25.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.632+0000 7f1b73e00640 1 -- 172.21.15.123:0/145181097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b7c15c5d0 msgr2=0x7f1b7c1f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:25.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.632+0000 7f1b73e00640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b7c15c5d0 0x7f1b7c1f5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.632+0000 7f1b73e00640 1 -- 172.21.15.123:0/145181097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c0adee0 msgr2=0x7f1b7c1f2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:25.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.632+0000 7f1b73e00640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c0adee0 0x7f1b7c1f2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.632+0000 7f1b73e00640 1 -- 172.21.15.123:0/145181097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c159ac0 con 0x7f1b7c0ae2e0 2024-04-24T05:47:25.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.632+0000 7f1b7a800640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b7c15c5d0 0x7f1b7c1f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:25.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.632+0000 7f1b73e00640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0ae2e0 0x7f1b7c1f2a90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1b64000c00 tx=0x7f1b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:25.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.632+0000 7f1b72000640 1 -- 172.21.15.123:0/145181097 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64041020 con 0x7f1b7c0ae2e0 2024-04-24T05:47:25.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.632+0000 7f1b72000640 1 -- 172.21.15.123:0/145181097 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b640306c0 con 0x7f1b7c0ae2e0 2024-04-24T05:47:25.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.632+0000 7f1b7b200640 1 -- 172.21.15.123:0/145181097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c14c2f0 con 0x7f1b7c0ae2e0 2024-04-24T05:47:25.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.632+0000 7f1b72000640 1 -- 172.21.15.123:0/145181097 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64037040 con 0x7f1b7c0ae2e0 2024-04-24T05:47:25.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.632+0000 7f1b7b200640 1 -- 172.21.15.123:0/145181097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c15e060 con 0x7f1b7c0ae2e0 2024-04-24T05:47:25.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.634+0000 7f1b7b200640 1 -- 172.21.15.123:0/145181097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7c1493a0 con 0x7f1b7c0ae2e0 2024-04-24T05:47:25.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.635+0000 7f1b72000640 1 -- 172.21.15.123:0/145181097 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b64044020 con 0x7f1b7c0ae2e0 2024-04-24T05:47:25.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.635+0000 7f1b72000640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1b5806c8f0 0x7f1b5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:25.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.636+0000 7f1b72000640 1 -- 172.21.15.123:0/145181097 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f1b640ac7c0 con 0x7f1b7c0ae2e0 2024-04-24T05:47:25.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.636+0000 7f1b79e00640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1b5806c8f0 0x7f1b5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:25.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.637+0000 7f1b79e00640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1b5806c8f0 0x7f1b5806edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c006fd0 tx=0x7f1b6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:25.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.640+0000 7f1b72000640 1 -- 172.21.15.123:0/145181097 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b7c1493a0 con 0x7f1b7c0ae2e0 2024-04-24T05:47:25.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.841+0000 7f1b7b200640 1 -- 172.21.15.123:0/145181097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b7c075b70 con 0x7f1b7c0ae2e0 2024-04-24T05:47:25.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.843+0000 7f1b72000640 1 -- 172.21.15.123:0/145181097 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7f1b6404b030 con 0x7f1b7c0ae2e0 2024-04-24T05:47:25.845 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:25.845 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:25.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.845+0000 7f1b7b200640 1 -- 172.21.15.123:0/145181097 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1b5806c8f0 msgr2=0x7f1b5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:25.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.845+0000 7f1b7b200640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1b5806c8f0 0x7f1b5806edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c006fd0 tx=0x7f1b6c008040 comp rx=0 tx=0).stop 2024-04-24T05:47:25.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.845+0000 7f1b7b200640 1 -- 172.21.15.123:0/145181097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0ae2e0 msgr2=0x7f1b7c1f2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:25.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.845+0000 7f1b7b200640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0ae2e0 0x7f1b7c1f2a90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1b64000c00 tx=0x7f1b640046c0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.845+0000 7f1b7b200640 1 -- 172.21.15.123:0/145181097 shutdown_connections 2024-04-24T05:47:25.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.845+0000 7f1b7b200640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1b5806c8f0 0x7f1b5806edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.845+0000 7f1b7b200640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b7c15c5d0 0x7f1b7c1f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.845+0000 7f1b7b200640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b7c0ae2e0 0x7f1b7c1f2a90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.846+0000 7f1b7b200640 1 --2- 172.21.15.123:0/145181097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c0adee0 0x7f1b7c1f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:25.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.846+0000 7f1b7b200640 1 -- 172.21.15.123:0/145181097 >> 172.21.15.123:0/145181097 conn(0x7f1b7c0d0070 msgr2=0x7f1b7c0afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:25.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.846+0000 7f1b7b200640 1 -- 172.21.15.123:0/145181097 shutdown_connections 2024-04-24T05:47:25.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:25.846+0000 7f1b7b200640 1 -- 172.21.15.123:0/145181097 wait complete. 2024-04-24T05:47:25.867 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T05:45:57.615179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T05:45:58.627580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T05:46:02.230550+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T05:47:25.868 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:47:26.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.069+0000 7f147d600640 1 Processor -- start 2024-04-24T05:47:26.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.069+0000 7f147d600640 1 -- start start 2024-04-24T05:47:26.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.071+0000 7f147d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14780b2e30 0x7f147815c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.071+0000 7f147d600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14780d11a0 con 0x7f14780ce7c0 2024-04-24T05:47:26.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.071+0000 7f147d600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f147815d250 con 0x7f14780717e0 2024-04-24T05:47:26.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.071+0000 7f147d600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f147815d530 con 0x7f14780b2e30 2024-04-24T05:47:26.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.072+0000 7f146f400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f14780ce7c0 0x7f14780d1930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43230/0 (socket says 172.21.15.123:43230) 2024-04-24T05:47:26.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.072+0000 7f146f400640 1 -- 172.21.15.123:0/1777837771 learned_addr learned my addr 172.21.15.123:0/1777837771 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:26.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.072+0000 7f1477e00640 1 --2- 172.21.15.123:0/1777837771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14780b2e30 0x7f147815c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:26.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.072+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1635996746 0 0) 0x7f14780d11a0 con 0x7f14780ce7c0 2024-04-24T05:47:26.075 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.072+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f145c003520 con 0x7f14780ce7c0 2024-04-24T05:47:26.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.072+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1689173210 0 0) 0x7f147815d250 con 0x7f14780717e0 2024-04-24T05:47:26.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.072+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14780d11a0 con 0x7f14780717e0 2024-04-24T05:47:26.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.072+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 473140040 0 0) 0x7f145c003520 con 0x7f14780ce7c0 2024-04-24T05:47:26.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.072+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14780b2e30 msgr2=0x7f147815c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:26.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.072+0000 7f1477400640 1 --2- 172.21.15.123:0/1777837771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14780b2e30 0x7f147815c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.072+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 >> v1:172.21.15.155:6789/0 conn(0x7f14780717e0 legacy=0x7f14780b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.072+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14780d1410 con 0x7f14780ce7c0 2024-04-24T05:47:26.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.072+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f1460001670 con 0x7f14780ce7c0 2024-04-24T05:47:26.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.073+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 >> v1:172.21.15.123:6789/0 conn(0x7f14780ce7c0 legacy=0x7f14780d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.073+0000 7f1477400640 1 --2- 172.21.15.123:0/1777837771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f145c003b70 0x7f145c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.073+0000 7f1477400640 1 --2- 172.21.15.123:0/1777837771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14780b2e30 0x7f145c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.073+0000 7f1477400640 1 --2- 172.21.15.123:0/1777837771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f145c004d40 0x7f145c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.073+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f147815d530 con 0x7f145c003b70 2024-04-24T05:47:26.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.073+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f145c00bb50 con 0x7f145c004d40 2024-04-24T05:47:26.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.073+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f145c00bd20 con 0x7f14780b2e30 2024-04-24T05:47:26.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.073+0000 7f1477e00640 1 --2- 172.21.15.123:0/1777837771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f145c003b70 0x7f145c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:26.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.073+0000 7f1477e00640 1 -- 172.21.15.123:0/1777837771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14780b2e30 msgr2=0x7f145c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:26.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.073+0000 7f1477e00640 1 --2- 172.21.15.123:0/1777837771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14780b2e30 0x7f145c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.073+0000 7f1477e00640 1 -- 172.21.15.123:0/1777837771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f145c004d40 msgr2=0x7f145c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:26.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.073+0000 7f1477e00640 1 --2- 172.21.15.123:0/1777837771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f145c004d40 0x7f145c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.073+0000 7f1477e00640 1 -- 172.21.15.123:0/1777837771 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c003520 con 0x7f145c003b70 2024-04-24T05:47:26.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.074+0000 7f1477e00640 1 --2- 172.21.15.123:0/1777837771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f145c003b70 0x7f145c003f70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f146800e170 tx=0x7f145c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78942aa72ca9ced9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:26.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.075+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468034020 con 0x7f145c003b70 2024-04-24T05:47:26.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.075+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1468030430 con 0x7f145c003b70 2024-04-24T05:47:26.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.075+0000 7f1477400640 1 -- 172.21.15.123:0/1777837771 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468009c10 con 0x7f145c003b70 2024-04-24T05:47:26.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.075+0000 7f147d600640 1 -- 172.21.15.123:0/1777837771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f145c003b70 msgr2=0x7f145c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.075+0000 7f147d600640 1 --2- 172.21.15.123:0/1777837771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f145c003b70 0x7f145c003f70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f146800e170 tx=0x7f145c005830 comp rx=0 tx=0).stop 2024-04-24T05:47:26.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.075+0000 7f147cc00640 1 -- 172.21.15.123:0/1777837771 reap_dead start 2024-04-24T05:47:26.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.075+0000 7f147d600640 1 -- 172.21.15.123:0/1777837771 shutdown_connections 2024-04-24T05:47:26.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.075+0000 7f147d600640 1 -- 172.21.15.123:0/1777837771 >> 172.21.15.123:0/1777837771 conn(0x7f14780b4180 msgr2=0x7f14780b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:26.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.075+0000 7f147d600640 1 -- 172.21.15.123:0/1777837771 shutdown_connections 2024-04-24T05:47:26.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.075+0000 7f147d600640 1 -- 172.21.15.123:0/1777837771 wait complete. 2024-04-24T05:47:26.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.077+0000 7f147d600640 1 Processor -- start 2024-04-24T05:47:26.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.077+0000 7f147d600640 1 -- start start 2024-04-24T05:47:26.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.078+0000 7f147d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f145c003b70 0x7f147814d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.078+0000 7f147d600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f147814df10 0x7f14780709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.078+0000 7f147d600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1478070ef0 0x7f147806dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.078+0000 7f147d600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f147815d810 con 0x7f145c003b70 2024-04-24T05:47:26.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.078+0000 7f147d600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14780d2040 con 0x7f147814df10 2024-04-24T05:47:26.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.078+0000 7f147d600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f14780b5ee0 con 0x7f1478070ef0 2024-04-24T05:47:26.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.079+0000 7f1477e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f145c003b70 0x7f147814d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:26.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.079+0000 7f1477e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f145c003b70 0x7f147814d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33866/0 (socket says 172.21.15.123:33866) 2024-04-24T05:47:26.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.079+0000 7f1477e00640 1 -- 172.21.15.123:0/549206182 learned_addr learned my addr 172.21.15.123:0/549206182 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:26.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.079+0000 7f1477e00640 1 -- 172.21.15.123:0/549206182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1478070ef0 msgr2=0x7f147806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.079+0000 7f1477e00640 1 --2- 172.21.15.123:0/549206182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1478070ef0 0x7f147806dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.079+0000 7f1477e00640 1 -- 172.21.15.123:0/549206182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f147814df10 msgr2=0x7f14780709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.079+0000 7f1477e00640 1 --2- 172.21.15.123:0/549206182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f147814df10 0x7f14780709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.079+0000 7f1477e00640 1 -- 172.21.15.123:0/549206182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147806f910 con 0x7f145c003b70 2024-04-24T05:47:26.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.079+0000 7f1477e00640 1 --2- 172.21.15.123:0/549206182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f145c003b70 0x7f147814d9d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1468033040 tx=0x7f1468009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:26.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.080+0000 7f1474c00640 1 -- 172.21.15.123:0/549206182 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468034040 con 0x7f145c003b70 2024-04-24T05:47:26.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.080+0000 7f1474c00640 1 -- 172.21.15.123:0/549206182 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1468003bb0 con 0x7f145c003b70 2024-04-24T05:47:26.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.080+0000 7f1474c00640 1 -- 172.21.15.123:0/549206182 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468004020 con 0x7f145c003b70 2024-04-24T05:47:26.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.080+0000 7f147d600640 1 -- 172.21.15.123:0/549206182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147806fbf0 con 0x7f145c003b70 2024-04-24T05:47:26.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.080+0000 7f147d600640 1 -- 172.21.15.123:0/549206182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14780701b0 con 0x7f145c003b70 2024-04-24T05:47:26.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.081+0000 7f147d600640 1 -- 172.21.15.123:0/549206182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1478069130 con 0x7f145c003b70 2024-04-24T05:47:26.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.082+0000 7f1474c00640 1 -- 172.21.15.123:0/549206182 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1468037020 con 0x7f145c003b70 2024-04-24T05:47:26.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.083+0000 7f1474c00640 1 --2- 172.21.15.123:0/549206182 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f145406c920 0x7f145406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.083+0000 7f1474c00640 1 -- 172.21.15.123:0/549206182 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f14680acdf0 con 0x7f145c003b70 2024-04-24T05:47:26.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.083+0000 7f146f400640 1 --2- 172.21.15.123:0/549206182 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f145406c920 0x7f145406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:26.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.084+0000 7f146f400640 1 --2- 172.21.15.123:0/549206182 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f145406c920 0x7f145406ede0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1460000fd0 tx=0x7f14600014f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:26.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.087+0000 7f1474c00640 1 -- 172.21.15.123:0/549206182 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f146807ad70 con 0x7f145c003b70 2024-04-24T05:47:26.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.288+0000 7f147d600640 1 -- 172.21.15.123:0/549206182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f14780b3470 con 0x7f145c003b70 2024-04-24T05:47:26.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.289+0000 7f1474c00640 1 -- 172.21.15.123:0/549206182 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f146807a670 con 0x7f145c003b70 2024-04-24T05:47:26.290 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:47:26.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.291+0000 7f147d600640 1 -- 172.21.15.123:0/549206182 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f145406c920 msgr2=0x7f145406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.291+0000 7f147d600640 1 --2- 172.21.15.123:0/549206182 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f145406c920 0x7f145406ede0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1460000fd0 tx=0x7f14600014f0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.293 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.291+0000 7f147d600640 1 -- 172.21.15.123:0/549206182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f145c003b70 msgr2=0x7f147814d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.291+0000 7f147d600640 1 --2- 172.21.15.123:0/549206182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f145c003b70 0x7f147814d9d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1468033040 tx=0x7f1468009750 comp rx=0 tx=0).stop 2024-04-24T05:47:26.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.292+0000 7f147d600640 1 -- 172.21.15.123:0/549206182 shutdown_connections 2024-04-24T05:47:26.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.292+0000 7f147d600640 1 --2- 172.21.15.123:0/549206182 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f145406c920 0x7f145406ede0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.292+0000 7f147d600640 1 --2- 172.21.15.123:0/549206182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1478070ef0 0x7f147806dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.292+0000 7f147d600640 1 --2- 172.21.15.123:0/549206182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f147814df10 0x7f14780709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.292+0000 7f147d600640 1 --2- 172.21.15.123:0/549206182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f145c003b70 0x7f147814d9d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.292+0000 7f147d600640 1 -- 172.21.15.123:0/549206182 >> 172.21.15.123:0/549206182 conn(0x7f14780b4180 msgr2=0x7f147815d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:26.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.292+0000 7f147d600640 1 -- 172.21.15.123:0/549206182 shutdown_connections 2024-04-24T05:47:26.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.292+0000 7f147d600640 1 -- 172.21.15.123:0/549206182 wait complete. 2024-04-24T05:47:26.312 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T05:47:26.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.516+0000 7ff772600640 1 Processor -- start 2024-04-24T05:47:26.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.516+0000 7ff772600640 1 -- start start 2024-04-24T05:47:26.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.518+0000 7ff772600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff76c06f070 0x7ff76c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.518+0000 7ff772600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff76c06bb30 con 0x7ff76c06dcb0 2024-04-24T05:47:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.518+0000 7ff772600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff76c06be10 con 0x7ff76c158e00 2024-04-24T05:47:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.518+0000 7ff772600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff76c06f590 con 0x7ff76c06f070 2024-04-24T05:47:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.518+0000 7ff763e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff76c06dcb0 0x7ff76c06e960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43234/0 (socket says 172.21.15.123:43234) 2024-04-24T05:47:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.518+0000 7ff763e00640 1 -- 172.21.15.123:0/438670850 learned_addr learned my addr 172.21.15.123:0/438670850 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:26.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.518+0000 7ff76be00640 1 --2- 172.21.15.123:0/438670850 >> v2:172.21.15.123:3300/0 conn(0x7ff76c06f070 0x7ff76c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:26.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.519+0000 7ff76be00640 1 -- 172.21.15.123:0/438670850 >> v1:172.21.15.155:6789/0 conn(0x7ff76c158e00 legacy=0x7ff76c06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.519+0000 7ff76be00640 1 -- 172.21.15.123:0/438670850 >> v1:172.21.15.123:6789/0 conn(0x7ff76c06dcb0 legacy=0x7ff76c06e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.519+0000 7ff76be00640 1 -- 172.21.15.123:0/438670850 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76c06c0f0 con 0x7ff76c06f070 2024-04-24T05:47:26.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.519+0000 7ff76aa00640 1 -- 172.21.15.123:0/438670850 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3802188569 0 0) 0x7ff76c06bb30 con 0x7ff76c06dcb0 2024-04-24T05:47:26.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.520+0000 7ff76be00640 1 --2- 172.21.15.123:0/438670850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76c06f070 0x7ff76c06e160 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff764009d90 tx=0x7ff76402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7228113c00686caf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:26.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.520+0000 7ff76aa00640 1 -- 172.21.15.123:0/438670850 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764007740 con 0x7ff76c06f070 2024-04-24T05:47:26.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.520+0000 7ff76aa00640 1 -- 172.21.15.123:0/438670850 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7640078e0 con 0x7ff76c06f070 2024-04-24T05:47:26.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.520+0000 7ff76aa00640 1 -- 172.21.15.123:0/438670850 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764007cb0 con 0x7ff76c06f070 2024-04-24T05:47:26.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.520+0000 7ff772600640 1 -- 172.21.15.123:0/438670850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76c06f070 msgr2=0x7ff76c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.520+0000 7ff772600640 1 --2- 172.21.15.123:0/438670850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76c06f070 0x7ff76c06e160 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff764009d90 tx=0x7ff76402a660 comp rx=0 tx=0).stop 2024-04-24T05:47:26.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.521+0000 7ff772600640 1 -- 172.21.15.123:0/438670850 shutdown_connections 2024-04-24T05:47:26.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.521+0000 7ff772600640 1 --2- 172.21.15.123:0/438670850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76c06f070 0x7ff76c06e160 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.521+0000 7ff772600640 1 -- 172.21.15.123:0/438670850 >> 172.21.15.123:0/438670850 conn(0x7ff76c0b51f0 msgr2=0x7ff76c0b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:26.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.521+0000 7ff772600640 1 -- 172.21.15.123:0/438670850 shutdown_connections 2024-04-24T05:47:26.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.521+0000 7ff772600640 1 -- 172.21.15.123:0/438670850 wait complete. 2024-04-24T05:47:26.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.522+0000 7ff772600640 1 Processor -- start 2024-04-24T05:47:26.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.523+0000 7ff772600640 1 -- start start 2024-04-24T05:47:26.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.523+0000 7ff772600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff76c06dcb0 0x7ff76c154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.523+0000 7ff772600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76c06f070 0x7ff76c1553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.524+0000 7ff772600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff76c158e00 0x7ff76c0ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.524+0000 7ff772600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1591e0 con 0x7ff76c06f070 2024-04-24T05:47:26.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.524+0000 7ff772600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c06c460 con 0x7ff76c158e00 2024-04-24T05:47:26.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.524+0000 7ff772600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff76c154780 con 0x7ff76c06dcb0 2024-04-24T05:47:26.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.524+0000 7ff763e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76c06f070 0x7ff76c1553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:26.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.524+0000 7ff763e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76c06f070 0x7ff76c1553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33882/0 (socket says 172.21.15.123:33882) 2024-04-24T05:47:26.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.524+0000 7ff763e00640 1 -- 172.21.15.123:0/3222223443 learned_addr learned my addr 172.21.15.123:0/3222223443 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:26.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.524+0000 7ff76b400640 1 --2- 172.21.15.123:0/3222223443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff76c06dcb0 0x7ff76c154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:26.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.524+0000 7ff76be00640 1 --2- 172.21.15.123:0/3222223443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff76c158e00 0x7ff76c0ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.524+0000 7ff763e00640 1 -- 172.21.15.123:0/3222223443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff76c06dcb0 msgr2=0x7ff76c154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.524+0000 7ff763e00640 1 --2- 172.21.15.123:0/3222223443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff76c06dcb0 0x7ff76c154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.524+0000 7ff763e00640 1 -- 172.21.15.123:0/3222223443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff76c158e00 msgr2=0x7ff76c0ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.524+0000 7ff763e00640 1 --2- 172.21.15.123:0/3222223443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff76c158e00 0x7ff76c0ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.524+0000 7ff763e00640 1 -- 172.21.15.123:0/3222223443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76c09db90 con 0x7ff76c06f070 2024-04-24T05:47:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.525+0000 7ff763e00640 1 --2- 172.21.15.123:0/3222223443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76c06f070 0x7ff76c1553c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff75400c450 tx=0x7ff75400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.525+0000 7ff763400640 1 -- 172.21.15.123:0/3222223443 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754018070 con 0x7ff76c06f070 2024-04-24T05:47:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.525+0000 7ff763400640 1 -- 172.21.15.123:0/3222223443 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7540074e0 con 0x7ff76c06f070 2024-04-24T05:47:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.525+0000 7ff763400640 1 -- 172.21.15.123:0/3222223443 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754004a20 con 0x7ff76c06f070 2024-04-24T05:47:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.525+0000 7ff772600640 1 -- 172.21.15.123:0/3222223443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76c09de70 con 0x7ff76c06f070 2024-04-24T05:47:26.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.525+0000 7ff772600640 1 -- 172.21.15.123:0/3222223443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff76c0c3c50 con 0x7ff76c06f070 2024-04-24T05:47:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.526+0000 7ff772600640 1 -- 172.21.15.123:0/3222223443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff76c068a20 con 0x7ff76c06f070 2024-04-24T05:47:26.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.527+0000 7ff763400640 1 -- 172.21.15.123:0/3222223443 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff754004bc0 con 0x7ff76c06f070 2024-04-24T05:47:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.529+0000 7ff763400640 1 --2- 172.21.15.123:0/3222223443 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff74806c8f0 0x7ff74806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.529+0000 7ff763400640 1 -- 172.21.15.123:0/3222223443 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7ff75408b220 con 0x7ff76c06f070 2024-04-24T05:47:26.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.529+0000 7ff76b400640 1 --2- 172.21.15.123:0/3222223443 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff74806c8f0 0x7ff74806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:26.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.529+0000 7ff76b400640 1 --2- 172.21.15.123:0/3222223443 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff74806c8f0 0x7ff74806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0cd7b0 tx=0x7ff75c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:26.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.533+0000 7ff763400640 1 -- 172.21.15.123:0/3222223443 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7540591a0 con 0x7ff76c06f070 2024-04-24T05:47:26.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.732+0000 7ff772600640 1 -- 172.21.15.123:0/3222223443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff76c163330 con 0x7ff76c06f070 2024-04-24T05:47:26.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.733+0000 7ff763400640 1 -- 172.21.15.123:0/3222223443 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7ff754058aa0 con 0x7ff76c06f070 2024-04-24T05:47:26.734 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.735+0000 7ff772600640 1 -- 172.21.15.123:0/3222223443 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff74806c8f0 msgr2=0x7ff74806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.735+0000 7ff772600640 1 --2- 172.21.15.123:0/3222223443 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff74806c8f0 0x7ff74806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0cd7b0 tx=0x7ff75c009040 comp rx=0 tx=0).stop 2024-04-24T05:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.735+0000 7ff772600640 1 -- 172.21.15.123:0/3222223443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76c06f070 msgr2=0x7ff76c1553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.735+0000 7ff772600640 1 --2- 172.21.15.123:0/3222223443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76c06f070 0x7ff76c1553c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff75400c450 tx=0x7ff75400c910 comp rx=0 tx=0).stop 2024-04-24T05:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.736+0000 7ff772600640 1 -- 172.21.15.123:0/3222223443 shutdown_connections 2024-04-24T05:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.736+0000 7ff772600640 1 --2- 172.21.15.123:0/3222223443 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff74806c8f0 0x7ff74806edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.736+0000 7ff772600640 1 --2- 172.21.15.123:0/3222223443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff76c158e00 0x7ff76c0ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.736+0000 7ff772600640 1 --2- 172.21.15.123:0/3222223443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff76c06f070 0x7ff76c1553c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.736+0000 7ff772600640 1 --2- 172.21.15.123:0/3222223443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff76c06dcb0 0x7ff76c154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.736+0000 7ff772600640 1 -- 172.21.15.123:0/3222223443 >> 172.21.15.123:0/3222223443 conn(0x7ff76c0b51f0 msgr2=0x7ff76c071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.736+0000 7ff772600640 1 -- 172.21.15.123:0/3222223443 shutdown_connections 2024-04-24T05:47:26.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.736+0000 7ff772600640 1 -- 172.21.15.123:0/3222223443 wait complete. 2024-04-24T05:47:26.757 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T05:47:26.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.961+0000 7fbbb6400640 1 Processor -- start 2024-04-24T05:47:26.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.962+0000 7fbbb6400640 1 -- start start 2024-04-24T05:47:26.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.963+0000 7fbbb6400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbbb00cc5d0 0x7fbbb00acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.963+0000 7fbbb6400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbbb00ad510 0x7fbbb006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.963+0000 7fbbb6400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbb006ba00 con 0x7fbbb015c2f0 2024-04-24T05:47:26.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.963+0000 7fbbb6400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbbb006bd30 con 0x7fbbb00ad510 2024-04-24T05:47:26.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.963+0000 7fbbb6400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbbb00ad940 con 0x7fbbb00cc5d0 2024-04-24T05:47:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.963+0000 7fbbb5000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbbb015c2f0 0x7fbbb0141ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43250/0 (socket says 172.21.15.123:43250) 2024-04-24T05:47:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.963+0000 7fbbb5000640 1 -- 172.21.15.123:0/1969550645 learned_addr learned my addr 172.21.15.123:0/1969550645 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.963+0000 7fbbb5a00640 1 --2- 172.21.15.123:0/1969550645 >> v2:172.21.15.123:3300/0 conn(0x7fbbb00ad510 0x7fbbb006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.963+0000 7fbbafe00640 1 --2- 172.21.15.123:0/1969550645 >> v2:172.21.15.155:3300/0 conn(0x7fbbb00cc5d0 0x7fbbb00acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.964+0000 7fbbaf400640 1 -- 172.21.15.123:0/1969550645 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 292548324 0 0) 0x7fbbb006ba00 con 0x7fbbb015c2f0 2024-04-24T05:47:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.964+0000 7fbbaf400640 1 -- 172.21.15.123:0/1969550645 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb94003520 con 0x7fbbb015c2f0 2024-04-24T05:47:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.964+0000 7fbbb5a00640 1 -- 172.21.15.123:0/1969550645 >> v2:172.21.15.155:3300/0 conn(0x7fbbb00cc5d0 msgr2=0x7fbbb00acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.964+0000 7fbbb5a00640 1 --2- 172.21.15.123:0/1969550645 >> v2:172.21.15.155:3300/0 conn(0x7fbbb00cc5d0 0x7fbbb00acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.964+0000 7fbbb5a00640 1 -- 172.21.15.123:0/1969550645 >> v1:172.21.15.123:6789/0 conn(0x7fbbb015c2f0 legacy=0x7fbbb0141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.964+0000 7fbbb5a00640 1 -- 172.21.15.123:0/1969550645 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb0142530 con 0x7fbbb00ad510 2024-04-24T05:47:26.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.965+0000 7fbbb5a00640 1 --2- 172.21.15.123:0/1969550645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb00ad510 0x7fbbb006c1c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0073fd0 tx=0x7fbba802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec4014bd1560b05 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:26.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.965+0000 7fbbaf400640 1 -- 172.21.15.123:0/1969550645 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8007740 con 0x7fbbb00ad510 2024-04-24T05:47:26.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.965+0000 7fbbaf400640 1 -- 172.21.15.123:0/1969550645 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba80078e0 con 0x7fbbb00ad510 2024-04-24T05:47:26.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.965+0000 7fbbaf400640 1 -- 172.21.15.123:0/1969550645 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8007cb0 con 0x7fbbb00ad510 2024-04-24T05:47:26.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.966+0000 7fbbb6400640 1 -- 172.21.15.123:0/1969550645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb00ad510 msgr2=0x7fbbb006c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.966+0000 7fbbb6400640 1 --2- 172.21.15.123:0/1969550645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb00ad510 0x7fbbb006c1c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0073fd0 tx=0x7fbba802a5f0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.966+0000 7fbbb6400640 1 -- 172.21.15.123:0/1969550645 shutdown_connections 2024-04-24T05:47:26.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.966+0000 7fbbb6400640 1 --2- 172.21.15.123:0/1969550645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb00ad510 0x7fbbb006c1c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.966+0000 7fbbb6400640 1 --2- 172.21.15.123:0/1969550645 >> v2:172.21.15.155:3300/0 conn(0x7fbbb00cc5d0 0x7fbbb00acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.966+0000 7fbbb6400640 1 -- 172.21.15.123:0/1969550645 >> 172.21.15.123:0/1969550645 conn(0x7fbbb00b1cf0 msgr2=0x7fbbb00b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:26.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.966+0000 7fbbb6400640 1 -- 172.21.15.123:0/1969550645 shutdown_connections 2024-04-24T05:47:26.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.966+0000 7fbbb6400640 1 -- 172.21.15.123:0/1969550645 wait complete. 2024-04-24T05:47:26.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.967+0000 7fbbb6400640 1 Processor -- start 2024-04-24T05:47:26.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.968+0000 7fbbb6400640 1 -- start start 2024-04-24T05:47:26.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.968+0000 7fbbb6400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbb00ad510 0x7fbbb00cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.969+0000 7fbbb6400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb00cc5d0 0x7fbbb0071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.969+0000 7fbbb6400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbb015c2f0 0x7fbbb00c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.969+0000 7fbbb6400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb015c6d0 con 0x7fbbb00cc5d0 2024-04-24T05:47:26.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.969+0000 7fbbb6400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0142af0 con 0x7fbbb015c2f0 2024-04-24T05:47:26.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.969+0000 7fbbb6400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb015c850 con 0x7fbbb00ad510 2024-04-24T05:47:26.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.969+0000 7fbbafe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb00cc5d0 0x7fbbb0071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:26.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.969+0000 7fbbafe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb00cc5d0 0x7fbbb0071b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33900/0 (socket says 172.21.15.123:33900) 2024-04-24T05:47:26.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.969+0000 7fbbafe00640 1 -- 172.21.15.123:0/2445140538 learned_addr learned my addr 172.21.15.123:0/2445140538 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:26.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.969+0000 7fbbafe00640 1 -- 172.21.15.123:0/2445140538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbb00ad510 msgr2=0x7fbbb00cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.969+0000 7fbbafe00640 1 --2- 172.21.15.123:0/2445140538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbb00ad510 0x7fbbb00cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.969+0000 7fbbafe00640 1 -- 172.21.15.123:0/2445140538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbb015c2f0 msgr2=0x7fbbb00c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:26.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.969+0000 7fbbafe00640 1 --2- 172.21.15.123:0/2445140538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbb015c2f0 0x7fbbb00c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:26.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.969+0000 7fbbafe00640 1 -- 172.21.15.123:0/2445140538 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb00723d0 con 0x7fbbb00cc5d0 2024-04-24T05:47:26.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.969+0000 7fbbafe00640 1 --2- 172.21.15.123:0/2445140538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb00cc5d0 0x7fbbb0071b80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00d5f0 tx=0x7fbb9c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:26.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.970+0000 7fbbacc00640 1 -- 172.21.15.123:0/2445140538 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0125f0 con 0x7fbbb00cc5d0 2024-04-24T05:47:26.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.970+0000 7fbbacc00640 1 -- 172.21.15.123:0/2445140538 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb9c019070 con 0x7fbbb00cc5d0 2024-04-24T05:47:26.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.970+0000 7fbbacc00640 1 -- 172.21.15.123:0/2445140538 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0129d0 con 0x7fbbb00cc5d0 2024-04-24T05:47:26.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.970+0000 7fbbb6400640 1 -- 172.21.15.123:0/2445140538 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb00726b0 con 0x7fbbb00cc5d0 2024-04-24T05:47:26.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.970+0000 7fbbb6400640 1 -- 172.21.15.123:0/2445140538 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb0072c70 con 0x7fbbb00cc5d0 2024-04-24T05:47:26.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.971+0000 7fbbb6400640 1 -- 172.21.15.123:0/2445140538 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb0068a20 con 0x7fbbb00cc5d0 2024-04-24T05:47:26.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.972+0000 7fbbacc00640 1 -- 172.21.15.123:0/2445140538 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb9c012b70 con 0x7fbbb00cc5d0 2024-04-24T05:47:26.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.973+0000 7fbbacc00640 1 --2- 172.21.15.123:0/2445140538 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbb9006c8f0 0x7fbb9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:26.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.973+0000 7fbbacc00640 1 -- 172.21.15.123:0/2445140538 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fbb9c08c3b0 con 0x7fbbb00cc5d0 2024-04-24T05:47:26.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.973+0000 7fbbb5000640 1 --2- 172.21.15.123:0/2445140538 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbb9006c8f0 0x7fbb9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:26.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.974+0000 7fbbb5000640 1 --2- 172.21.15.123:0/2445140538 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbb9006c8f0 0x7fbb9006edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbba0000f60 tx=0x7fbba00078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:26.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:26.977+0000 7fbbacc00640 1 -- 172.21.15.123:0/2445140538 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb9c05a330 con 0x7fbbb00cc5d0 2024-04-24T05:47:27.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.178+0000 7fbbb6400640 1 -- 172.21.15.123:0/2445140538 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbbb006d100 con 0x7fbbb00cc5d0 2024-04-24T05:47:27.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.179+0000 7fbbacc00640 1 -- 172.21.15.123:0/2445140538 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fbb9c059c30 con 0x7fbbb00cc5d0 2024-04-24T05:47:27.181 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-04-24T05:47:27.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.182+0000 7fbbb6400640 1 -- 172.21.15.123:0/2445140538 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbb9006c8f0 msgr2=0x7fbb9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:27.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.182+0000 7fbbb6400640 1 --2- 172.21.15.123:0/2445140538 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbb9006c8f0 0x7fbb9006edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbba0000f60 tx=0x7fbba00078f0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.182+0000 7fbbb6400640 1 -- 172.21.15.123:0/2445140538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb00cc5d0 msgr2=0x7fbbb0071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:27.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.182+0000 7fbbb6400640 1 --2- 172.21.15.123:0/2445140538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb00cc5d0 0x7fbbb0071b80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00d5f0 tx=0x7fbb9c00dab0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.182+0000 7fbbb6400640 1 -- 172.21.15.123:0/2445140538 shutdown_connections 2024-04-24T05:47:27.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.182+0000 7fbbb6400640 1 --2- 172.21.15.123:0/2445140538 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbb9006c8f0 0x7fbb9006edb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.182+0000 7fbbb6400640 1 --2- 172.21.15.123:0/2445140538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbb015c2f0 0x7fbbb00c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.182+0000 7fbbb6400640 1 --2- 172.21.15.123:0/2445140538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbb00cc5d0 0x7fbbb0071b80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.182+0000 7fbbb6400640 1 --2- 172.21.15.123:0/2445140538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbbb00ad510 0x7fbbb00cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.182+0000 7fbbb6400640 1 -- 172.21.15.123:0/2445140538 >> 172.21.15.123:0/2445140538 conn(0x7fbbb00b1cf0 msgr2=0x7fbbb0156820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:27.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.182+0000 7fbbb6400640 1 -- 172.21.15.123:0/2445140538 shutdown_connections 2024-04-24T05:47:27.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.182+0000 7fbbb6400640 1 -- 172.21.15.123:0/2445140538 wait complete. 2024-04-24T05:47:27.203 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T05:47:27.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.397+0000 7f970ce00640 1 Processor -- start 2024-04-24T05:47:27.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.397+0000 7f970ce00640 1 -- start start 2024-04-24T05:47:27.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.399+0000 7f970ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970815e5f0 0x7f970815f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:27.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.399+0000 7f970ce00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97080d15f0 0x7f97080cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:27.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.399+0000 7f970ce00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9708162a60 con 0x7f9708162250 2024-04-24T05:47:27.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.399+0000 7f970ce00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f97080ceb50 con 0x7f97080d15f0 2024-04-24T05:47:27.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.399+0000 7f970ce00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f97080cee30 con 0x7f970815e5f0 2024-04-24T05:47:27.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.399+0000 7f9706a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9708162250 0x7f97080d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:43256/0 (socket says 172.21.15.123:43256) 2024-04-24T05:47:27.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.399+0000 7f9706a00640 1 -- 172.21.15.123:0/729664187 learned_addr learned my addr 172.21.15.123:0/729664187 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:27.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.399+0000 7f9707e00640 1 --2- 172.21.15.123:0/729664187 >> v2:172.21.15.123:3300/0 conn(0x7f97080d15f0 0x7f97080cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:27.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.400+0000 7f9707400640 1 --2- 172.21.15.123:0/729664187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970815e5f0 0x7f970815f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:27.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.400+0000 7f9706000640 1 -- 172.21.15.123:0/729664187 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2723077964 0 0) 0x7f9708162a60 con 0x7f9708162250 2024-04-24T05:47:27.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.400+0000 7f9706000640 1 -- 172.21.15.123:0/729664187 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96ec003520 con 0x7f9708162250 2024-04-24T05:47:27.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.400+0000 7f9707e00640 1 -- 172.21.15.123:0/729664187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970815e5f0 msgr2=0x7f970815f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:27.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.400+0000 7f9707e00640 1 --2- 172.21.15.123:0/729664187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970815e5f0 0x7f970815f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.400+0000 7f9707e00640 1 -- 172.21.15.123:0/729664187 >> v1:172.21.15.123:6789/0 conn(0x7f9708162250 legacy=0x7f97080d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:27.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.400+0000 7f9707e00640 1 -- 172.21.15.123:0/729664187 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9708162d40 con 0x7f97080d15f0 2024-04-24T05:47:27.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.401+0000 7f9707e00640 1 --2- 172.21.15.123:0/729664187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97080d15f0 0x7f97080cf2c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f97000097a0 tx=0x7f970002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd65a20cc15154d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:27.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.401+0000 7f9706000640 1 -- 172.21.15.123:0/729664187 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700037070 con 0x7f97080d15f0 2024-04-24T05:47:27.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.401+0000 7f9706000640 1 -- 172.21.15.123:0/729664187 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9700032440 con 0x7f97080d15f0 2024-04-24T05:47:27.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.401+0000 7f9706000640 1 -- 172.21.15.123:0/729664187 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000327f0 con 0x7f97080d15f0 2024-04-24T05:47:27.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.402+0000 7f970ce00640 1 -- 172.21.15.123:0/729664187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97080d15f0 msgr2=0x7f97080cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:27.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.402+0000 7f970ce00640 1 --2- 172.21.15.123:0/729664187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97080d15f0 0x7f97080cf2c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f97000097a0 tx=0x7f970002a180 comp rx=0 tx=0).stop 2024-04-24T05:47:27.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.402+0000 7f970ce00640 1 -- 172.21.15.123:0/729664187 shutdown_connections 2024-04-24T05:47:27.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.402+0000 7f970ce00640 1 --2- 172.21.15.123:0/729664187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97080d15f0 0x7f97080cf2c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.402+0000 7f970ce00640 1 --2- 172.21.15.123:0/729664187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970815e5f0 0x7f970815f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.402+0000 7f970ce00640 1 -- 172.21.15.123:0/729664187 >> 172.21.15.123:0/729664187 conn(0x7f97080b9db0 msgr2=0x7f970809d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:27.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.402+0000 7f970ce00640 1 -- 172.21.15.123:0/729664187 shutdown_connections 2024-04-24T05:47:27.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.402+0000 7f970ce00640 1 -- 172.21.15.123:0/729664187 wait complete. 2024-04-24T05:47:27.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.404+0000 7f970ce00640 1 Processor -- start 2024-04-24T05:47:27.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.404+0000 7f970ce00640 1 -- start start 2024-04-24T05:47:27.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f970ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97080d15f0 0x7f97080b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:27.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f970ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970815e5f0 0x7f9708152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:27.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f970ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9708162250 0x7f97081f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:27.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f970ce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f970815e9d0 con 0x7f9708162250 2024-04-24T05:47:27.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f970ce00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f97081626d0 con 0x7f97080d15f0 2024-04-24T05:47:27.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f970ce00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f970815eb50 con 0x7f970815e5f0 2024-04-24T05:47:27.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f9707e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9708162250 0x7f97081f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:27.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f9707e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9708162250 0x7f97081f7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33926/0 (socket says 172.21.15.123:33926) 2024-04-24T05:47:27.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f9707e00640 1 -- 172.21.15.123:0/669723015 learned_addr learned my addr 172.21.15.123:0/669723015 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:27.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f9707e00640 1 -- 172.21.15.123:0/669723015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970815e5f0 msgr2=0x7f9708152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:27.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f9706a00640 1 --2- 172.21.15.123:0/669723015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970815e5f0 0x7f9708152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:27.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f9707400640 1 --2- 172.21.15.123:0/669723015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97080d15f0 0x7f97080b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:27.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f9707e00640 1 --2- 172.21.15.123:0/669723015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970815e5f0 0x7f9708152650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f9707e00640 1 -- 172.21.15.123:0/669723015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97080d15f0 msgr2=0x7f97080b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:27.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f9707e00640 1 --2- 172.21.15.123:0/669723015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97080d15f0 0x7f97080b68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.405+0000 7f9707e00640 1 -- 172.21.15.123:0/669723015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97080cc9e0 con 0x7f9708162250 2024-04-24T05:47:27.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.406+0000 7f9707e00640 1 --2- 172.21.15.123:0/669723015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9708162250 0x7f97081f7e00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9700035040 tx=0x7f97000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:27.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.406+0000 7f96ff400640 1 -- 172.21.15.123:0/669723015 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700041030 con 0x7f9708162250 2024-04-24T05:47:27.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.406+0000 7f96ff400640 1 -- 172.21.15.123:0/669723015 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9700030440 con 0x7f9708162250 2024-04-24T05:47:27.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.406+0000 7f96ff400640 1 -- 172.21.15.123:0/669723015 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700037040 con 0x7f9708162250 2024-04-24T05:47:27.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.406+0000 7f970ce00640 1 -- 172.21.15.123:0/669723015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97080cccc0 con 0x7f9708162250 2024-04-24T05:47:27.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.406+0000 7f970ce00640 1 -- 172.21.15.123:0/669723015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97080b4ec0 con 0x7f9708162250 2024-04-24T05:47:27.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.407+0000 7f970ce00640 1 -- 172.21.15.123:0/669723015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9708068a20 con 0x7f9708162250 2024-04-24T05:47:27.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.408+0000 7f96ff400640 1 -- 172.21.15.123:0/669723015 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9700044070 con 0x7f9708162250 2024-04-24T05:47:27.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.409+0000 7f96ff400640 1 --2- 172.21.15.123:0/669723015 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f96e406c920 0x7f96e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:27.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.410+0000 7f96ff400640 1 -- 172.21.15.123:0/669723015 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f97000ac700 con 0x7f9708162250 2024-04-24T05:47:27.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.410+0000 7f9707400640 1 --2- 172.21.15.123:0/669723015 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f96e406c920 0x7f96e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:27.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.410+0000 7f9707400640 1 --2- 172.21.15.123:0/669723015 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f96e406c920 0x7f96e406ede0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009f30 tx=0x7f96f800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:27.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.414+0000 7f96ff400640 1 -- 172.21.15.123:0/669723015 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f970007a720 con 0x7f9708162250 2024-04-24T05:47:27.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.597+0000 7f970ce00640 1 -- 172.21.15.123:0/669723015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f970806eff0 con 0x7f9708162250 2024-04-24T05:47:27.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.598+0000 7f96ff400640 1 -- 172.21.15.123:0/669723015 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f970007a020 con 0x7f9708162250 2024-04-24T05:47:27.599 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-04-24T05:47:27.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.600+0000 7f970ce00640 1 -- 172.21.15.123:0/669723015 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f96e406c920 msgr2=0x7f96e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:27.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.600+0000 7f970ce00640 1 --2- 172.21.15.123:0/669723015 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f96e406c920 0x7f96e406ede0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009f30 tx=0x7f96f800b040 comp rx=0 tx=0).stop 2024-04-24T05:47:27.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.600+0000 7f970ce00640 1 -- 172.21.15.123:0/669723015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9708162250 msgr2=0x7f97081f7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:27.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.600+0000 7f970ce00640 1 --2- 172.21.15.123:0/669723015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9708162250 0x7f97081f7e00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9700035040 tx=0x7f97000046c0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.601+0000 7f970ce00640 1 -- 172.21.15.123:0/669723015 shutdown_connections 2024-04-24T05:47:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.601+0000 7f970ce00640 1 --2- 172.21.15.123:0/669723015 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f96e406c920 0x7f96e406ede0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.601+0000 7f970ce00640 1 --2- 172.21.15.123:0/669723015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9708162250 0x7f97081f7e00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.601+0000 7f970ce00640 1 --2- 172.21.15.123:0/669723015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970815e5f0 0x7f9708152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.601+0000 7f970ce00640 1 --2- 172.21.15.123:0/669723015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97080d15f0 0x7f97080b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.601+0000 7f970ce00640 1 -- 172.21.15.123:0/669723015 >> 172.21.15.123:0/669723015 conn(0x7f97080b9db0 msgr2=0x7f97080d34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.601+0000 7f970ce00640 1 -- 172.21.15.123:0/669723015 shutdown_connections 2024-04-24T05:47:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.601+0000 7f970ce00640 1 -- 172.21.15.123:0/669723015 wait complete. 2024-04-24T05:47:27.621 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T05:47:27.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.814+0000 7fd2a5c00640 1 Processor -- start 2024-04-24T05:47:27.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.814+0000 7fd2a5c00640 1 -- start start 2024-04-24T05:47:27.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.816+0000 7fd2a5c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a015e600 0x7fd2a00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:27.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.816+0000 7fd2a5c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd2a00d4b20 0x7fd2a00d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:27.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.816+0000 7fd2a5c00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2a00cffb0 con 0x7fd2a00d1490 2024-04-24T05:47:27.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.816+0000 7fd2a5c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a0162560 con 0x7fd2a00d4b20 2024-04-24T05:47:27.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.816+0000 7fd2a5c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a0162840 con 0x7fd2a015e600 2024-04-24T05:47:27.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.816+0000 7fd2a5200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd2a00d4b20 0x7fd2a00d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:27.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.816+0000 7fd2a5200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd2a00d4b20 0x7fd2a00d1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33940/0 (socket says 172.21.15.123:33940) 2024-04-24T05:47:27.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.816+0000 7fd2a5200640 1 -- 172.21.15.123:0/1185674021 learned_addr learned my addr 172.21.15.123:0/1185674021 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:27.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.817+0000 7fd29fe00640 1 --2- 172.21.15.123:0/1185674021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a015e600 0x7fd2a00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:27.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.817+0000 7fd2a5200640 1 -- 172.21.15.123:0/1185674021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a015e600 msgr2=0x7fd2a00cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:27.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.817+0000 7fd2a5200640 1 --2- 172.21.15.123:0/1185674021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a015e600 0x7fd2a00cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.817+0000 7fd2a5200640 1 -- 172.21.15.123:0/1185674021 >> v1:172.21.15.155:6789/0 conn(0x7fd2a00d1490 legacy=0x7fd2a00d4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:27.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.817+0000 7fd2a5200640 1 -- 172.21.15.123:0/1185674021 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a0162b20 con 0x7fd2a00d4b20 2024-04-24T05:47:27.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.818+0000 7fd2a5200640 1 --2- 172.21.15.123:0/1185674021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a00d4b20 0x7fd2a00d1910 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd298009e50 tx=0x7fd29802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b3b2630e3e09d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:27.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.818+0000 7fd29ea00640 1 -- 172.21.15.123:0/1185674021 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298036070 con 0x7fd2a00d4b20 2024-04-24T05:47:27.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.818+0000 7fd29ea00640 1 -- 172.21.15.123:0/1185674021 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd298031600 con 0x7fd2a00d4b20 2024-04-24T05:47:27.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.818+0000 7fd29ea00640 1 -- 172.21.15.123:0/1185674021 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2980319d0 con 0x7fd2a00d4b20 2024-04-24T05:47:27.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.819+0000 7fd2a5c00640 1 -- 172.21.15.123:0/1185674021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a00d4b20 msgr2=0x7fd2a00d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:27.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.819+0000 7fd2a5c00640 1 --2- 172.21.15.123:0/1185674021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a00d4b20 0x7fd2a00d1910 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd298009e50 tx=0x7fd29802a630 comp rx=0 tx=0).stop 2024-04-24T05:47:27.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.819+0000 7fd2a5c00640 1 -- 172.21.15.123:0/1185674021 shutdown_connections 2024-04-24T05:47:27.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.819+0000 7fd2a5c00640 1 --2- 172.21.15.123:0/1185674021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a00d4b20 0x7fd2a00d1910 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.819+0000 7fd2a5c00640 1 --2- 172.21.15.123:0/1185674021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a015e600 0x7fd2a00cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.819+0000 7fd2a5c00640 1 -- 172.21.15.123:0/1185674021 >> 172.21.15.123:0/1185674021 conn(0x7fd2a00b9440 msgr2=0x7fd2a00b9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:27.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.819+0000 7fd2a5c00640 1 -- 172.21.15.123:0/1185674021 shutdown_connections 2024-04-24T05:47:27.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.819+0000 7fd2a5c00640 1 -- 172.21.15.123:0/1185674021 wait complete. 2024-04-24T05:47:27.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.820+0000 7fd2a5c00640 1 Processor -- start 2024-04-24T05:47:27.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.821+0000 7fd2a5c00640 1 -- start start 2024-04-24T05:47:27.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.821+0000 7fd2a5c00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a00d1490 0x7fd2a01554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:27.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd2a5c00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00d4b20 0x7fd2a0161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:27.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd2a5c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a015e600 0x7fd2a01f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:27.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd2a5c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a015e9e0 con 0x7fd2a015e600 2024-04-24T05:47:27.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd2a5c00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a01621f0 con 0x7fd2a00d4b20 2024-04-24T05:47:27.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd2a5c00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a0161bc0 con 0x7fd2a00d1490 2024-04-24T05:47:27.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd2a5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a015e600 0x7fd2a01f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:27.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd2a5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a015e600 0x7fd2a01f7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33942/0 (socket says 172.21.15.123:33942) 2024-04-24T05:47:27.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd2a5200640 1 -- 172.21.15.123:0/2886920360 learned_addr learned my addr 172.21.15.123:0/2886920360 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:27.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd29fe00640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a00d1490 0x7fd2a01554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:27.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd29f400640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00d4b20 0x7fd2a0161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:27.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd2a5200640 1 -- 172.21.15.123:0/2886920360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a00d1490 msgr2=0x7fd2a01554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:27.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd2a5200640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a00d1490 0x7fd2a01554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd2a5200640 1 -- 172.21.15.123:0/2886920360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00d4b20 msgr2=0x7fd2a0161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:27.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd2a5200640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00d4b20 0x7fd2a0161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:27.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd2a5200640 1 -- 172.21.15.123:0/2886920360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a00cd2b0 con 0x7fd2a015e600 2024-04-24T05:47:27.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.822+0000 7fd29f400640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00d4b20 0x7fd2a0161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:27.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.823+0000 7fd2a5200640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a015e600 0x7fd2a01f7c90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd298009e50 tx=0x7fd298004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:27.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.823+0000 7fd297e00640 1 -- 172.21.15.123:0/2886920360 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298036050 con 0x7fd2a015e600 2024-04-24T05:47:27.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.823+0000 7fd297e00640 1 -- 172.21.15.123:0/2886920360 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd298042070 con 0x7fd2a015e600 2024-04-24T05:47:27.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.823+0000 7fd297e00640 1 -- 172.21.15.123:0/2886920360 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298009510 con 0x7fd2a015e600 2024-04-24T05:47:27.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.823+0000 7fd2a5c00640 1 -- 172.21.15.123:0/2886920360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a00cd550 con 0x7fd2a015e600 2024-04-24T05:47:27.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.823+0000 7fd2a5c00640 1 -- 172.21.15.123:0/2886920360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a00cdb10 con 0x7fd2a015e600 2024-04-24T05:47:27.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.824+0000 7fd2a5c00640 1 -- 172.21.15.123:0/2886920360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a0075b70 con 0x7fd2a015e600 2024-04-24T05:47:27.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.825+0000 7fd297e00640 1 -- 172.21.15.123:0/2886920360 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd298007bd0 con 0x7fd2a015e600 2024-04-24T05:47:27.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.826+0000 7fd297e00640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd28006c8f0 0x7fd28006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:27.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.826+0000 7fd297e00640 1 -- 172.21.15.123:0/2886920360 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fd2980acf30 con 0x7fd2a015e600 2024-04-24T05:47:27.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.826+0000 7fd29fe00640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd28006c8f0 0x7fd28006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:27.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.827+0000 7fd29fe00640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd28006c8f0 0x7fd28006edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00cd1e0 tx=0x7fd290008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:27.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:27.830+0000 7fd297e00640 1 -- 172.21.15.123:0/2886920360 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd298045360 con 0x7fd2a015e600 2024-04-24T05:47:28.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.052+0000 7fd2a5c00640 1 -- 172.21.15.123:0/2886920360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2a00b9a70 con 0x7fd2a015e600 2024-04-24T05:47:28.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.053+0000 7fd297e00640 1 -- 172.21.15.123:0/2886920360 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7fd29807af40 con 0x7fd2a015e600 2024-04-24T05:47:28.057 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:28.057 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2743935658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2743935658},{"type":"v1","addr":"172.21.15.155:6834","nonce":2743935658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1148561953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1148561953},{"type":"v1","addr":"172.21.15.123:6836","nonce":1148561953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2992052647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2992052647},{"type":"v1","addr":"172.21.15.155:6835","nonce":2992052647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T05:45:59.639548+0000","modified":"2024-04-24T05:46:19.813408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/3370584410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3370584410},{"type":"v1","addr":"172.21.15.123:6837","nonce":3370584410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T05:47:28.059 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-24T05:47:28.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.056+0000 7fd2a5c00640 1 -- 172.21.15.123:0/2886920360 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd28006c8f0 msgr2=0x7fd28006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:28.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.056+0000 7fd2a5c00640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd28006c8f0 0x7fd28006edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00cd1e0 tx=0x7fd290008040 comp rx=0 tx=0).stop 2024-04-24T05:47:28.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.056+0000 7fd2a5c00640 1 -- 172.21.15.123:0/2886920360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a015e600 msgr2=0x7fd2a01f7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:28.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.056+0000 7fd2a5c00640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a015e600 0x7fd2a01f7c90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd298009e50 tx=0x7fd298004a40 comp rx=0 tx=0).stop 2024-04-24T05:47:28.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.056+0000 7fd2a5c00640 1 -- 172.21.15.123:0/2886920360 shutdown_connections 2024-04-24T05:47:28.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.056+0000 7fd2a5c00640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fd28006c8f0 0x7fd28006edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.056+0000 7fd2a5c00640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a015e600 0x7fd2a01f7c90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.056+0000 7fd2a5c00640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2a00d4b20 0x7fd2a0161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.056+0000 7fd2a5c00640 1 --2- 172.21.15.123:0/2886920360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2a00d1490 0x7fd2a01554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.056+0000 7fd2a5c00640 1 -- 172.21.15.123:0/2886920360 >> 172.21.15.123:0/2886920360 conn(0x7fd2a00b9440 msgr2=0x7fd2a01568c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:28.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.056+0000 7fd2a5c00640 1 -- 172.21.15.123:0/2886920360 shutdown_connections 2024-04-24T05:47:28.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.056+0000 7fd2a5c00640 1 -- 172.21.15.123:0/2886920360 wait complete. 2024-04-24T05:47:28.076 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:28.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.266+0000 7fb96f600640 1 Processor -- start 2024-04-24T05:47:28.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.267+0000 7fb96f600640 1 -- start start 2024-04-24T05:47:28.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.268+0000 7fb96f600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9680ad860 0x7fb9680af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:28.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.268+0000 7fb96f600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb9680af780 0x7fb96806bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:28.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.268+0000 7fb96f600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9680afbd0 con 0x7fb96815a2a0 2024-04-24T05:47:28.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.268+0000 7fb96f600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb96815a680 con 0x7fb9680af780 2024-04-24T05:47:28.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.268+0000 7fb96f600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb96815da00 con 0x7fb9680ad860 2024-04-24T05:47:28.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.269+0000 7fb95fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9680ad860 0x7fb9680af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:28.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.269+0000 7fb95fe00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9680ad860 0x7fb9680af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:44684/0 (socket says 172.21.15.123:44684) 2024-04-24T05:47:28.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.269+0000 7fb95fe00640 1 -- 172.21.15.123:0/4029653528 learned_addr learned my addr 172.21.15.123:0/4029653528 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:28.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.269+0000 7fb96cc00640 1 --1- 172.21.15.123:0/4029653528 >> v1:172.21.15.155:6789/0 conn(0x7fb96815a2a0 0x7fb9680ad150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.123:34616/0 (socket says 172.21.15.123:34616) 2024-04-24T05:47:28.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.269+0000 7fb96d600640 1 --2- 172.21.15.123:0/4029653528 >> v2:172.21.15.155:3300/0 conn(0x7fb9680af780 0x7fb96806bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:28.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.270+0000 7fb95fe00640 1 -- 172.21.15.123:0/4029653528 >> v2:172.21.15.155:3300/0 conn(0x7fb9680af780 msgr2=0x7fb96806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:28.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.270+0000 7fb95fe00640 1 --2- 172.21.15.123:0/4029653528 >> v2:172.21.15.155:3300/0 conn(0x7fb9680af780 0x7fb96806bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.270+0000 7fb95fe00640 1 -- 172.21.15.123:0/4029653528 >> v1:172.21.15.155:6789/0 conn(0x7fb96815a2a0 legacy=0x7fb9680ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:28.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.270+0000 7fb95fe00640 1 -- 172.21.15.123:0/4029653528 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96815cdf0 con 0x7fb9680ad860 2024-04-24T05:47:28.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.270+0000 7fb967e00640 1 -- 172.21.15.123:0/4029653528 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1317235742 0 0) 0x7fb9680afbd0 con 0x7fb96815a2a0 2024-04-24T05:47:28.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.271+0000 7fb95fe00640 1 --2- 172.21.15.123:0/4029653528 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9680ad860 0x7fb9680af240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb950009870 tx=0x7fb95002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0e48ce5762c9b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.272+0000 7fb967e00640 1 -- 172.21.15.123:0/4029653528 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950037070 con 0x7fb9680ad860 2024-04-24T05:47:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.272+0000 7fb967e00640 1 -- 172.21.15.123:0/4029653528 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb950032440 con 0x7fb9680ad860 2024-04-24T05:47:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.272+0000 7fb967e00640 1 -- 172.21.15.123:0/4029653528 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950032810 con 0x7fb9680ad860 2024-04-24T05:47:28.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.272+0000 7fb96f600640 1 -- 172.21.15.123:0/4029653528 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9680ad860 msgr2=0x7fb9680af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:28.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.272+0000 7fb96f600640 1 --2- 172.21.15.123:0/4029653528 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9680ad860 0x7fb9680af240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb950009870 tx=0x7fb95002a1c0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.272+0000 7fb96f600640 1 -- 172.21.15.123:0/4029653528 shutdown_connections 2024-04-24T05:47:28.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.272+0000 7fb96f600640 1 --2- 172.21.15.123:0/4029653528 >> v2:172.21.15.155:3300/0 conn(0x7fb9680af780 0x7fb96806bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.272+0000 7fb96f600640 1 --2- 172.21.15.123:0/4029653528 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9680ad860 0x7fb9680af240 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.272+0000 7fb96f600640 1 -- 172.21.15.123:0/4029653528 >> 172.21.15.123:0/4029653528 conn(0x7fb9680b1e70 msgr2=0x7fb9680b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:28.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.273+0000 7fb96f600640 1 -- 172.21.15.123:0/4029653528 shutdown_connections 2024-04-24T05:47:28.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.273+0000 7fb96f600640 1 -- 172.21.15.123:0/4029653528 wait complete. 2024-04-24T05:47:28.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.274+0000 7fb96f600640 1 Processor -- start 2024-04-24T05:47:28.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.274+0000 7fb96f600640 1 -- start start 2024-04-24T05:47:28.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.275+0000 7fb96f600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9680ad860 0x7fb9681f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:28.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.275+0000 7fb96f600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9680af780 0x7fb968158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:28.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.275+0000 7fb96f600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb96815a2a0 0x7fb96815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:28.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.275+0000 7fb96f600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb96815d690 con 0x7fb9680af780 2024-04-24T05:47:28.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.275+0000 7fb96f600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb96815d3b0 con 0x7fb9680ad860 2024-04-24T05:47:28.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.275+0000 7fb96f600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb96806c040 con 0x7fb96815a2a0 2024-04-24T05:47:28.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.275+0000 7fb95fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9680af780 0x7fb968158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:28.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.276+0000 7fb95fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9680af780 0x7fb968158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33956/0 (socket says 172.21.15.123:33956) 2024-04-24T05:47:28.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.276+0000 7fb95fe00640 1 -- 172.21.15.123:0/197517777 learned_addr learned my addr 172.21.15.123:0/197517777 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:28.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.276+0000 7fb96cc00640 1 --2- 172.21.15.123:0/197517777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9680ad860 0x7fb9681f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:28.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.276+0000 7fb96d600640 1 --2- 172.21.15.123:0/197517777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb96815a2a0 0x7fb96815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:28.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.276+0000 7fb95fe00640 1 -- 172.21.15.123:0/197517777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb96815a2a0 msgr2=0x7fb96815ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:28.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.276+0000 7fb95fe00640 1 --2- 172.21.15.123:0/197517777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb96815a2a0 0x7fb96815ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.276+0000 7fb95fe00640 1 -- 172.21.15.123:0/197517777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9680ad860 msgr2=0x7fb9681f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:28.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.276+0000 7fb95fe00640 1 --2- 172.21.15.123:0/197517777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9680ad860 0x7fb9681f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.276+0000 7fb95fe00640 1 -- 172.21.15.123:0/197517777 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9681f4180 con 0x7fb9680af780 2024-04-24T05:47:28.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.276+0000 7fb95fe00640 1 --2- 172.21.15.123:0/197517777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9680af780 0x7fb968158e20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb950000c00 tx=0x7fb9500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:28.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.277+0000 7fb965600640 1 -- 172.21.15.123:0/197517777 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950037040 con 0x7fb9680af780 2024-04-24T05:47:28.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.277+0000 7fb965600640 1 -- 172.21.15.123:0/197517777 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb950042070 con 0x7fb9680af780 2024-04-24T05:47:28.279 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.277+0000 7fb965600640 1 -- 172.21.15.123:0/197517777 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950030940 con 0x7fb9680af780 2024-04-24T05:47:28.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.277+0000 7fb96f600640 1 -- 172.21.15.123:0/197517777 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9681f4460 con 0x7fb9680af780 2024-04-24T05:47:28.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.277+0000 7fb96f600640 1 -- 172.21.15.123:0/197517777 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb968159360 con 0x7fb9680af780 2024-04-24T05:47:28.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.278+0000 7fb96f600640 1 -- 172.21.15.123:0/197517777 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb968068a20 con 0x7fb9680af780 2024-04-24T05:47:28.280 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.279+0000 7fb965600640 1 -- 172.21.15.123:0/197517777 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb95002ad10 con 0x7fb9680af780 2024-04-24T05:47:28.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.279+0000 7fb965600640 1 --2- 172.21.15.123:0/197517777 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb94406c920 0x7fb94406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:28.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.280+0000 7fb965600640 1 -- 172.21.15.123:0/197517777 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fb9500ac460 con 0x7fb9680af780 2024-04-24T05:47:28.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.280+0000 7fb96cc00640 1 --2- 172.21.15.123:0/197517777 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb94406c920 0x7fb94406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:28.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.280+0000 7fb96cc00640 1 --2- 172.21.15.123:0/197517777 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb94406c920 0x7fb94406ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb958000f60 tx=0x7fb958007d00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:28.285 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.284+0000 7fb965600640 1 -- 172.21.15.123:0/197517777 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb95007a470 con 0x7fb9680af780 2024-04-24T05:47:28.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.483+0000 7fb96f600640 1 -- 172.21.15.123:0/197517777 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb96806fdf0 con 0x7fb9680af780 2024-04-24T05:47:28.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.485+0000 7fb965600640 1 -- 172.21.15.123:0/197517777 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7fb950079d70 con 0x7fb9680af780 2024-04-24T05:47:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:28.487 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:28.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.488+0000 7fb96f600640 1 -- 172.21.15.123:0/197517777 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb94406c920 msgr2=0x7fb94406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.488+0000 7fb96f600640 1 --2- 172.21.15.123:0/197517777 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb94406c920 0x7fb94406ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb958000f60 tx=0x7fb958007d00 comp rx=0 tx=0).stop 2024-04-24T05:47:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.488+0000 7fb96f600640 1 -- 172.21.15.123:0/197517777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9680af780 msgr2=0x7fb968158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.488+0000 7fb96f600640 1 --2- 172.21.15.123:0/197517777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9680af780 0x7fb968158e20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb950000c00 tx=0x7fb9500046c0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.488+0000 7fb96f600640 1 -- 172.21.15.123:0/197517777 shutdown_connections 2024-04-24T05:47:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.488+0000 7fb96f600640 1 --2- 172.21.15.123:0/197517777 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb94406c920 0x7fb94406ede0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.488+0000 7fb96f600640 1 --2- 172.21.15.123:0/197517777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb96815a2a0 0x7fb96815ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.488+0000 7fb96f600640 1 --2- 172.21.15.123:0/197517777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9680af780 0x7fb968158e20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.488+0000 7fb96f600640 1 --2- 172.21.15.123:0/197517777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9680ad860 0x7fb9681f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.488+0000 7fb96f600640 1 -- 172.21.15.123:0/197517777 >> 172.21.15.123:0/197517777 conn(0x7fb9680b1e70 msgr2=0x7fb96815f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.488+0000 7fb96f600640 1 -- 172.21.15.123:0/197517777 shutdown_connections 2024-04-24T05:47:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.488+0000 7fb96f600640 1 -- 172.21.15.123:0/197517777 wait complete. 2024-04-24T05:47:28.509 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T05:47:28.510 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T05:47:28.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.702+0000 7f782e000640 1 Processor -- start 2024-04-24T05:47:28.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.702+0000 7f782e000640 1 -- start start 2024-04-24T05:47:28.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.704+0000 7f782e000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f782815e720 0x7f78280cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.704+0000 7f782e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f78280d0040 0x7f78281623d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.704+0000 7f782e000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78280d04c0 con 0x7f7828162910 2024-04-24T05:47:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.704+0000 7f782e000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f78280b5f40 con 0x7f78280d0040 2024-04-24T05:47:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.704+0000 7f782e000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7828162d40 con 0x7f782815e720 2024-04-24T05:47:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.704+0000 7f7826a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f78280d0040 0x7f78281623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.704+0000 7f7826a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f78280d0040 0x7f78281623d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33970/0 (socket says 172.21.15.123:33970) 2024-04-24T05:47:28.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.704+0000 7f7826a00640 1 -- 172.21.15.123:0/3590571919 learned_addr learned my addr 172.21.15.123:0/3590571919 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:28.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.704+0000 7f7827400640 1 --2- 172.21.15.123:0/3590571919 >> v2:172.21.15.155:3300/0 conn(0x7f782815e720 0x7f78280cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:28.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.705+0000 7f7826a00640 1 -- 172.21.15.123:0/3590571919 >> v2:172.21.15.155:3300/0 conn(0x7f782815e720 msgr2=0x7f78280cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:28.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.705+0000 7f7826a00640 1 --2- 172.21.15.123:0/3590571919 >> v2:172.21.15.155:3300/0 conn(0x7f782815e720 0x7f78280cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.705+0000 7f7826a00640 1 -- 172.21.15.123:0/3590571919 >> v1:172.21.15.123:6789/0 conn(0x7f7828162910 legacy=0x7f78280cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:28.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.705+0000 7f7826a00640 1 -- 172.21.15.123:0/3590571919 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78280d07a0 con 0x7f78280d0040 2024-04-24T05:47:28.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.706+0000 7f7826a00640 1 --2- 172.21.15.123:0/3590571919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78280d0040 0x7f78281623d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f782806be90 tx=0x7f781402a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56196ee5e71b1f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:28.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.706+0000 7f7826000640 1 -- 172.21.15.123:0/3590571919 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814037070 con 0x7f78280d0040 2024-04-24T05:47:28.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.706+0000 7f7826000640 1 -- 172.21.15.123:0/3590571919 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7814032640 con 0x7f78280d0040 2024-04-24T05:47:28.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.706+0000 7f7826000640 1 -- 172.21.15.123:0/3590571919 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781402ae60 con 0x7f78280d0040 2024-04-24T05:47:28.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.707+0000 7f782e000640 1 -- 172.21.15.123:0/3590571919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78280d0040 msgr2=0x7f78281623d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:28.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.707+0000 7f782e000640 1 --2- 172.21.15.123:0/3590571919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78280d0040 0x7f78281623d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f782806be90 tx=0x7f781402a310 comp rx=0 tx=0).stop 2024-04-24T05:47:28.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.707+0000 7f782e000640 1 -- 172.21.15.123:0/3590571919 shutdown_connections 2024-04-24T05:47:28.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.707+0000 7f782e000640 1 --2- 172.21.15.123:0/3590571919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78280d0040 0x7f78281623d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.707+0000 7f782e000640 1 --2- 172.21.15.123:0/3590571919 >> v2:172.21.15.155:3300/0 conn(0x7f782815e720 0x7f78280cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.707+0000 7f782e000640 1 -- 172.21.15.123:0/3590571919 >> 172.21.15.123:0/3590571919 conn(0x7f78280705e0 msgr2=0x7f78280709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:28.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.707+0000 7f782e000640 1 -- 172.21.15.123:0/3590571919 shutdown_connections 2024-04-24T05:47:28.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.707+0000 7f782e000640 1 -- 172.21.15.123:0/3590571919 wait complete. 2024-04-24T05:47:28.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.709+0000 7f782e000640 1 Processor -- start 2024-04-24T05:47:28.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.709+0000 7f782e000640 1 -- start start 2024-04-24T05:47:28.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.710+0000 7f782e000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78280d0040 0x7f782815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:28.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.710+0000 7f782e000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782815e720 0x7f7828160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:28.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.710+0000 7f782e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7828162910 0x7f782814d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:28.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.710+0000 7f782e000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78280b4a10 con 0x7f7828162910 2024-04-24T05:47:28.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.710+0000 7f782e000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78280b4730 con 0x7f78280d0040 2024-04-24T05:47:28.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.710+0000 7f782e000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f78280ceb00 con 0x7f782815e720 2024-04-24T05:47:28.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.710+0000 7f7827e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7828162910 0x7f782814d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:28.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.710+0000 7f7827e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7828162910 0x7f782814d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33980/0 (socket says 172.21.15.123:33980) 2024-04-24T05:47:28.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.710+0000 7f7827e00640 1 -- 172.21.15.123:0/3025321720 learned_addr learned my addr 172.21.15.123:0/3025321720 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:28.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.710+0000 7f7827400640 1 --2- 172.21.15.123:0/3025321720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78280d0040 0x7f782815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:28.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.711+0000 7f7826a00640 1 --2- 172.21.15.123:0/3025321720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782815e720 0x7f7828160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:28.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.711+0000 7f7827e00640 1 -- 172.21.15.123:0/3025321720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782815e720 msgr2=0x7f7828160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.711+0000 7f7827e00640 1 --2- 172.21.15.123:0/3025321720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782815e720 0x7f7828160150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.711+0000 7f7827e00640 1 -- 172.21.15.123:0/3025321720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78280d0040 msgr2=0x7f782815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.711+0000 7f7827e00640 1 --2- 172.21.15.123:0/3025321720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78280d0040 0x7f782815c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.711+0000 7f7827e00640 1 -- 172.21.15.123:0/3025321720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782815d990 con 0x7f7828162910 2024-04-24T05:47:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.711+0000 7f7827e00640 1 --2- 172.21.15.123:0/3025321720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7828162910 0x7f782814d5c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f782000c6b0 tx=0x7f782000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.711+0000 7f781f400640 1 -- 172.21.15.123:0/3025321720 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820003b70 con 0x7f7828162910 2024-04-24T05:47:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.711+0000 7f781f400640 1 -- 172.21.15.123:0/3025321720 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7820003d30 con 0x7f7828162910 2024-04-24T05:47:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.711+0000 7f781f400640 1 -- 172.21.15.123:0/3025321720 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820005310 con 0x7f7828162910 2024-04-24T05:47:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.711+0000 7f782e000640 1 -- 172.21.15.123:0/3025321720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782815f6b0 con 0x7f7828162910 2024-04-24T05:47:28.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.711+0000 7f782e000640 1 -- 172.21.15.123:0/3025321720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f782815f950 con 0x7f7828162910 2024-04-24T05:47:28.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.712+0000 7f782e000640 1 -- 172.21.15.123:0/3025321720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7828068a20 con 0x7f7828162910 2024-04-24T05:47:28.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.713+0000 7f781f400640 1 -- 172.21.15.123:0/3025321720 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78200054b0 con 0x7f7828162910 2024-04-24T05:47:28.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.714+0000 7f781f400640 1 --2- 172.21.15.123:0/3025321720 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f780806c8f0 0x7f780806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:28.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.714+0000 7f781f400640 1 -- 172.21.15.123:0/3025321720 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f782008be60 con 0x7f7828162910 2024-04-24T05:47:28.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.715+0000 7f7827400640 1 --2- 172.21.15.123:0/3025321720 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f780806c8f0 0x7f780806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:28.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.715+0000 7f7827400640 1 --2- 172.21.15.123:0/3025321720 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f780806c8f0 0x7f780806edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7818006fd0 tx=0x7f7818008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:28.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.719+0000 7f781f400640 1 -- 172.21.15.123:0/3025321720 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7820059de0 con 0x7f7828162910 2024-04-24T05:47:28.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.967+0000 7f782e000640 1 -- 172.21.15.123:0/3025321720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78280b6690 con 0x7f7828162910 2024-04-24T05:47:28.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.968+0000 7f781f400640 1 -- 172.21.15.123:0/3025321720 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f78200596e0 con 0x7f7828162910 2024-04-24T05:47:28.970 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:28.970 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2743935658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2743935658},{"type":"v1","addr":"172.21.15.155:6834","nonce":2743935658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1148561953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1148561953},{"type":"v1","addr":"172.21.15.123:6836","nonce":1148561953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2992052647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2992052647},{"type":"v1","addr":"172.21.15.155:6835","nonce":2992052647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T05:45:59.639548+0000","modified":"2024-04-24T05:46:19.813408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/3370584410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3370584410},{"type":"v1","addr":"172.21.15.123:6837","nonce":3370584410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T05:47:28.970 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-24T05:47:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.971+0000 7f782e000640 1 -- 172.21.15.123:0/3025321720 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f780806c8f0 msgr2=0x7f780806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.971+0000 7f782e000640 1 --2- 172.21.15.123:0/3025321720 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f780806c8f0 0x7f780806edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7818006fd0 tx=0x7f7818008040 comp rx=0 tx=0).stop 2024-04-24T05:47:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.971+0000 7f782e000640 1 -- 172.21.15.123:0/3025321720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7828162910 msgr2=0x7f782814d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.971+0000 7f782e000640 1 --2- 172.21.15.123:0/3025321720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7828162910 0x7f782814d5c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f782000c6b0 tx=0x7f782000cb70 comp rx=0 tx=0).stop 2024-04-24T05:47:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.971+0000 7f782e000640 1 -- 172.21.15.123:0/3025321720 shutdown_connections 2024-04-24T05:47:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.971+0000 7f782e000640 1 --2- 172.21.15.123:0/3025321720 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f780806c8f0 0x7f780806edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.971+0000 7f782e000640 1 --2- 172.21.15.123:0/3025321720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7828162910 0x7f782814d5c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.971+0000 7f782e000640 1 --2- 172.21.15.123:0/3025321720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f782815e720 0x7f7828160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.971+0000 7f782e000640 1 --2- 172.21.15.123:0/3025321720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78280d0040 0x7f782815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:28.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.971+0000 7f782e000640 1 -- 172.21.15.123:0/3025321720 >> 172.21.15.123:0/3025321720 conn(0x7f78280705e0 msgr2=0x7f78280cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:28.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.971+0000 7f782e000640 1 -- 172.21.15.123:0/3025321720 shutdown_connections 2024-04-24T05:47:28.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:28.971+0000 7f782e000640 1 -- 172.21.15.123:0/3025321720 wait complete. 2024-04-24T05:47:28.991 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T05:47:29.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.183+0000 7f6c4a000640 1 Processor -- start 2024-04-24T05:47:29.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.183+0000 7f6c4a000640 1 -- start start 2024-04-24T05:47:29.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.184+0000 7f6c4a000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6c440ad650 0x7f6c440ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.184+0000 7f6c4a000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c44160120 con 0x7f6c4415c5d0 2024-04-24T05:47:29.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.184+0000 7f6c4a000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c4406dd80 con 0x7f6c440adee0 2024-04-24T05:47:29.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.184+0000 7f6c4a000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6c44160470 con 0x7f6c440ad650 2024-04-24T05:47:29.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.185+0000 7f6c48c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6c4415c5d0 0x7f6c44160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33514/0 (socket says 172.21.15.123:33514) 2024-04-24T05:47:29.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.185+0000 7f6c48c00640 1 -- 172.21.15.123:0/796279642 learned_addr learned my addr 172.21.15.123:0/796279642 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:29.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.185+0000 7f6c49600640 1 --2- 172.21.15.123:0/796279642 >> v2:172.21.15.155:3300/0 conn(0x7f6c440ad650 0x7f6c440ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:29.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.185+0000 7f6c43400640 1 -- 172.21.15.123:0/796279642 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1894153506 0 0) 0x7f6c44160120 con 0x7f6c4415c5d0 2024-04-24T05:47:29.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.185+0000 7f6c43400640 1 -- 172.21.15.123:0/796279642 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c28003520 con 0x7f6c4415c5d0 2024-04-24T05:47:29.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c43400640 1 -- 172.21.15.123:0/796279642 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1610655886 0 0) 0x7f6c28003520 con 0x7f6c4415c5d0 2024-04-24T05:47:29.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c43400640 1 -- 172.21.15.123:0/796279642 >> v2:172.21.15.155:3300/0 conn(0x7f6c440ad650 msgr2=0x7f6c440ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:29.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c43400640 1 --2- 172.21.15.123:0/796279642 >> v2:172.21.15.155:3300/0 conn(0x7f6c440ad650 0x7f6c440ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c43400640 1 -- 172.21.15.123:0/796279642 >> v1:172.21.15.155:6789/0 conn(0x7f6c440adee0 legacy=0x7f6c440acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:29.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c43400640 1 -- 172.21.15.123:0/796279642 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4406e060 con 0x7f6c4415c5d0 2024-04-24T05:47:29.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c43400640 1 -- 172.21.15.123:0/796279642 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f6c340019f0 con 0x7f6c4415c5d0 2024-04-24T05:47:29.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c43400640 1 -- 172.21.15.123:0/796279642 >> v1:172.21.15.123:6789/0 conn(0x7f6c4415c5d0 legacy=0x7f6c44160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:29.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c43400640 1 --2- 172.21.15.123:0/796279642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c280039b0 0x7f6c28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c43400640 1 --2- 172.21.15.123:0/796279642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c28004800 0x7f6c28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c43400640 1 --2- 172.21.15.123:0/796279642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c280051c0 0x7f6c2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c43400640 1 -- 172.21.15.123:0/796279642 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44160470 con 0x7f6c280051c0 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c43400640 1 -- 172.21.15.123:0/796279642 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2800c110 con 0x7f6c28004800 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c43400640 1 -- 172.21.15.123:0/796279642 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2800c310 con 0x7f6c280039b0 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c49600640 1 --2- 172.21.15.123:0/796279642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c280051c0 0x7f6c2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c48c00640 1 --2- 172.21.15.123:0/796279642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c280039b0 0x7f6c28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.186+0000 7f6c43e00640 1 --2- 172.21.15.123:0/796279642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c28004800 0x7f6c28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.187+0000 7f6c49600640 1 -- 172.21.15.123:0/796279642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c280039b0 msgr2=0x7f6c28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.187+0000 7f6c49600640 1 --2- 172.21.15.123:0/796279642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c280039b0 0x7f6c28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.187+0000 7f6c49600640 1 -- 172.21.15.123:0/796279642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c28004800 msgr2=0x7f6c28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.187+0000 7f6c49600640 1 --2- 172.21.15.123:0/796279642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c28004800 0x7f6c28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.187+0000 7f6c49600640 1 -- 172.21.15.123:0/796279642 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c28003520 con 0x7f6c280051c0 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.187+0000 7f6c43e00640 1 --2- 172.21.15.123:0/796279642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c28004800 0x7f6c28004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.187+0000 7f6c49600640 1 -- 172.21.15.123:0/796279642 reap_dead start 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.188+0000 7f6c49600640 1 --2- 172.21.15.123:0/796279642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c280051c0 0x7f6c2800b9c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6c44071b30 tx=0x7f6c28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4559b406730eab1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:29.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.188+0000 7f6c43400640 1 -- 172.21.15.123:0/796279642 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c030070 con 0x7f6c280051c0 2024-04-24T05:47:29.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.188+0000 7f6c43400640 1 -- 172.21.15.123:0/796279642 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c3c02adb0 con 0x7f6c280051c0 2024-04-24T05:47:29.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.188+0000 7f6c43400640 1 -- 172.21.15.123:0/796279642 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c00b610 con 0x7f6c280051c0 2024-04-24T05:47:29.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.188+0000 7f6c4a000640 1 -- 172.21.15.123:0/796279642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c280051c0 msgr2=0x7f6c2800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:29.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.188+0000 7f6c4a000640 1 --2- 172.21.15.123:0/796279642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c280051c0 0x7f6c2800b9c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6c44071b30 tx=0x7f6c28005830 comp rx=0 tx=0).stop 2024-04-24T05:47:29.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.189+0000 7f6c4a000640 1 -- 172.21.15.123:0/796279642 shutdown_connections 2024-04-24T05:47:29.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.189+0000 7f6c4a000640 1 --2- 172.21.15.123:0/796279642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c280051c0 0x7f6c2800b9c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.189+0000 7f6c4a000640 1 -- 172.21.15.123:0/796279642 >> 172.21.15.123:0/796279642 conn(0x7f6c440d0070 msgr2=0x7f6c440d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:29.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.189+0000 7f6c4a000640 1 -- 172.21.15.123:0/796279642 shutdown_connections 2024-04-24T05:47:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.189+0000 7f6c4a000640 1 -- 172.21.15.123:0/796279642 wait complete. 2024-04-24T05:47:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.190+0000 7f6c4a000640 1 Processor -- start 2024-04-24T05:47:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.191+0000 7f6c4a000640 1 -- start start 2024-04-24T05:47:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.191+0000 7f6c4a000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c280051c0 0x7f6c44151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.191+0000 7f6c4a000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c44159d40 0x7f6c4415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.191+0000 7f6c4a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c4415eb00 0x7f6c4415ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.191+0000 7f6c4a000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c440d0ce0 con 0x7f6c4415eb00 2024-04-24T05:47:29.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.191+0000 7f6c4a000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6c440d0b60 con 0x7f6c44159d40 2024-04-24T05:47:29.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.191+0000 7f6c4a000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6c440ae2e0 con 0x7f6c280051c0 2024-04-24T05:47:29.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.192+0000 7f6c49600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c4415eb00 0x7f6c4415ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:29.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.192+0000 7f6c49600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c4415eb00 0x7f6c4415ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44956/0 (socket says 172.21.15.123:44956) 2024-04-24T05:47:29.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.192+0000 7f6c49600640 1 -- 172.21.15.123:0/4035739592 learned_addr learned my addr 172.21.15.123:0/4035739592 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:29.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.192+0000 7f6c48c00640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c280051c0 0x7f6c44151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:29.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.192+0000 7f6c43e00640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c44159d40 0x7f6c4415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:29.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.192+0000 7f6c49600640 1 -- 172.21.15.123:0/4035739592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c280051c0 msgr2=0x7f6c44151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:29.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.192+0000 7f6c49600640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c280051c0 0x7f6c44151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.192+0000 7f6c49600640 1 -- 172.21.15.123:0/4035739592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c44159d40 msgr2=0x7f6c4415a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:29.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.192+0000 7f6c49600640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c44159d40 0x7f6c4415a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.192+0000 7f6c49600640 1 -- 172.21.15.123:0/4035739592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c441527a0 con 0x7f6c4415eb00 2024-04-24T05:47:29.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.192+0000 7f6c48c00640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c280051c0 0x7f6c44151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:29.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.193+0000 7f6c49600640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c4415eb00 0x7f6c4415ef80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c033cb0 tx=0x7f6c3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.193+0000 7f6c40c00640 1 -- 172.21.15.123:0/4035739592 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c00b610 con 0x7f6c4415eb00 2024-04-24T05:47:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.193+0000 7f6c40c00640 1 -- 172.21.15.123:0/4035739592 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c3c043070 con 0x7f6c4415eb00 2024-04-24T05:47:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.193+0000 7f6c4a000640 1 -- 172.21.15.123:0/4035739592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c44151700 con 0x7f6c4415eb00 2024-04-24T05:47:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.193+0000 7f6c40c00640 1 -- 172.21.15.123:0/4035739592 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c03d8b0 con 0x7f6c4415eb00 2024-04-24T05:47:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.193+0000 7f6c4a000640 1 -- 172.21.15.123:0/4035739592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c44151cc0 con 0x7f6c4415eb00 2024-04-24T05:47:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.194+0000 7f6c4a000640 1 -- 172.21.15.123:0/4035739592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c44075b70 con 0x7f6c4415eb00 2024-04-24T05:47:29.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.195+0000 7f6c40c00640 1 -- 172.21.15.123:0/4035739592 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c3c03da50 con 0x7f6c4415eb00 2024-04-24T05:47:29.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.196+0000 7f6c40c00640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6c2406c920 0x7f6c2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.196+0000 7f6c40c00640 1 -- 172.21.15.123:0/4035739592 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f6c3c0ac4d0 con 0x7f6c4415eb00 2024-04-24T05:47:29.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.196+0000 7f6c48c00640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6c2406c920 0x7f6c2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:29.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.197+0000 7f6c48c00640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6c2406c920 0x7f6c2406ede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c34001580 tx=0x7f6c340012d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:29.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.201+0000 7f6c40c00640 1 -- 172.21.15.123:0/4035739592 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c3c07a580 con 0x7f6c4415eb00 2024-04-24T05:47:29.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.440+0000 7f6c4a000640 1 -- 172.21.15.123:0/4035739592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c4415c140 con 0x7f6c4415eb00 2024-04-24T05:47:29.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.441+0000 7f6c40c00640 1 -- 172.21.15.123:0/4035739592 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5189 (secure 0 0 0) 0x7f6c3c046e40 con 0x7f6c4415eb00 2024-04-24T05:47:29.443 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:29.443 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2743935658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2743935658},{"type":"v1","addr":"172.21.15.155:6834","nonce":2743935658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1148561953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1148561953},{"type":"v1","addr":"172.21.15.123:6836","nonce":1148561953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2992052647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2992052647},{"type":"v1","addr":"172.21.15.155:6835","nonce":2992052647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T05:45:59.639548+0000","modified":"2024-04-24T05:46:19.813408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/3370584410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3370584410},{"type":"v1","addr":"172.21.15.123:6837","nonce":3370584410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-24T05:47:29.444 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-04-24T05:47:29.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.444+0000 7f6c4a000640 1 -- 172.21.15.123:0/4035739592 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6c2406c920 msgr2=0x7f6c2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:29.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.444+0000 7f6c4a000640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6c2406c920 0x7f6c2406ede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c34001580 tx=0x7f6c340012d0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.444+0000 7f6c4a000640 1 -- 172.21.15.123:0/4035739592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c4415eb00 msgr2=0x7f6c4415ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:29.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.444+0000 7f6c4a000640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c4415eb00 0x7f6c4415ef80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c033cb0 tx=0x7f6c3c004730 comp rx=0 tx=0).stop 2024-04-24T05:47:29.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.444+0000 7f6c4a000640 1 -- 172.21.15.123:0/4035739592 shutdown_connections 2024-04-24T05:47:29.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.444+0000 7f6c4a000640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6c2406c920 0x7f6c2406ede0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.444+0000 7f6c4a000640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c4415eb00 0x7f6c4415ef80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.444+0000 7f6c4a000640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c44159d40 0x7f6c4415a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.444+0000 7f6c4a000640 1 --2- 172.21.15.123:0/4035739592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c280051c0 0x7f6c44151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.444+0000 7f6c4a000640 1 -- 172.21.15.123:0/4035739592 >> 172.21.15.123:0/4035739592 conn(0x7f6c440d0070 msgr2=0x7f6c440d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:29.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.444+0000 7f6c4a000640 1 -- 172.21.15.123:0/4035739592 shutdown_connections 2024-04-24T05:47:29.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.444+0000 7f6c4a000640 1 -- 172.21.15.123:0/4035739592 wait complete. 2024-04-24T05:47:29.464 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:29.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.655+0000 7f6b43200640 1 Processor -- start 2024-04-24T05:47:29.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.655+0000 7f6b43200640 1 -- start start 2024-04-24T05:47:29.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.656+0000 7f6b43200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6b3c0af950 0x7f6b3c167580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.656+0000 7f6b43200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b3c15cdf0 con 0x7f6b3c15a2a0 2024-04-24T05:47:29.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.656+0000 7f6b43200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b3c15d0d0 con 0x7f6b3c0ad860 2024-04-24T05:47:29.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.656+0000 7f6b43200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6b3c15a680 con 0x7f6b3c0af950 2024-04-24T05:47:29.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.657+0000 7f6b3be00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6b3c15a2a0 0x7f6b3c0ad150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33530/0 (socket says 172.21.15.123:33530) 2024-04-24T05:47:29.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.657+0000 7f6b3be00640 1 -- 172.21.15.123:0/2009136169 learned_addr learned my addr 172.21.15.123:0/2009136169 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:29.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.657+0000 7f6b41200640 1 --2- 172.21.15.123:0/2009136169 >> v2:172.21.15.155:3300/0 conn(0x7f6b3c0af950 0x7f6b3c167580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:29.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.657+0000 7f6b3b400640 1 -- 172.21.15.123:0/2009136169 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 273391651 0 0) 0x7f6b3c15cdf0 con 0x7f6b3c15a2a0 2024-04-24T05:47:29.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.657+0000 7f6b3b400640 1 -- 172.21.15.123:0/2009136169 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b20003520 con 0x7f6b3c15a2a0 2024-04-24T05:47:29.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.657+0000 7f6b3b400640 1 -- 172.21.15.123:0/2009136169 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1490373047 0 0) 0x7f6b20003520 con 0x7f6b3c15a2a0 2024-04-24T05:47:29.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.657+0000 7f6b3b400640 1 -- 172.21.15.123:0/2009136169 >> v2:172.21.15.155:3300/0 conn(0x7f6b3c0af950 msgr2=0x7f6b3c167580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:29.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.657+0000 7f6b3b400640 1 --2- 172.21.15.123:0/2009136169 >> v2:172.21.15.155:3300/0 conn(0x7f6b3c0af950 0x7f6b3c167580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.657+0000 7f6b3b400640 1 -- 172.21.15.123:0/2009136169 >> v1:172.21.15.155:6789/0 conn(0x7f6b3c0ad860 legacy=0x7f6b3c0af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:29.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.657+0000 7f6b3b400640 1 -- 172.21.15.123:0/2009136169 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c15d3b0 con 0x7f6b3c15a2a0 2024-04-24T05:47:29.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b3b400640 1 -- 172.21.15.123:0/2009136169 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f6b2c001670 con 0x7f6b3c15a2a0 2024-04-24T05:47:29.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b3b400640 1 -- 172.21.15.123:0/2009136169 >> v1:172.21.15.123:6789/0 conn(0x7f6b3c15a2a0 legacy=0x7f6b3c0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:29.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b3b400640 1 --2- 172.21.15.123:0/2009136169 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b200039b0 0x7f6b20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b3b400640 1 --2- 172.21.15.123:0/2009136169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b20004800 0x7f6b20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b3b400640 1 --2- 172.21.15.123:0/2009136169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b200051c0 0x7f6b2000fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b3b400640 1 -- 172.21.15.123:0/2009136169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c15a680 con 0x7f6b200051c0 2024-04-24T05:47:29.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b3b400640 1 -- 172.21.15.123:0/2009136169 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6b20010570 con 0x7f6b20004800 2024-04-24T05:47:29.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b3b400640 1 -- 172.21.15.123:0/2009136169 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6b20010770 con 0x7f6b200039b0 2024-04-24T05:47:29.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b41200640 1 --2- 172.21.15.123:0/2009136169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b200051c0 0x7f6b2000fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:29.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b33e00640 1 --2- 172.21.15.123:0/2009136169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b20004800 0x7f6b20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:29.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b41200640 1 -- 172.21.15.123:0/2009136169 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b200039b0 msgr2=0x7f6b20003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:29.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b3be00640 1 --2- 172.21.15.123:0/2009136169 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b200039b0 0x7f6b20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:29.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b41200640 1 --2- 172.21.15.123:0/2009136169 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b200039b0 0x7f6b20003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b41200640 1 -- 172.21.15.123:0/2009136169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b20004800 msgr2=0x7f6b20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:29.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b41200640 1 --2- 172.21.15.123:0/2009136169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b20004800 0x7f6b20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b41200640 1 -- 172.21.15.123:0/2009136169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b20003520 con 0x7f6b200051c0 2024-04-24T05:47:29.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.658+0000 7f6b33e00640 1 --2- 172.21.15.123:0/2009136169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b20004800 0x7f6b20004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:29.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.659+0000 7f6b41200640 1 -- 172.21.15.123:0/2009136169 reap_dead start 2024-04-24T05:47:29.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.659+0000 7f6b41200640 1 --2- 172.21.15.123:0/2009136169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b200051c0 0x7f6b2000fdd0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009fc0 tx=0x7f6b20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8c46a9f4b5c806 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:29.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.660+0000 7f6b3b400640 1 -- 172.21.15.123:0/2009136169 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34035020 con 0x7f6b200051c0 2024-04-24T05:47:29.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.660+0000 7f6b3b400640 1 -- 172.21.15.123:0/2009136169 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b34003bd0 con 0x7f6b200051c0 2024-04-24T05:47:29.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.660+0000 7f6b3b400640 1 -- 172.21.15.123:0/2009136169 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34009c10 con 0x7f6b200051c0 2024-04-24T05:47:29.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.660+0000 7f6b43200640 1 -- 172.21.15.123:0/2009136169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b200051c0 msgr2=0x7f6b2000fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:29.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.660+0000 7f6b43200640 1 --2- 172.21.15.123:0/2009136169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b200051c0 0x7f6b2000fdd0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009fc0 tx=0x7f6b20005830 comp rx=0 tx=0).stop 2024-04-24T05:47:29.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.660+0000 7f6b43200640 1 -- 172.21.15.123:0/2009136169 shutdown_connections 2024-04-24T05:47:29.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.660+0000 7f6b43200640 1 --2- 172.21.15.123:0/2009136169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b200051c0 0x7f6b2000fdd0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.660+0000 7f6b43200640 1 -- 172.21.15.123:0/2009136169 >> 172.21.15.123:0/2009136169 conn(0x7f6b3c0b1e70 msgr2=0x7f6b3c0b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:29.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.660+0000 7f6b43200640 1 -- 172.21.15.123:0/2009136169 shutdown_connections 2024-04-24T05:47:29.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.660+0000 7f6b43200640 1 -- 172.21.15.123:0/2009136169 wait complete. 2024-04-24T05:47:29.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.662+0000 7f6b43200640 1 Processor -- start 2024-04-24T05:47:29.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.662+0000 7f6b43200640 1 -- start start 2024-04-24T05:47:29.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.663+0000 7f6b43200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b200051c0 0x7f6b3c154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.663+0000 7f6b43200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b3c0cf520 0x7f6b3c1f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.663+0000 7f6b43200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b3c154180 0x7f6b3c0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.663+0000 7f6b43200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c0af240 con 0x7f6b200051c0 2024-04-24T05:47:29.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.663+0000 7f6b43200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c15da00 con 0x7f6b3c154180 2024-04-24T05:47:29.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.663+0000 7f6b43200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3c0af3c0 con 0x7f6b3c0cf520 2024-04-24T05:47:29.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.663+0000 7f6b3be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b200051c0 0x7f6b3c154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:29.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.664+0000 7f6b3be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b200051c0 0x7f6b3c154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44976/0 (socket says 172.21.15.123:44976) 2024-04-24T05:47:29.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.664+0000 7f6b3be00640 1 -- 172.21.15.123:0/2242238715 learned_addr learned my addr 172.21.15.123:0/2242238715 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:29.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.664+0000 7f6b3be00640 1 -- 172.21.15.123:0/2242238715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b3c0cf520 msgr2=0x7f6b3c1f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:29.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.664+0000 7f6b3be00640 1 --2- 172.21.15.123:0/2242238715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b3c0cf520 0x7f6b3c1f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.664+0000 7f6b3be00640 1 -- 172.21.15.123:0/2242238715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b3c154180 msgr2=0x7f6b3c0b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:29.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.664+0000 7f6b3be00640 1 --2- 172.21.15.123:0/2242238715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b3c154180 0x7f6b3c0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.664+0000 7f6b3be00640 1 -- 172.21.15.123:0/2242238715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c073e90 con 0x7f6b200051c0 2024-04-24T05:47:29.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.664+0000 7f6b3be00640 1 --2- 172.21.15.123:0/2242238715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b200051c0 0x7f6b3c154ab0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00d750 tx=0x7f6b2c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:29.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.664+0000 7f6b38c00640 1 -- 172.21.15.123:0/2242238715 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c019070 con 0x7f6b200051c0 2024-04-24T05:47:29.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.664+0000 7f6b38c00640 1 -- 172.21.15.123:0/2242238715 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2c0072e0 con 0x7f6b200051c0 2024-04-24T05:47:29.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.664+0000 7f6b43200640 1 -- 172.21.15.123:0/2242238715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3c1551b0 con 0x7f6b200051c0 2024-04-24T05:47:29.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.665+0000 7f6b38c00640 1 -- 172.21.15.123:0/2242238715 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0047e0 con 0x7f6b200051c0 2024-04-24T05:47:29.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.665+0000 7f6b43200640 1 -- 172.21.15.123:0/2242238715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3c155770 con 0x7f6b200051c0 2024-04-24T05:47:29.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.666+0000 7f6b43200640 1 -- 172.21.15.123:0/2242238715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b3c0b29a0 con 0x7f6b200051c0 2024-04-24T05:47:29.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.667+0000 7f6b38c00640 1 -- 172.21.15.123:0/2242238715 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b2c01c070 con 0x7f6b200051c0 2024-04-24T05:47:29.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.667+0000 7f6b38c00640 1 --2- 172.21.15.123:0/2242238715 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6b1806c8f0 0x7f6b1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:29.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.668+0000 7f6b38c00640 1 -- 172.21.15.123:0/2242238715 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f6b2c08b980 con 0x7f6b200051c0 2024-04-24T05:47:29.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.668+0000 7f6b33e00640 1 --2- 172.21.15.123:0/2242238715 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6b1806c8f0 0x7f6b1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:29.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.668+0000 7f6b33e00640 1 --2- 172.21.15.123:0/2242238715 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6b1806c8f0 0x7f6b1806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6b24000f60 tx=0x7f6b24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:29.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.672+0000 7f6b38c00640 1 -- 172.21.15.123:0/2242238715 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b3c0b29a0 con 0x7f6b200051c0 2024-04-24T05:47:29.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.862+0000 7f6b43200640 1 -- 172.21.15.123:0/2242238715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b3c068a20 con 0x7f6b200051c0 2024-04-24T05:47:29.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.864+0000 7f6b38c00640 1 -- 172.21.15.123:0/2242238715 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18956 (secure 0 0 0) 0x7f6b2c089e20 con 0x7f6b200051c0 2024-04-24T05:47:29.866 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:29.866 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":27,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:46:41.596509+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:45:57.615179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T05:45:58.627580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T05:46:02.230550+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:29.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.867+0000 7f6b43200640 1 -- 172.21.15.123:0/2242238715 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6b1806c8f0 msgr2=0x7f6b1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:29.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.867+0000 7f6b43200640 1 --2- 172.21.15.123:0/2242238715 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6b1806c8f0 0x7f6b1806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6b24000f60 tx=0x7f6b24009040 comp rx=0 tx=0).stop 2024-04-24T05:47:29.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.867+0000 7f6b43200640 1 -- 172.21.15.123:0/2242238715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b200051c0 msgr2=0x7f6b3c154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:29.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.867+0000 7f6b43200640 1 --2- 172.21.15.123:0/2242238715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b200051c0 0x7f6b3c154ab0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00d750 tx=0x7f6b2c00dc10 comp rx=0 tx=0).stop 2024-04-24T05:47:29.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.867+0000 7f6b43200640 1 -- 172.21.15.123:0/2242238715 shutdown_connections 2024-04-24T05:47:29.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.867+0000 7f6b43200640 1 --2- 172.21.15.123:0/2242238715 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6b1806c8f0 0x7f6b1806edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.867+0000 7f6b43200640 1 --2- 172.21.15.123:0/2242238715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b3c154180 0x7f6b3c0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.867+0000 7f6b43200640 1 --2- 172.21.15.123:0/2242238715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b3c0cf520 0x7f6b3c1f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.867+0000 7f6b43200640 1 --2- 172.21.15.123:0/2242238715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b200051c0 0x7f6b3c154ab0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:29.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.867+0000 7f6b43200640 1 -- 172.21.15.123:0/2242238715 >> 172.21.15.123:0/2242238715 conn(0x7f6b3c0b1e70 msgr2=0x7f6b3c06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:29.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.867+0000 7f6b43200640 1 -- 172.21.15.123:0/2242238715 shutdown_connections 2024-04-24T05:47:29.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:29.867+0000 7f6b43200640 1 -- 172.21.15.123:0/2242238715 wait complete. 2024-04-24T05:47:29.887 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T05:47:30.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.076+0000 7f866d200640 1 Processor -- start 2024-04-24T05:47:30.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.076+0000 7f866d200640 1 -- start start 2024-04-24T05:47:30.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.077+0000 7f866d200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f866815e950 0x7f86680ca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:30.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.077+0000 7f866d200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f86680cbfa0 0x7f866806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:30.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.077+0000 7f866d200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f866815ef30 con 0x7f866806cf50 2024-04-24T05:47:30.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.077+0000 7f866d200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f866806d870 con 0x7f86680cbfa0 2024-04-24T05:47:30.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.077+0000 7f866d200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f866806d3a0 con 0x7f866815e950 2024-04-24T05:47:30.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.078+0000 7f8666a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f86680cbfa0 0x7f866806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:30.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.078+0000 7f8666a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f86680cbfa0 0x7f866806ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44978/0 (socket says 172.21.15.123:44978) 2024-04-24T05:47:30.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.078+0000 7f8666a00640 1 -- 172.21.15.123:0/2631036478 learned_addr learned my addr 172.21.15.123:0/2631036478 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:30.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.078+0000 7f8666a00640 1 -- 172.21.15.123:0/2631036478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f866815e950 msgr2=0x7f86680ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:30.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.078+0000 7f8666a00640 1 --2- 172.21.15.123:0/2631036478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f866815e950 0x7f86680ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.078+0000 7f8666a00640 1 -- 172.21.15.123:0/2631036478 >> v1:172.21.15.155:6789/0 conn(0x7f866806cf50 legacy=0x7f86680ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:30.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.078+0000 7f8666a00640 1 -- 172.21.15.123:0/2631036478 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866815f1d0 con 0x7f86680cbfa0 2024-04-24T05:47:30.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.079+0000 7f8666a00640 1 --2- 172.21.15.123:0/2631036478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86680cbfa0 0x7f866806ca10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8654009870 tx=0x7f865402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ee971c8e4f01ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:30.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.079+0000 7f8666000640 1 -- 172.21.15.123:0/2631036478 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654037070 con 0x7f86680cbfa0 2024-04-24T05:47:30.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.079+0000 7f8666000640 1 -- 172.21.15.123:0/2631036478 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8654032440 con 0x7f86680cbfa0 2024-04-24T05:47:30.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.079+0000 7f8666000640 1 -- 172.21.15.123:0/2631036478 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654032810 con 0x7f86680cbfa0 2024-04-24T05:47:30.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.080+0000 7f866d200640 1 -- 172.21.15.123:0/2631036478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86680cbfa0 msgr2=0x7f866806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:30.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.080+0000 7f866d200640 1 --2- 172.21.15.123:0/2631036478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86680cbfa0 0x7f866806ca10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8654009870 tx=0x7f865402a180 comp rx=0 tx=0).stop 2024-04-24T05:47:30.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.080+0000 7f866d200640 1 -- 172.21.15.123:0/2631036478 shutdown_connections 2024-04-24T05:47:30.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.080+0000 7f866d200640 1 --2- 172.21.15.123:0/2631036478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86680cbfa0 0x7f866806ca10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.080+0000 7f866d200640 1 --2- 172.21.15.123:0/2631036478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f866815e950 0x7f86680ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.080+0000 7f866d200640 1 -- 172.21.15.123:0/2631036478 >> 172.21.15.123:0/2631036478 conn(0x7f86680b1b10 msgr2=0x7f86680b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:30.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.080+0000 7f866d200640 1 -- 172.21.15.123:0/2631036478 shutdown_connections 2024-04-24T05:47:30.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.080+0000 7f866d200640 1 -- 172.21.15.123:0/2631036478 wait complete. 2024-04-24T05:47:30.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.082+0000 7f866d200640 1 Processor -- start 2024-04-24T05:47:30.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.082+0000 7f866d200640 1 -- start start 2024-04-24T05:47:30.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.083+0000 7f866d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f866806cf50 0x7f86680b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:30.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.083+0000 7f866d200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86680cbfa0 0x7f86680af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:30.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.083+0000 7f866d200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f866815e950 0x7f86680b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:30.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.083+0000 7f866d200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f866815f790 con 0x7f866806cf50 2024-04-24T05:47:30.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.083+0000 7f866d200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f866815fb00 con 0x7f86680cbfa0 2024-04-24T05:47:30.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.083+0000 7f866d200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f86680cbc40 con 0x7f866815e950 2024-04-24T05:47:30.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.083+0000 7f8667400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f866806cf50 0x7f86680b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:30.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.083+0000 7f8667400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f866806cf50 0x7f86680b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44990/0 (socket says 172.21.15.123:44990) 2024-04-24T05:47:30.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.083+0000 7f8667400640 1 -- 172.21.15.123:0/2383754591 learned_addr learned my addr 172.21.15.123:0/2383754591 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:30.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.083+0000 7f8667400640 1 -- 172.21.15.123:0/2383754591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f866815e950 msgr2=0x7f86680b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:30.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.083+0000 7f8667400640 1 --2- 172.21.15.123:0/2383754591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f866815e950 0x7f86680b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.084+0000 7f8667400640 1 -- 172.21.15.123:0/2383754591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86680cbfa0 msgr2=0x7f86680af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:30.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.084+0000 7f8667400640 1 --2- 172.21.15.123:0/2383754591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86680cbfa0 0x7f86680af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.084+0000 7f8667400640 1 -- 172.21.15.123:0/2383754591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86680b4f70 con 0x7f866806cf50 2024-04-24T05:47:30.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.084+0000 7f8667400640 1 --2- 172.21.15.123:0/2383754591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f866806cf50 0x7f86680b02b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f865800b4d0 tx=0x7f865800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:30.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.084+0000 7f865f400640 1 -- 172.21.15.123:0/2383754591 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658014070 con 0x7f866806cf50 2024-04-24T05:47:30.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.084+0000 7f865f400640 1 -- 172.21.15.123:0/2383754591 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8658010870 con 0x7f866806cf50 2024-04-24T05:47:30.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.084+0000 7f865f400640 1 -- 172.21.15.123:0/2383754591 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658010c60 con 0x7f866806cf50 2024-04-24T05:47:30.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.084+0000 7f866d200640 1 -- 172.21.15.123:0/2383754591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86680b09c0 con 0x7f866806cf50 2024-04-24T05:47:30.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.084+0000 7f866d200640 1 -- 172.21.15.123:0/2383754591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86680b0f80 con 0x7f866806cf50 2024-04-24T05:47:30.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.085+0000 7f866d200640 1 -- 172.21.15.123:0/2383754591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8668075b70 con 0x7f866806cf50 2024-04-24T05:47:30.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.086+0000 7f865f400640 1 -- 172.21.15.123:0/2383754591 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8658017070 con 0x7f866806cf50 2024-04-24T05:47:30.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.087+0000 7f865f400640 1 --2- 172.21.15.123:0/2383754591 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f864806c8f0 0x7f864806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:30.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.087+0000 7f865f400640 1 -- 172.21.15.123:0/2383754591 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f865808be70 con 0x7f866806cf50 2024-04-24T05:47:30.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.087+0000 7f8666a00640 1 --2- 172.21.15.123:0/2383754591 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f864806c8f0 0x7f864806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:30.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.088+0000 7f8666a00640 1 --2- 172.21.15.123:0/2383754591 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f864806c8f0 0x7f864806edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8654034000 tx=0x7f865402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:30.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.092+0000 7f865f400640 1 -- 172.21.15.123:0/2383754591 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8658059d60 con 0x7f866806cf50 2024-04-24T05:47:30.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.332+0000 7f866d200640 1 -- 172.21.15.123:0/2383754591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f866815d390 con 0x7f866806cf50 2024-04-24T05:47:30.334 INFO:tasks.ceph.mon.a.smithi123.stderr:2024-04-24T05:47:30.333+0000 7f3209a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T05:47:30.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.336+0000 7f865f400640 1 -- 172.21.15.123:0/2383754591 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f8658059660 con 0x7f866806cf50 2024-04-24T05:47:30.337 INFO:teuthology.orchestra.run.smithi123.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T05:47:30.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.339+0000 7f866d200640 1 -- 172.21.15.123:0/2383754591 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f864806c8f0 msgr2=0x7f864806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:30.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.339+0000 7f866d200640 1 --2- 172.21.15.123:0/2383754591 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f864806c8f0 0x7f864806edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8654034000 tx=0x7f865402c000 comp rx=0 tx=0).stop 2024-04-24T05:47:30.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.339+0000 7f866d200640 1 -- 172.21.15.123:0/2383754591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f866806cf50 msgr2=0x7f86680b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:30.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.339+0000 7f866d200640 1 --2- 172.21.15.123:0/2383754591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f866806cf50 0x7f86680b02b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f865800b4d0 tx=0x7f865800b990 comp rx=0 tx=0).stop 2024-04-24T05:47:30.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.339+0000 7f866d200640 1 -- 172.21.15.123:0/2383754591 shutdown_connections 2024-04-24T05:47:30.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.339+0000 7f866d200640 1 --2- 172.21.15.123:0/2383754591 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f864806c8f0 0x7f864806edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.339+0000 7f866d200640 1 --2- 172.21.15.123:0/2383754591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f866815e950 0x7f86680b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.339+0000 7f866d200640 1 --2- 172.21.15.123:0/2383754591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86680cbfa0 0x7f86680af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.339+0000 7f866d200640 1 --2- 172.21.15.123:0/2383754591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f866806cf50 0x7f86680b02b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.339+0000 7f866d200640 1 -- 172.21.15.123:0/2383754591 >> 172.21.15.123:0/2383754591 conn(0x7f86680b1b10 msgr2=0x7f8668071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:30.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.339+0000 7f866d200640 1 -- 172.21.15.123:0/2383754591 shutdown_connections 2024-04-24T05:47:30.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.340+0000 7f866d200640 1 -- 172.21.15.123:0/2383754591 wait complete. 2024-04-24T05:47:30.353 INFO:tasks.ceph.mds.a.smithi123.stderr: -5762> 2024-04-24T05:46:05.717+0000 7f45ce000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T05:47:30.360 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T05:47:30.414 INFO:tasks.ceph.mds.a.smithi123.stdout:starting mds.a at 2024-04-24T05:47:30.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.535+0000 7fe5fb600640 1 Processor -- start 2024-04-24T05:47:30.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.535+0000 7fe5fb600640 1 -- start start 2024-04-24T05:47:30.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.537+0000 7fe5fb600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe5f40cb8a0 0x7fe5f40ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:30.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.537+0000 7fe5fb600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5f4166410 0x7fe5f4168800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:30.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.537+0000 7fe5fb600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5f406cdb0 con 0x7fe5f415ea40 2024-04-24T05:47:30.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.537+0000 7fe5fb600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe5f40ce710 con 0x7fe5f40cb8a0 2024-04-24T05:47:30.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.537+0000 7fe5fb600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f406c7f0 con 0x7fe5f4166410 2024-04-24T05:47:30.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.537+0000 7fe5f3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe5f40cb8a0 0x7fe5f40ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:30.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.537+0000 7fe5f3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe5f40cb8a0 0x7fe5f40ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45032/0 (socket says 172.21.15.123:45032) 2024-04-24T05:47:30.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.537+0000 7fe5f3e00640 1 -- 172.21.15.123:0/1663141132 learned_addr learned my addr 172.21.15.123:0/1663141132 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:30.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.537+0000 7fe5f3e00640 1 -- 172.21.15.123:0/1663141132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5f4166410 msgr2=0x7fe5f4168800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:30.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.537+0000 7fe5f3e00640 1 --2- 172.21.15.123:0/1663141132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5f4166410 0x7fe5f4168800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.537+0000 7fe5f3e00640 1 -- 172.21.15.123:0/1663141132 >> v1:172.21.15.155:6789/0 conn(0x7fe5f415ea40 legacy=0x7fe5f40cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:30.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.538+0000 7fe5f3e00640 1 -- 172.21.15.123:0/1663141132 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f406cad0 con 0x7fe5f40cb8a0 2024-04-24T05:47:30.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.538+0000 7fe5f3e00640 1 --2- 172.21.15.123:0/1663141132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5f40cb8a0 0x7fe5f40ce0e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40720b0 tx=0x7fe5e002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0c7f970ae9f5c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:30.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.539+0000 7fe5f3400640 1 -- 172.21.15.123:0/1663141132 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0037070 con 0x7fe5f40cb8a0 2024-04-24T05:47:30.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.539+0000 7fe5f3400640 1 -- 172.21.15.123:0/1663141132 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e0032440 con 0x7fe5f40cb8a0 2024-04-24T05:47:30.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.539+0000 7fe5f3400640 1 -- 172.21.15.123:0/1663141132 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0032810 con 0x7fe5f40cb8a0 2024-04-24T05:47:30.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.539+0000 7fe5fb600640 1 -- 172.21.15.123:0/1663141132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5f40cb8a0 msgr2=0x7fe5f40ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:30.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.539+0000 7fe5fb600640 1 --2- 172.21.15.123:0/1663141132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5f40cb8a0 0x7fe5f40ce0e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40720b0 tx=0x7fe5e002a1e0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.539+0000 7fe5fb600640 1 -- 172.21.15.123:0/1663141132 shutdown_connections 2024-04-24T05:47:30.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.540+0000 7fe5fb600640 1 --2- 172.21.15.123:0/1663141132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5f4166410 0x7fe5f4168800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.540+0000 7fe5fb600640 1 --2- 172.21.15.123:0/1663141132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5f40cb8a0 0x7fe5f40ce0e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.540+0000 7fe5fb600640 1 -- 172.21.15.123:0/1663141132 >> 172.21.15.123:0/1663141132 conn(0x7fe5f40cd5c0 msgr2=0x7fe5f40cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:30.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.540+0000 7fe5fb600640 1 -- 172.21.15.123:0/1663141132 shutdown_connections 2024-04-24T05:47:30.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.540+0000 7fe5fb600640 1 -- 172.21.15.123:0/1663141132 wait complete. 2024-04-24T05:47:30.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.541+0000 7fe5fb600640 1 Processor -- start 2024-04-24T05:47:30.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.541+0000 7fe5fb600640 1 -- start start 2024-04-24T05:47:30.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.542+0000 7fe5fb600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f40cb8a0 0x7fe5f40ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:30.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.542+0000 7fe5fb600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5f415ea40 0x7fe5f415eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:30.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.542+0000 7fe5fb600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5f4166410 0x7fe5f40aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:30.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.542+0000 7fe5fb600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f406fdc0 con 0x7fe5f4166410 2024-04-24T05:47:30.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.542+0000 7fe5fb600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f406e890 con 0x7fe5f40cb8a0 2024-04-24T05:47:30.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.542+0000 7fe5fb600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f406da30 con 0x7fe5f415ea40 2024-04-24T05:47:30.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.543+0000 7fe5f9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5f4166410 0x7fe5f40aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:30.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.543+0000 7fe5f9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5f4166410 0x7fe5f40aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45042/0 (socket says 172.21.15.123:45042) 2024-04-24T05:47:30.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.543+0000 7fe5f9600640 1 -- 172.21.15.123:0/2839398113 learned_addr learned my addr 172.21.15.123:0/2839398113 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:30.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.543+0000 7fe5f8c00640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f40cb8a0 0x7fe5f40ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:30.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.543+0000 7fe5f3e00640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5f415ea40 0x7fe5f415eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:30.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.543+0000 7fe5f9600640 1 -- 172.21.15.123:0/2839398113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5f415ea40 msgr2=0x7fe5f415eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:30.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.543+0000 7fe5f9600640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5f415ea40 0x7fe5f415eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.543+0000 7fe5f9600640 1 -- 172.21.15.123:0/2839398113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f40cb8a0 msgr2=0x7fe5f40ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:30.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.543+0000 7fe5f9600640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f40cb8a0 0x7fe5f40ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:30.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.543+0000 7fe5f9600640 1 -- 172.21.15.123:0/2839398113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f41532d0 con 0x7fe5f4166410 2024-04-24T05:47:30.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.543+0000 7fe5f8c00640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f40cb8a0 0x7fe5f40ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:30.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.543+0000 7fe5f3e00640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5f415ea40 0x7fe5f415eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:30.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.543+0000 7fe5f9600640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5f4166410 0x7fe5f40aec50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00b9e0 tx=0x7fe5ec00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:30.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.544+0000 7fe5f0c00640 1 -- 172.21.15.123:0/2839398113 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec019070 con 0x7fe5f4166410 2024-04-24T05:47:30.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.544+0000 7fe5f0c00640 1 -- 172.21.15.123:0/2839398113 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5ec00cb40 con 0x7fe5f4166410 2024-04-24T05:47:30.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.544+0000 7fe5f0c00640 1 -- 172.21.15.123:0/2839398113 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec014440 con 0x7fe5f4166410 2024-04-24T05:47:30.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.544+0000 7fe5fb600640 1 -- 172.21.15.123:0/2839398113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f415ad00 con 0x7fe5f4166410 2024-04-24T05:47:30.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.544+0000 7fe5fb600640 1 -- 172.21.15.123:0/2839398113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f415b2c0 con 0x7fe5f4166410 2024-04-24T05:47:30.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.545+0000 7fe5fb600640 1 -- 172.21.15.123:0/2839398113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f4069130 con 0x7fe5f4166410 2024-04-24T05:47:30.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.546+0000 7fe5f0c00640 1 -- 172.21.15.123:0/2839398113 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5ec01c070 con 0x7fe5f4166410 2024-04-24T05:47:30.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.546+0000 7fe5f0c00640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe5d406c8f0 0x7fe5d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:30.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.547+0000 7fe5f0c00640 1 -- 172.21.15.123:0/2839398113 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6829+0+0 (secure 0 0 0) 0x7fe5ec08bd80 con 0x7fe5f4166410 2024-04-24T05:47:30.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.547+0000 7fe5f8c00640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe5d406c8f0 0x7fe5d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:30.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.547+0000 7fe5f8c00640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe5d406c8f0 0x7fe5d406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4003eb0 tx=0x7fe5e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:30.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.552+0000 7fe5f0c00640 1 -- 172.21.15.123:0/2839398113 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe5ec059cb0 con 0x7fe5f4166410 2024-04-24T05:47:30.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:30.773+0000 7fe5fb600640 1 -- 172.21.15.123:0/2839398113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe5f415cd40 con 0x7fe5f4166410 2024-04-24T05:47:31.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.348+0000 7fe5f0c00640 1 -- 172.21.15.123:0/2839398113 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fe5ec0595b0 con 0x7fe5f4166410 2024-04-24T05:47:31.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.352+0000 7fe5fb600640 1 -- 172.21.15.123:0/2839398113 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe5d406c8f0 msgr2=0x7fe5d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:31.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.352+0000 7fe5fb600640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe5d406c8f0 0x7fe5d406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4003eb0 tx=0x7fe5e4009040 comp rx=0 tx=0).stop 2024-04-24T05:47:31.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.352+0000 7fe5fb600640 1 -- 172.21.15.123:0/2839398113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5f4166410 msgr2=0x7fe5f40aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:31.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.352+0000 7fe5fb600640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5f4166410 0x7fe5f40aec50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00b9e0 tx=0x7fe5ec00bea0 comp rx=0 tx=0).stop 2024-04-24T05:47:31.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.352+0000 7fe5fb600640 1 -- 172.21.15.123:0/2839398113 shutdown_connections 2024-04-24T05:47:31.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.352+0000 7fe5fb600640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe5d406c8f0 0x7fe5d406edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:31.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.352+0000 7fe5fb600640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5f4166410 0x7fe5f40aec50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:31.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.352+0000 7fe5fb600640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5f415ea40 0x7fe5f415eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:31.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.352+0000 7fe5fb600640 1 --2- 172.21.15.123:0/2839398113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f40cb8a0 0x7fe5f40ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:31.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.353+0000 7fe5fb600640 1 -- 172.21.15.123:0/2839398113 >> 172.21.15.123:0/2839398113 conn(0x7fe5f40cd5c0 msgr2=0x7fe5f415d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:31.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.353+0000 7fe5fb600640 1 -- 172.21.15.123:0/2839398113 shutdown_connections 2024-04-24T05:47:31.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.353+0000 7fe5fb600640 1 -- 172.21.15.123:0/2839398113 wait complete. 2024-04-24T05:47:31.379 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T05:47:31.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.558+0000 7f1a58a00640 1 Processor -- start 2024-04-24T05:47:31.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.558+0000 7f1a58a00640 1 -- start start 2024-04-24T05:47:31.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.559+0000 7f1a58a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a5415e760 0x7f1a540b6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:31.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.559+0000 7f1a58a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1a540b5ed0 0x7f1a540d4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:31.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.560+0000 7f1a58a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a540d4c30 con 0x7f1a540d47c0 2024-04-24T05:47:31.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.560+0000 7f1a58a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1a540d2030 con 0x7f1a540b5ed0 2024-04-24T05:47:31.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.560+0000 7f1a58a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1a540d2310 con 0x7f1a5415e760 2024-04-24T05:47:31.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.560+0000 7f1a52800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a5415e760 0x7f1a540b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:31.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.560+0000 7f1a52800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a5415e760 0x7f1a540b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:49390/0 (socket says 172.21.15.123:49390) 2024-04-24T05:47:31.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.560+0000 7f1a52800640 1 -- 172.21.15.123:0/3541945486 learned_addr learned my addr 172.21.15.123:0/3541945486 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:31.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.561+0000 7f1a51e00640 1 --2- 172.21.15.123:0/3541945486 >> v2:172.21.15.155:3300/0 conn(0x7f1a540b5ed0 0x7f1a540d4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:31.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.561+0000 7f1a51400640 1 -- 172.21.15.123:0/3541945486 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68802095 0 0) 0x7f1a540d4c30 con 0x7f1a540d47c0 2024-04-24T05:47:31.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.561+0000 7f1a51400640 1 -- 172.21.15.123:0/3541945486 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a38003520 con 0x7f1a540d47c0 2024-04-24T05:47:31.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.561+0000 7f1a52800640 1 -- 172.21.15.123:0/3541945486 >> v2:172.21.15.155:3300/0 conn(0x7f1a540b5ed0 msgr2=0x7f1a540d4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:31.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.561+0000 7f1a52800640 1 --2- 172.21.15.123:0/3541945486 >> v2:172.21.15.155:3300/0 conn(0x7f1a540b5ed0 0x7f1a540d4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:31.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.561+0000 7f1a52800640 1 -- 172.21.15.123:0/3541945486 >> v1:172.21.15.155:6789/0 conn(0x7f1a540d47c0 legacy=0x7f1a54162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:31.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.561+0000 7f1a52800640 1 -- 172.21.15.123:0/3541945486 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a540d25f0 con 0x7f1a5415e760 2024-04-24T05:47:31.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.562+0000 7f1a51e00640 1 --2- 172.21.15.123:0/3541945486 >> v2:172.21.15.155:3300/0 conn(0x7f1a540b5ed0 0x7f1a540d4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:47:31.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.563+0000 7f1a52800640 1 --2- 172.21.15.123:0/3541945486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a5415e760 0x7f1a540b6660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009870 tx=0x7f1a4402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bc3b1fcaaa85574 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:31.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.563+0000 7f1a51400640 1 -- 172.21.15.123:0/3541945486 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44037070 con 0x7f1a5415e760 2024-04-24T05:47:31.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.563+0000 7f1a51400640 1 -- 172.21.15.123:0/3541945486 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a44032440 con 0x7f1a5415e760 2024-04-24T05:47:31.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.563+0000 7f1a51400640 1 -- 172.21.15.123:0/3541945486 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44032810 con 0x7f1a5415e760 2024-04-24T05:47:31.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.564+0000 7f1a58a00640 1 -- 172.21.15.123:0/3541945486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a5415e760 msgr2=0x7f1a540b6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:31.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.564+0000 7f1a58a00640 1 --2- 172.21.15.123:0/3541945486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a5415e760 0x7f1a540b6660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009870 tx=0x7f1a4402a1c0 comp rx=0 tx=0).stop 2024-04-24T05:47:31.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.564+0000 7f1a58a00640 1 -- 172.21.15.123:0/3541945486 shutdown_connections 2024-04-24T05:47:31.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.564+0000 7f1a58a00640 1 --2- 172.21.15.123:0/3541945486 >> v2:172.21.15.155:3300/0 conn(0x7f1a540b5ed0 0x7f1a540d4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:31.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.564+0000 7f1a58a00640 1 --2- 172.21.15.123:0/3541945486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a5415e760 0x7f1a540b6660 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:31.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.564+0000 7f1a58a00640 1 -- 172.21.15.123:0/3541945486 >> 172.21.15.123:0/3541945486 conn(0x7f1a5409d7c0 msgr2=0x7f1a540b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:31.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.564+0000 7f1a58a00640 1 -- 172.21.15.123:0/3541945486 shutdown_connections 2024-04-24T05:47:31.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.564+0000 7f1a58a00640 1 -- 172.21.15.123:0/3541945486 wait complete. 2024-04-24T05:47:31.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.566+0000 7f1a58a00640 1 Processor -- start 2024-04-24T05:47:31.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.566+0000 7f1a58a00640 1 -- start start 2024-04-24T05:47:31.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.567+0000 7f1a58a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a540b5ed0 0x7f1a5414d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:31.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.567+0000 7f1a58a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a540d47c0 0x7f1a5406ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:31.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.567+0000 7f1a58a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a5415e760 0x7f1a540b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:31.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.567+0000 7f1a58a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a540b6cd0 con 0x7f1a5415e760 2024-04-24T05:47:31.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.567+0000 7f1a58a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1a540d0930 con 0x7f1a540b5ed0 2024-04-24T05:47:31.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.567+0000 7f1a58a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1a5415eb40 con 0x7f1a540d47c0 2024-04-24T05:47:31.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.567+0000 7f1a53200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a5415e760 0x7f1a540b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:31.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.567+0000 7f1a53200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a5415e760 0x7f1a540b3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45052/0 (socket says 172.21.15.123:45052) 2024-04-24T05:47:31.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.567+0000 7f1a53200640 1 -- 172.21.15.123:0/3939805739 learned_addr learned my addr 172.21.15.123:0/3939805739 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:31.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.568+0000 7f1a51e00640 1 --2- 172.21.15.123:0/3939805739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a540d47c0 0x7f1a5406ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:31.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.567+0000 7f1a52800640 1 --2- 172.21.15.123:0/3939805739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a540b5ed0 0x7f1a5414d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:31.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.568+0000 7f1a53200640 1 -- 172.21.15.123:0/3939805739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a540d47c0 msgr2=0x7f1a5406ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:31.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.568+0000 7f1a53200640 1 --2- 172.21.15.123:0/3939805739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a540d47c0 0x7f1a5406ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:31.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.568+0000 7f1a53200640 1 -- 172.21.15.123:0/3939805739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a540b5ed0 msgr2=0x7f1a5414d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:31.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.568+0000 7f1a53200640 1 --2- 172.21.15.123:0/3939805739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a540b5ed0 0x7f1a5414d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:31.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.568+0000 7f1a53200640 1 -- 172.21.15.123:0/3939805739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5414ddc0 con 0x7f1a5415e760 2024-04-24T05:47:31.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.568+0000 7f1a53200640 1 --2- 172.21.15.123:0/3939805739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a5415e760 0x7f1a540b3eb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c002230 tx=0x7f1a4c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:31.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.568+0000 7f1a4aa00640 1 -- 172.21.15.123:0/3939805739 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c003b70 con 0x7f1a5415e760 2024-04-24T05:47:31.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.568+0000 7f1a4aa00640 1 -- 172.21.15.123:0/3939805739 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a4c003d10 con 0x7f1a5415e760 2024-04-24T05:47:31.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.568+0000 7f1a4aa00640 1 -- 172.21.15.123:0/3939805739 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0052d0 con 0x7f1a5415e760 2024-04-24T05:47:31.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.568+0000 7f1a58a00640 1 -- 172.21.15.123:0/3939805739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5406c370 con 0x7f1a5415e760 2024-04-24T05:47:31.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.568+0000 7f1a58a00640 1 -- 172.21.15.123:0/3939805739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a5406c930 con 0x7f1a5415e760 2024-04-24T05:47:31.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.569+0000 7f1a58a00640 1 -- 172.21.15.123:0/3939805739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a54068a20 con 0x7f1a5415e760 2024-04-24T05:47:31.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.570+0000 7f1a4aa00640 1 -- 172.21.15.123:0/3939805739 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a4c005680 con 0x7f1a5415e760 2024-04-24T05:47:31.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.571+0000 7f1a4aa00640 1 --2- 172.21.15.123:0/3939805739 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1a3406c8f0 0x7f1a3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:31.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.571+0000 7f1a4aa00640 1 -- 172.21.15.123:0/3939805739 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6829+0+0 (secure 0 0 0) 0x7f1a4c08c180 con 0x7f1a5415e760 2024-04-24T05:47:31.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.572+0000 7f1a52800640 1 --2- 172.21.15.123:0/3939805739 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1a3406c8f0 0x7f1a3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:31.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.572+0000 7f1a52800640 1 --2- 172.21.15.123:0/3939805739 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1a3406c8f0 0x7f1a3406edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1a44034000 tx=0x7f1a4402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:31.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.576+0000 7f1a4aa00640 1 -- 172.21.15.123:0/3939805739 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a5414ddc0 con 0x7f1a5415e760 2024-04-24T05:47:31.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:31.762+0000 7f1a58a00640 1 -- 172.21.15.123:0/3939805739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1a540d5050 con 0x7f1a5415e760 2024-04-24T05:47:32.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.352+0000 7f1a4aa00640 1 -- 172.21.15.123:0/3939805739 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7f1a4c05a0b0 con 0x7f1a5415e760 2024-04-24T05:47:32.354 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' removed 2024-04-24T05:47:32.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.357+0000 7f1a58a00640 1 -- 172.21.15.123:0/3939805739 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1a3406c8f0 msgr2=0x7f1a3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:32.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.357+0000 7f1a58a00640 1 --2- 172.21.15.123:0/3939805739 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1a3406c8f0 0x7f1a3406edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1a44034000 tx=0x7f1a4402e000 comp rx=0 tx=0).stop 2024-04-24T05:47:32.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.357+0000 7f1a58a00640 1 -- 172.21.15.123:0/3939805739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a5415e760 msgr2=0x7f1a540b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:32.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.357+0000 7f1a58a00640 1 --2- 172.21.15.123:0/3939805739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a5415e760 0x7f1a540b3eb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c002230 tx=0x7f1a4c00cb90 comp rx=0 tx=0).stop 2024-04-24T05:47:32.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.358+0000 7f1a58a00640 1 -- 172.21.15.123:0/3939805739 shutdown_connections 2024-04-24T05:47:32.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.358+0000 7f1a58a00640 1 --2- 172.21.15.123:0/3939805739 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1a3406c8f0 0x7f1a3406edb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:32.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.358+0000 7f1a58a00640 1 --2- 172.21.15.123:0/3939805739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a5415e760 0x7f1a540b3eb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:32.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.358+0000 7f1a58a00640 1 --2- 172.21.15.123:0/3939805739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a540d47c0 0x7f1a5406ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:32.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.358+0000 7f1a58a00640 1 --2- 172.21.15.123:0/3939805739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a540b5ed0 0x7f1a5414d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:32.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.358+0000 7f1a58a00640 1 -- 172.21.15.123:0/3939805739 >> 172.21.15.123:0/3939805739 conn(0x7f1a5409d7c0 msgr2=0x7f1a540b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:32.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.358+0000 7f1a58a00640 1 -- 172.21.15.123:0/3939805739 shutdown_connections 2024-04-24T05:47:32.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.359+0000 7f1a58a00640 1 -- 172.21.15.123:0/3939805739 wait complete. 2024-04-24T05:47:32.384 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T05:47:32.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.581+0000 7f7a20000640 1 Processor -- start 2024-04-24T05:47:32.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.581+0000 7f7a20000640 1 -- start start 2024-04-24T05:47:32.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.582+0000 7f7a20000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7a180ae990 0x7f7a180d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:32.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.582+0000 7f7a20000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1806c410 0x7f7a180ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:32.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.582+0000 7f7a20000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a180d0740 con 0x7f7a180ce610 2024-04-24T05:47:32.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.582+0000 7f7a20000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7a180cea70 con 0x7f7a180ae990 2024-04-24T05:47:32.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.582+0000 7f7a20000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a1806dcb0 con 0x7f7a1806c410 2024-04-24T05:47:32.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.582+0000 7f7a1d600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7a180ae990 0x7f7a180d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:32.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.582+0000 7f7a1d600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7a180ae990 0x7f7a180d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45068/0 (socket says 172.21.15.123:45068) 2024-04-24T05:47:32.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.582+0000 7f7a1d600640 1 -- 172.21.15.123:0/1979947034 learned_addr learned my addr 172.21.15.123:0/1979947034 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:32.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.583+0000 7f7a1cc00640 1 --2- 172.21.15.123:0/1979947034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1806c410 0x7f7a180ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:32.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.583+0000 7f7a1d600640 1 -- 172.21.15.123:0/1979947034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1806c410 msgr2=0x7f7a180ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:32.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.583+0000 7f7a1d600640 1 --2- 172.21.15.123:0/1979947034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1806c410 0x7f7a180ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:32.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.583+0000 7f7a1d600640 1 -- 172.21.15.123:0/1979947034 >> v1:172.21.15.155:6789/0 conn(0x7f7a180ce610 legacy=0x7f7a1815f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:32.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.583+0000 7f7a1d600640 1 -- 172.21.15.123:0/1979947034 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a180d0a20 con 0x7f7a180ae990 2024-04-24T05:47:32.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.583+0000 7f7a1cc00640 1 --2- 172.21.15.123:0/1979947034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1806c410 0x7f7a180ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:32.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.584+0000 7f7a1d600640 1 --2- 172.21.15.123:0/1979947034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a180ae990 0x7f7a180d0160 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009870 tx=0x7f7a0c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642101f1558a919d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:32.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.584+0000 7f7a17e00640 1 -- 172.21.15.123:0/1979947034 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c037070 con 0x7f7a180ae990 2024-04-24T05:47:32.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.584+0000 7f7a17e00640 1 -- 172.21.15.123:0/1979947034 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a0c032440 con 0x7f7a180ae990 2024-04-24T05:47:32.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.584+0000 7f7a17e00640 1 -- 172.21.15.123:0/1979947034 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c032810 con 0x7f7a180ae990 2024-04-24T05:47:32.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.585+0000 7f7a20000640 1 -- 172.21.15.123:0/1979947034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a180ae990 msgr2=0x7f7a180d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:32.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.585+0000 7f7a20000640 1 --2- 172.21.15.123:0/1979947034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a180ae990 0x7f7a180d0160 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009870 tx=0x7f7a0c02a210 comp rx=0 tx=0).stop 2024-04-24T05:47:32.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.585+0000 7f7a20000640 1 -- 172.21.15.123:0/1979947034 shutdown_connections 2024-04-24T05:47:32.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.585+0000 7f7a20000640 1 --2- 172.21.15.123:0/1979947034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1806c410 0x7f7a180ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:32.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.585+0000 7f7a20000640 1 --2- 172.21.15.123:0/1979947034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a180ae990 0x7f7a180d0160 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:32.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.585+0000 7f7a20000640 1 -- 172.21.15.123:0/1979947034 >> 172.21.15.123:0/1979947034 conn(0x7f7a180b51f0 msgr2=0x7f7a180b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:32.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.585+0000 7f7a20000640 1 -- 172.21.15.123:0/1979947034 shutdown_connections 2024-04-24T05:47:32.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.585+0000 7f7a20000640 1 -- 172.21.15.123:0/1979947034 wait complete. 2024-04-24T05:47:32.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.587+0000 7f7a20000640 1 Processor -- start 2024-04-24T05:47:32.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.587+0000 7f7a20000640 1 -- start start 2024-04-24T05:47:32.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.588+0000 7f7a20000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1806c410 0x7f7a180c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:32.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.588+0000 7f7a20000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a180ae990 0x7f7a1809a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:32.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.588+0000 7f7a20000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a180ce610 0x7f7a1809a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:32.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.588+0000 7f7a20000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a180aed70 con 0x7f7a180ce610 2024-04-24T05:47:32.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.588+0000 7f7a20000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1806e270 con 0x7f7a180ae990 2024-04-24T05:47:32.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.588+0000 7f7a20000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a1806c870 con 0x7f7a1806c410 2024-04-24T05:47:32.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.588+0000 7f7a1e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a180ce610 0x7f7a1809a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:32.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.589+0000 7f7a1e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a180ce610 0x7f7a1809a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45082/0 (socket says 172.21.15.123:45082) 2024-04-24T05:47:32.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.589+0000 7f7a1e000640 1 -- 172.21.15.123:0/570013364 learned_addr learned my addr 172.21.15.123:0/570013364 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:32.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.589+0000 7f7a1d600640 1 --2- 172.21.15.123:0/570013364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1806c410 0x7f7a180c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:32.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.589+0000 7f7a1cc00640 1 --2- 172.21.15.123:0/570013364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a180ae990 0x7f7a1809a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:32.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.589+0000 7f7a1e000640 1 -- 172.21.15.123:0/570013364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1806c410 msgr2=0x7f7a180c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:32.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.589+0000 7f7a1e000640 1 --2- 172.21.15.123:0/570013364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1806c410 0x7f7a180c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:32.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.589+0000 7f7a1e000640 1 -- 172.21.15.123:0/570013364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a180ae990 msgr2=0x7f7a1809a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:32.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.589+0000 7f7a1e000640 1 --2- 172.21.15.123:0/570013364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a180ae990 0x7f7a1809a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:32.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.589+0000 7f7a1e000640 1 -- 172.21.15.123:0/570013364 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1809a690 con 0x7f7a180ce610 2024-04-24T05:47:32.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.589+0000 7f7a1e000640 1 --2- 172.21.15.123:0/570013364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a180ce610 0x7f7a1809a9f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000c780 tx=0x7f7a1000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:32.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.589+0000 7f7a15600640 1 -- 172.21.15.123:0/570013364 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10004b60 con 0x7f7a180ce610 2024-04-24T05:47:32.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.590+0000 7f7a15600640 1 -- 172.21.15.123:0/570013364 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a10004d00 con 0x7f7a180ce610 2024-04-24T05:47:32.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.590+0000 7f7a15600640 1 -- 172.21.15.123:0/570013364 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000aa90 con 0x7f7a180ce610 2024-04-24T05:47:32.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.590+0000 7f7a20000640 1 -- 172.21.15.123:0/570013364 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1809d910 con 0x7f7a180ce610 2024-04-24T05:47:32.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.590+0000 7f7a20000640 1 -- 172.21.15.123:0/570013364 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1809ded0 con 0x7f7a180ce610 2024-04-24T05:47:32.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.591+0000 7f7a20000640 1 -- 172.21.15.123:0/570013364 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a18068a20 con 0x7f7a180ce610 2024-04-24T05:47:32.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.592+0000 7f7a15600640 1 -- 172.21.15.123:0/570013364 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a1000ac30 con 0x7f7a180ce610 2024-04-24T05:47:32.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.593+0000 7f7a15600640 1 --2- 172.21.15.123:0/570013364 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f79f806c8f0 0x7f79f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:32.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.593+0000 7f7a15600640 1 -- 172.21.15.123:0/570013364 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6391+0+0 (secure 0 0 0) 0x7f7a1008c0b0 con 0x7f7a180ce610 2024-04-24T05:47:32.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.593+0000 7f7a1d600640 1 --2- 172.21.15.123:0/570013364 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f79f806c8f0 0x7f79f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:32.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.594+0000 7f7a1d600640 1 --2- 172.21.15.123:0/570013364 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f79f806c8f0 0x7f79f806edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c034000 tx=0x7f7a0c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:32.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.598+0000 7f7a15600640 1 -- 172.21.15.123:0/570013364 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a10020070 con 0x7f7a180ce610 2024-04-24T05:47:32.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:32.799+0000 7f7a20000640 1 -- 172.21.15.123:0/570013364 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7a1806cb80 con 0x7f7a180ce610 2024-04-24T05:47:33.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.356+0000 7f7a15600640 1 -- 172.21.15.123:0/570013364 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7f7a1005a220 con 0x7f7a180ce610 2024-04-24T05:47:33.358 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' removed 2024-04-24T05:47:33.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.360+0000 7f7a20000640 1 -- 172.21.15.123:0/570013364 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f79f806c8f0 msgr2=0x7f79f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:33.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.360+0000 7f7a20000640 1 --2- 172.21.15.123:0/570013364 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f79f806c8f0 0x7f79f806edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c034000 tx=0x7f7a0c02e000 comp rx=0 tx=0).stop 2024-04-24T05:47:33.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.360+0000 7f7a20000640 1 -- 172.21.15.123:0/570013364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a180ce610 msgr2=0x7f7a1809a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:33.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.360+0000 7f7a20000640 1 --2- 172.21.15.123:0/570013364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a180ce610 0x7f7a1809a9f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000c780 tx=0x7f7a1000cc40 comp rx=0 tx=0).stop 2024-04-24T05:47:33.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.360+0000 7f7a20000640 1 -- 172.21.15.123:0/570013364 shutdown_connections 2024-04-24T05:47:33.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.360+0000 7f7a20000640 1 --2- 172.21.15.123:0/570013364 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f79f806c8f0 0x7f79f806edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:33.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.360+0000 7f7a20000640 1 --2- 172.21.15.123:0/570013364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a180ce610 0x7f7a1809a9f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:33.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.360+0000 7f7a20000640 1 --2- 172.21.15.123:0/570013364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a180ae990 0x7f7a1809a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:33.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.360+0000 7f7a20000640 1 --2- 172.21.15.123:0/570013364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a1806c410 0x7f7a180c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:33.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.360+0000 7f7a20000640 1 -- 172.21.15.123:0/570013364 >> 172.21.15.123:0/570013364 conn(0x7f7a180b51f0 msgr2=0x7f7a180ad350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:33.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.360+0000 7f7a20000640 1 -- 172.21.15.123:0/570013364 shutdown_connections 2024-04-24T05:47:33.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.360+0000 7f7a20000640 1 -- 172.21.15.123:0/570013364 wait complete. 2024-04-24T05:47:33.387 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-24T05:47:33.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.577+0000 7feb6ce00640 1 Processor -- start 2024-04-24T05:47:33.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.578+0000 7feb6ce00640 1 -- start start 2024-04-24T05:47:33.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.579+0000 7feb6ce00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feb680d45a0 0x7feb680d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:33.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.579+0000 7feb6ce00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb680b6eb0 con 0x7feb680d31f0 2024-04-24T05:47:33.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.579+0000 7feb6ce00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6815ee50 con 0x7feb680cfac0 2024-04-24T05:47:33.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.579+0000 7feb6ce00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7feb680d4ec0 con 0x7feb680d45a0 2024-04-24T05:47:33.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.580+0000 7feb66800640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7feb680d31f0 0x7feb680b67a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33544/0 (socket says 172.21.15.123:33544) 2024-04-24T05:47:33.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.580+0000 7feb66800640 1 -- 172.21.15.123:0/1000741325 learned_addr learned my addr 172.21.15.123:0/1000741325 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:33.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.580+0000 7feb65e00640 1 --2- 172.21.15.123:0/1000741325 >> v2:172.21.15.155:3300/0 conn(0x7feb680d45a0 0x7feb680d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:33.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.580+0000 7feb64a00640 1 -- 172.21.15.123:0/1000741325 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1314932969 0 0) 0x7feb680b6eb0 con 0x7feb680d31f0 2024-04-24T05:47:33.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.580+0000 7feb64a00640 1 -- 172.21.15.123:0/1000741325 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb4c003520 con 0x7feb680d31f0 2024-04-24T05:47:33.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.580+0000 7feb64a00640 1 -- 172.21.15.123:0/1000741325 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3115448784 0 0) 0x7feb4c003520 con 0x7feb680d31f0 2024-04-24T05:47:33.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.580+0000 7feb64a00640 1 -- 172.21.15.123:0/1000741325 >> v2:172.21.15.155:3300/0 conn(0x7feb680d45a0 msgr2=0x7feb680d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:33.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.580+0000 7feb64a00640 1 --2- 172.21.15.123:0/1000741325 >> v2:172.21.15.155:3300/0 conn(0x7feb680d45a0 0x7feb680d4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:33.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.580+0000 7feb64a00640 1 -- 172.21.15.123:0/1000741325 >> v1:172.21.15.155:6789/0 conn(0x7feb680cfac0 legacy=0x7feb680b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:33.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.580+0000 7feb64a00640 1 -- 172.21.15.123:0/1000741325 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb680cff80 con 0x7feb680d31f0 2024-04-24T05:47:33.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.580+0000 7feb64a00640 1 -- 172.21.15.123:0/1000741325 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7feb60001670 con 0x7feb680d31f0 2024-04-24T05:47:33.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.580+0000 7feb64a00640 1 -- 172.21.15.123:0/1000741325 >> v1:172.21.15.123:6789/0 conn(0x7feb680d31f0 legacy=0x7feb680b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:33.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.580+0000 7feb64a00640 1 --2- 172.21.15.123:0/1000741325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c0039b0 0x7feb4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:33.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.581+0000 7feb64a00640 1 --2- 172.21.15.123:0/1000741325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb4c004800 0x7feb4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:33.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.581+0000 7feb64a00640 1 --2- 172.21.15.123:0/1000741325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4c0051c0 0x7feb4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:33.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.581+0000 7feb64a00640 1 -- 172.21.15.123:0/1000741325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb680d4ec0 con 0x7feb4c0039b0 2024-04-24T05:47:33.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.581+0000 7feb64a00640 1 -- 172.21.15.123:0/1000741325 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feb4c00c110 con 0x7feb4c0051c0 2024-04-24T05:47:33.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.581+0000 7feb64a00640 1 -- 172.21.15.123:0/1000741325 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feb4c00c310 con 0x7feb4c004800 2024-04-24T05:47:33.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.581+0000 7feb65e00640 1 --2- 172.21.15.123:0/1000741325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c0039b0 0x7feb4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:33.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.581+0000 7feb65400640 1 --2- 172.21.15.123:0/1000741325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb4c004800 0x7feb4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:33.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.581+0000 7feb66800640 1 --2- 172.21.15.123:0/1000741325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4c0051c0 0x7feb4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:33.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.581+0000 7feb65e00640 1 -- 172.21.15.123:0/1000741325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb4c004800 msgr2=0x7feb4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:33.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.581+0000 7feb65e00640 1 --2- 172.21.15.123:0/1000741325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb4c004800 0x7feb4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:33.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.581+0000 7feb65e00640 1 -- 172.21.15.123:0/1000741325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4c0051c0 msgr2=0x7feb4c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:33.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.581+0000 7feb65e00640 1 --2- 172.21.15.123:0/1000741325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4c0051c0 0x7feb4c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:33.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.581+0000 7feb65e00640 1 -- 172.21.15.123:0/1000741325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4c003520 con 0x7feb4c0039b0 2024-04-24T05:47:33.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.581+0000 7feb66800640 1 -- 172.21.15.123:0/1000741325 reap_dead start 2024-04-24T05:47:33.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.582+0000 7feb65e00640 1 --2- 172.21.15.123:0/1000741325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c0039b0 0x7feb4c003dd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7feb5000ba30 tx=0x7feb4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a54dd87a9e9edc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:33.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.583+0000 7feb64a00640 1 -- 172.21.15.123:0/1000741325 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50030070 con 0x7feb4c0039b0 2024-04-24T05:47:33.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.583+0000 7feb64a00640 1 -- 172.21.15.123:0/1000741325 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5002ade0 con 0x7feb4c0039b0 2024-04-24T05:47:33.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.583+0000 7feb64a00640 1 -- 172.21.15.123:0/1000741325 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5000b600 con 0x7feb4c0039b0 2024-04-24T05:47:33.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.583+0000 7feb6ce00640 1 -- 172.21.15.123:0/1000741325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c0039b0 msgr2=0x7feb4c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:33.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.583+0000 7feb6ce00640 1 --2- 172.21.15.123:0/1000741325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c0039b0 0x7feb4c003dd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7feb5000ba30 tx=0x7feb4c005830 comp rx=0 tx=0).stop 2024-04-24T05:47:33.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.583+0000 7feb6ce00640 1 -- 172.21.15.123:0/1000741325 shutdown_connections 2024-04-24T05:47:33.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.583+0000 7feb6ce00640 1 --2- 172.21.15.123:0/1000741325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c0039b0 0x7feb4c003dd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:33.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.583+0000 7feb6ce00640 1 -- 172.21.15.123:0/1000741325 >> 172.21.15.123:0/1000741325 conn(0x7feb6809d970 msgr2=0x7feb680b95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:33.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.583+0000 7feb6ce00640 1 -- 172.21.15.123:0/1000741325 shutdown_connections 2024-04-24T05:47:33.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.583+0000 7feb6ce00640 1 -- 172.21.15.123:0/1000741325 wait complete. 2024-04-24T05:47:33.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.585+0000 7feb6ce00640 1 Processor -- start 2024-04-24T05:47:33.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.585+0000 7feb6ce00640 1 -- start start 2024-04-24T05:47:33.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.586+0000 7feb6ce00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4c0039b0 0x7feb68155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:33.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.586+0000 7feb6ce00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680b5da0 0x7feb680b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:33.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.586+0000 7feb6ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb6809ff00 0x7feb680a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:33.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.586+0000 7feb6ce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb680d0820 con 0x7feb6809ff00 2024-04-24T05:47:33.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.586+0000 7feb6ce00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feb680d0540 con 0x7feb4c0039b0 2024-04-24T05:47:33.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.586+0000 7feb6ce00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feb680d0b00 con 0x7feb680b5da0 2024-04-24T05:47:33.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.587+0000 7feb66800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb6809ff00 0x7feb680a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:33.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.587+0000 7feb66800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb6809ff00 0x7feb680a0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45098/0 (socket says 172.21.15.123:45098) 2024-04-24T05:47:33.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.587+0000 7feb66800640 1 -- 172.21.15.123:0/1716481147 learned_addr learned my addr 172.21.15.123:0/1716481147 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:33.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.587+0000 7feb65e00640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4c0039b0 0x7feb68155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:33.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.587+0000 7feb65400640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680b5da0 0x7feb680b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:33.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.587+0000 7feb66800640 1 -- 172.21.15.123:0/1716481147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680b5da0 msgr2=0x7feb680b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:33.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.587+0000 7feb66800640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680b5da0 0x7feb680b6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:33.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.587+0000 7feb66800640 1 -- 172.21.15.123:0/1716481147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4c0039b0 msgr2=0x7feb68155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:33.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.587+0000 7feb66800640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4c0039b0 0x7feb68155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:33.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.587+0000 7feb66800640 1 -- 172.21.15.123:0/1716481147 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb680a0bc0 con 0x7feb6809ff00 2024-04-24T05:47:33.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.587+0000 7feb65e00640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4c0039b0 0x7feb68155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:33.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.587+0000 7feb66800640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb6809ff00 0x7feb680a0380 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7feb60000c00 tx=0x7feb6000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:33.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.588+0000 7feb5e000640 1 -- 172.21.15.123:0/1716481147 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60007440 con 0x7feb6809ff00 2024-04-24T05:47:33.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.588+0000 7feb5e000640 1 -- 172.21.15.123:0/1716481147 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb600075e0 con 0x7feb6809ff00 2024-04-24T05:47:33.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.588+0000 7feb5e000640 1 -- 172.21.15.123:0/1716481147 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60004a20 con 0x7feb6809ff00 2024-04-24T05:47:33.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.588+0000 7feb6ce00640 1 -- 172.21.15.123:0/1716481147 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb680b4520 con 0x7feb6809ff00 2024-04-24T05:47:33.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.588+0000 7feb6ce00640 1 -- 172.21.15.123:0/1716481147 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb680b4ae0 con 0x7feb6809ff00 2024-04-24T05:47:33.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.589+0000 7feb6ce00640 1 -- 172.21.15.123:0/1716481147 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb68068a20 con 0x7feb6809ff00 2024-04-24T05:47:33.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.590+0000 7feb5e000640 1 -- 172.21.15.123:0/1716481147 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb60004bc0 con 0x7feb6809ff00 2024-04-24T05:47:33.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.591+0000 7feb5e000640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feb4406c920 0x7feb4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:33.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.591+0000 7feb5e000640 1 -- 172.21.15.123:0/1716481147 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7feb6008b810 con 0x7feb6809ff00 2024-04-24T05:47:33.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.591+0000 7feb65e00640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feb4406c920 0x7feb4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:33.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.592+0000 7feb65e00640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feb4406c920 0x7feb4406ede0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feb5000ba00 tx=0x7feb50030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:33.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.595+0000 7feb5e000640 1 -- 172.21.15.123:0/1716481147 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feb6001a060 con 0x7feb6809ff00 2024-04-24T05:47:33.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:33.795+0000 7feb6ce00640 1 -- 172.21.15.123:0/1716481147 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7feb680b1d00 con 0x7feb6809ff00 2024-04-24T05:47:34.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.373+0000 7feb5e000640 1 -- 172.21.15.123:0/1716481147 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7feb60059b10 con 0x7feb6809ff00 2024-04-24T05:47:34.375 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data_ec' removed 2024-04-24T05:47:34.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.377+0000 7feb6ce00640 1 -- 172.21.15.123:0/1716481147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feb4406c920 msgr2=0x7feb4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:34.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.377+0000 7feb6ce00640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feb4406c920 0x7feb4406ede0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feb5000ba00 tx=0x7feb50030000 comp rx=0 tx=0).stop 2024-04-24T05:47:34.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.377+0000 7feb6ce00640 1 -- 172.21.15.123:0/1716481147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb6809ff00 msgr2=0x7feb680a0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:34.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.377+0000 7feb6ce00640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb6809ff00 0x7feb680a0380 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7feb60000c00 tx=0x7feb6000d890 comp rx=0 tx=0).stop 2024-04-24T05:47:34.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.378+0000 7feb6ce00640 1 -- 172.21.15.123:0/1716481147 shutdown_connections 2024-04-24T05:47:34.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.378+0000 7feb6ce00640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feb4406c920 0x7feb4406ede0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:34.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.378+0000 7feb6ce00640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb6809ff00 0x7feb680a0380 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:34.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.378+0000 7feb6ce00640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb680b5da0 0x7feb680b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:34.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.378+0000 7feb6ce00640 1 --2- 172.21.15.123:0/1716481147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4c0039b0 0x7feb68155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:34.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.378+0000 7feb6ce00640 1 -- 172.21.15.123:0/1716481147 >> 172.21.15.123:0/1716481147 conn(0x7feb6809d970 msgr2=0x7feb6806c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:34.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.379+0000 7feb6ce00640 1 -- 172.21.15.123:0/1716481147 shutdown_connections 2024-04-24T05:47:34.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.379+0000 7feb6ce00640 1 -- 172.21.15.123:0/1716481147 wait complete. 2024-04-24T05:47:34.405 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-24T05:47:34.405 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-24T05:47:34.405 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-24T05:47:34.405 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T05:47:34.406 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-24T05:47:34.406 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-24T05:47:34.406 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-24T05:47:34.406 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T05:47:34.406 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-24T05:47:34.406 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-24T05:47:34.407 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-24T05:47:34.407 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T05:47:34.407 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-24T05:47:34.407 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-24T05:47:34.407 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-24T05:47:34.407 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T05:47:34.424 INFO:tasks.ceph.mds.a:Stopped 2024-04-24T05:47:34.424 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-24T05:47:34.428 INFO:tasks.ceph.mds.c:Stopped 2024-04-24T05:47:34.428 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-24T05:47:34.428 INFO:tasks.ceph.mds.b:Stopped 2024-04-24T05:47:34.428 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-24T05:47:34.429 INFO:tasks.ceph.mds.d:Stopped 2024-04-24T05:47:34.429 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-24T05:47:34.471 INFO:tasks.ceph.mds.a:Started 2024-04-24T05:47:34.472 DEBUG:teuthology.parallel:result is None 2024-04-24T05:47:34.475 INFO:tasks.ceph.mds.c:Started 2024-04-24T05:47:34.476 DEBUG:teuthology.parallel:result is None 2024-04-24T05:47:34.476 INFO:tasks.ceph.mds.b:Started 2024-04-24T05:47:34.477 DEBUG:teuthology.parallel:result is None 2024-04-24T05:47:34.485 INFO:tasks.ceph.mds.d:Started 2024-04-24T05:47:34.485 DEBUG:teuthology.parallel:result is None 2024-04-24T05:47:34.485 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-24T05:47:34.618 INFO:tasks.ceph.mds.c.smithi123.stdout:starting mds.c at 2024-04-24T05:47:34.619 INFO:tasks.ceph.mds.a.smithi123.stdout:starting mds.a at 2024-04-24T05:47:34.630 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-04-24T05:47:34.637 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-04-24T05:47:34.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.671+0000 7f65b5a00640 1 Processor -- start 2024-04-24T05:47:34.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.672+0000 7f65b5a00640 1 -- start start 2024-04-24T05:47:34.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.673+0000 7f65b5a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f65b015ebf0 0x7f65b006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:34.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.673+0000 7f65b5a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f65b00d35f0 0x7f65b00b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:34.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.673+0000 7f65b5a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65b00d3af0 con 0x7f65b00d1500 2024-04-24T05:47:34.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.673+0000 7f65b5a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f65b00b6030 con 0x7f65b015ebf0 2024-04-24T05:47:34.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.673+0000 7f65b5a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f65b00b6310 con 0x7f65b00d35f0 2024-04-24T05:47:34.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.673+0000 7f65aea00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f65b015ebf0 0x7f65b006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:34.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.673+0000 7f65aea00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f65b015ebf0 0x7f65b006ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45154/0 (socket says 172.21.15.123:45154) 2024-04-24T05:47:34.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.673+0000 7f65aea00640 1 -- 172.21.15.123:0/672696463 learned_addr learned my addr 172.21.15.123:0/672696463 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:34.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.674+0000 7f65ae000640 1 --2- 172.21.15.123:0/672696463 >> v2:172.21.15.155:3300/0 conn(0x7f65b00d35f0 0x7f65b00b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:34.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.674+0000 7f65aea00640 1 -- 172.21.15.123:0/672696463 >> v2:172.21.15.155:3300/0 conn(0x7f65b00d35f0 msgr2=0x7f65b00b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:34.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.674+0000 7f65aea00640 1 --2- 172.21.15.123:0/672696463 >> v2:172.21.15.155:3300/0 conn(0x7f65b00d35f0 0x7f65b00b67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:34.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.674+0000 7f65aea00640 1 -- 172.21.15.123:0/672696463 >> v1:172.21.15.155:6789/0 conn(0x7f65b00d1500 legacy=0x7f65b01625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:34.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.674+0000 7f65aea00640 1 -- 172.21.15.123:0/672696463 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b00d3dd0 con 0x7f65b015ebf0 2024-04-24T05:47:34.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.674+0000 7f65ae000640 1 --2- 172.21.15.123:0/672696463 >> v2:172.21.15.155:3300/0 conn(0x7f65b00d35f0 0x7f65b00b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:34.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.675+0000 7f65aea00640 1 --2- 172.21.15.123:0/672696463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b015ebf0 0x7f65b006ed20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f65b00c80b0 tx=0x7f65a002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee0c6c992b44cb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:34.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.675+0000 7f65ad600640 1 -- 172.21.15.123:0/672696463 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0036070 con 0x7f65b015ebf0 2024-04-24T05:47:34.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.675+0000 7f65ad600640 1 -- 172.21.15.123:0/672696463 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a0031600 con 0x7f65b015ebf0 2024-04-24T05:47:34.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.675+0000 7f65ad600640 1 -- 172.21.15.123:0/672696463 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a00319d0 con 0x7f65b015ebf0 2024-04-24T05:47:34.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.675+0000 7f65b5a00640 1 -- 172.21.15.123:0/672696463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b015ebf0 msgr2=0x7f65b006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:34.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.675+0000 7f65b5a00640 1 --2- 172.21.15.123:0/672696463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b015ebf0 0x7f65b006ed20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f65b00c80b0 tx=0x7f65a002a6f0 comp rx=0 tx=0).stop 2024-04-24T05:47:34.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.676+0000 7f65b5a00640 1 -- 172.21.15.123:0/672696463 shutdown_connections 2024-04-24T05:47:34.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.676+0000 7f65b5a00640 1 --2- 172.21.15.123:0/672696463 >> v2:172.21.15.155:3300/0 conn(0x7f65b00d35f0 0x7f65b00b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:34.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.676+0000 7f65b5a00640 1 --2- 172.21.15.123:0/672696463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b015ebf0 0x7f65b006ed20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:34.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.676+0000 7f65b5a00640 1 -- 172.21.15.123:0/672696463 >> 172.21.15.123:0/672696463 conn(0x7f65b00b9570 msgr2=0x7f65b00b9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:34.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.676+0000 7f65b5a00640 1 -- 172.21.15.123:0/672696463 shutdown_connections 2024-04-24T05:47:34.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.676+0000 7f65b5a00640 1 -- 172.21.15.123:0/672696463 wait complete. 2024-04-24T05:47:34.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.677+0000 7f65b5a00640 1 Processor -- start 2024-04-24T05:47:34.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.678+0000 7f65b5a00640 1 -- start start 2024-04-24T05:47:34.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.678+0000 7f65b5a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65b00d1500 0x7f65b01e7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:34.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65b5a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b00d35f0 0x7f65b0158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:34.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65b5a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b015ebf0 0x7f65b015f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:34.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65b5a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65b006f3e0 con 0x7f65b00d35f0 2024-04-24T05:47:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65b5a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f65b006f260 con 0x7f65b015ebf0 2024-04-24T05:47:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65b5a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f65b00b6ce0 con 0x7f65b00d1500 2024-04-24T05:47:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65ae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b00d35f0 0x7f65b0158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65ae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b00d35f0 0x7f65b0158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45160/0 (socket says 172.21.15.123:45160) 2024-04-24T05:47:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65ae000640 1 -- 172.21.15.123:0/2428680138 learned_addr learned my addr 172.21.15.123:0/2428680138 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65aea00640 1 --2- 172.21.15.123:0/2428680138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65b00d1500 0x7f65b01e7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65af400640 1 --2- 172.21.15.123:0/2428680138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b015ebf0 0x7f65b015f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65ae000640 1 -- 172.21.15.123:0/2428680138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65b00d1500 msgr2=0x7f65b01e7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65ae000640 1 --2- 172.21.15.123:0/2428680138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65b00d1500 0x7f65b01e7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65ae000640 1 -- 172.21.15.123:0/2428680138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b015ebf0 msgr2=0x7f65b015f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65ae000640 1 --2- 172.21.15.123:0/2428680138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b015ebf0 0x7f65b015f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65ae000640 1 -- 172.21.15.123:0/2428680138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b01e7860 con 0x7f65b00d35f0 2024-04-24T05:47:34.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.679+0000 7f65ae000640 1 --2- 172.21.15.123:0/2428680138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b00d35f0 0x7f65b0158f70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f659800c7f0 tx=0x7f659800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:34.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.680+0000 7f65a6a00640 1 -- 172.21.15.123:0/2428680138 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598015070 con 0x7f65b00d35f0 2024-04-24T05:47:34.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.680+0000 7f65a6a00640 1 -- 172.21.15.123:0/2428680138 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6598007a30 con 0x7f65b00d35f0 2024-04-24T05:47:34.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.680+0000 7f65a6a00640 1 -- 172.21.15.123:0/2428680138 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598007e00 con 0x7f65b00d35f0 2024-04-24T05:47:34.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.680+0000 7f65b5a00640 1 -- 172.21.15.123:0/2428680138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b01e7b40 con 0x7f65b00d35f0 2024-04-24T05:47:34.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.680+0000 7f65b5a00640 1 -- 172.21.15.123:0/2428680138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b01e8100 con 0x7f65b00d35f0 2024-04-24T05:47:34.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.681+0000 7f65b5a00640 1 -- 172.21.15.123:0/2428680138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b008ca80 con 0x7f65b00d35f0 2024-04-24T05:47:34.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.682+0000 7f65a6a00640 1 -- 172.21.15.123:0/2428680138 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6598018070 con 0x7f65b00d35f0 2024-04-24T05:47:34.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.683+0000 7f65a6a00640 1 --2- 172.21.15.123:0/2428680138 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f658c06c8f0 0x7f658c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:34.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.683+0000 7f65a6a00640 1 -- 172.21.15.123:0/2428680138 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5335+0+0 (secure 0 0 0) 0x7f659808b9e0 con 0x7f65b00d35f0 2024-04-24T05:47:34.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.683+0000 7f65aea00640 1 --2- 172.21.15.123:0/2428680138 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f658c06c8f0 0x7f658c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:34.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.684+0000 7f65aea00640 1 --2- 172.21.15.123:0/2428680138 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f658c06c8f0 0x7f658c06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f65a0033000 tx=0x7f65a0009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:34.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.687+0000 7f65a6a00640 1 -- 172.21.15.123:0/2428680138 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65b008ca80 con 0x7f65b00d35f0 2024-04-24T05:47:34.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:34.900+0000 7f65b5a00640 1 -- 172.21.15.123:0/2428680138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f65b0068a20 con 0x7f65b00d35f0 2024-04-24T05:47:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.414+0000 7f65a6a00640 1 -- 172.21.15.123:0/2428680138 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) ==== 97+0+0 (secure 0 0 0) 0x7f6598059890 con 0x7f65b00d35f0 2024-04-24T05:47:35.416 INFO:teuthology.orchestra.run.smithi123.stderr: removed all blocklist entries 2024-04-24T05:47:35.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.419+0000 7f65b5a00640 1 -- 172.21.15.123:0/2428680138 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f658c06c8f0 msgr2=0x7f658c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:35.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.419+0000 7f65b5a00640 1 --2- 172.21.15.123:0/2428680138 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f658c06c8f0 0x7f658c06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f65a0033000 tx=0x7f65a0009000 comp rx=0 tx=0).stop 2024-04-24T05:47:35.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.419+0000 7f65b5a00640 1 -- 172.21.15.123:0/2428680138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b00d35f0 msgr2=0x7f65b0158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:35.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.419+0000 7f65b5a00640 1 --2- 172.21.15.123:0/2428680138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b00d35f0 0x7f65b0158f70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f659800c7f0 tx=0x7f659800cbf0 comp rx=0 tx=0).stop 2024-04-24T05:47:35.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.420+0000 7f65b5a00640 1 -- 172.21.15.123:0/2428680138 shutdown_connections 2024-04-24T05:47:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.420+0000 7f65b5a00640 1 --2- 172.21.15.123:0/2428680138 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f658c06c8f0 0x7f658c06edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.420+0000 7f65b5a00640 1 --2- 172.21.15.123:0/2428680138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b015ebf0 0x7f65b015f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.420+0000 7f65b5a00640 1 --2- 172.21.15.123:0/2428680138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65b00d35f0 0x7f65b0158f70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.420+0000 7f65b5a00640 1 --2- 172.21.15.123:0/2428680138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65b00d1500 0x7f65b01e7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.420+0000 7f65b5a00640 1 -- 172.21.15.123:0/2428680138 >> 172.21.15.123:0/2428680138 conn(0x7f65b00b9570 msgr2=0x7f65b00b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.420+0000 7f65b5a00640 1 -- 172.21.15.123:0/2428680138 shutdown_connections 2024-04-24T05:47:35.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.420+0000 7f65b5a00640 1 -- 172.21.15.123:0/2428680138 wait complete. 2024-04-24T05:47:35.443 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-24T05:47:35.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.631+0000 7f6b3b600640 1 Processor -- start 2024-04-24T05:47:35.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.631+0000 7f6b3b600640 1 -- start start 2024-04-24T05:47:35.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.632+0000 7f6b3b600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6b3415a2a0 0x7f6b340ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:35.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.632+0000 7f6b3b600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b3415cdf0 con 0x7f6b340ad690 2024-04-24T05:47:35.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.632+0000 7f6b3b600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b3415d0d0 con 0x7f6b340af950 2024-04-24T05:47:35.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.632+0000 7f6b3b600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6b3415a680 con 0x7f6b3415a2a0 2024-04-24T05:47:35.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.633+0000 7f6b38c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6b3415a2a0 0x7f6b340ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:35.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.633+0000 7f6b33e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6b340ad690 0x7f6b340af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33568/0 (socket says 172.21.15.123:33568) 2024-04-24T05:47:35.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.633+0000 7f6b33e00640 1 -- 172.21.15.123:0/1715039058 learned_addr learned my addr 172.21.15.123:0/1715039058 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:35.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.633+0000 7f6b38c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6b3415a2a0 0x7f6b340ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45166/0 (socket says 172.21.15.123:45166) 2024-04-24T05:47:35.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.633+0000 7f6b38c00640 1 -- 172.21.15.123:0/1715039058 >> v1:172.21.15.155:6789/0 conn(0x7f6b340af950 legacy=0x7f6b3406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:35.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.633+0000 7f6b38c00640 1 -- 172.21.15.123:0/1715039058 >> v1:172.21.15.123:6789/0 conn(0x7f6b340ad690 legacy=0x7f6b340af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:35.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.633+0000 7f6b38c00640 1 -- 172.21.15.123:0/1715039058 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3415d3b0 con 0x7f6b3415a2a0 2024-04-24T05:47:35.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.634+0000 7f6b38c00640 1 --2- 172.21.15.123:0/1715039058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3415a2a0 0x7f6b340ad150 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6b280097a0 tx=0x7f6b2802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286150ae786b4aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.635+0000 7f6b33400640 1 -- 172.21.15.123:0/1715039058 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28037070 con 0x7f6b3415a2a0 2024-04-24T05:47:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.635+0000 7f6b33400640 1 -- 172.21.15.123:0/1715039058 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2802ae50 con 0x7f6b3415a2a0 2024-04-24T05:47:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.635+0000 7f6b33400640 1 -- 172.21.15.123:0/1715039058 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28032660 con 0x7f6b3415a2a0 2024-04-24T05:47:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.635+0000 7f6b3b600640 1 -- 172.21.15.123:0/1715039058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3415a2a0 msgr2=0x7f6b340ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.635+0000 7f6b3b600640 1 --2- 172.21.15.123:0/1715039058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3415a2a0 0x7f6b340ad150 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6b280097a0 tx=0x7f6b2802a140 comp rx=0 tx=0).stop 2024-04-24T05:47:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.635+0000 7f6b3b600640 1 -- 172.21.15.123:0/1715039058 shutdown_connections 2024-04-24T05:47:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.635+0000 7f6b3b600640 1 --2- 172.21.15.123:0/1715039058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3415a2a0 0x7f6b340ad150 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.635+0000 7f6b3b600640 1 -- 172.21.15.123:0/1715039058 >> 172.21.15.123:0/1715039058 conn(0x7f6b340b1e70 msgr2=0x7f6b340b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.635+0000 7f6b3b600640 1 -- 172.21.15.123:0/1715039058 shutdown_connections 2024-04-24T05:47:35.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.636+0000 7f6b3b600640 1 -- 172.21.15.123:0/1715039058 wait complete. 2024-04-24T05:47:35.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.638+0000 7f6b3b600640 1 Processor -- start 2024-04-24T05:47:35.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.638+0000 7f6b3b600640 1 -- start start 2024-04-24T05:47:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.639+0000 7f6b3b600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b340ad690 0x7f6b340cef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.639+0000 7f6b3b600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b340af950 0x7f6b340ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.639+0000 7f6b3b600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3415a2a0 0x7f6b340cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.639+0000 7f6b3b600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3406c210 con 0x7f6b3415a2a0 2024-04-24T05:47:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.639+0000 7f6b3b600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3415da00 con 0x7f6b340ad690 2024-04-24T05:47:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.639+0000 7f6b3b600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3406c390 con 0x7f6b340af950 2024-04-24T05:47:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.640+0000 7f6b39600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3415a2a0 0x7f6b340cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.640+0000 7f6b39600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3415a2a0 0x7f6b340cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45172/0 (socket says 172.21.15.123:45172) 2024-04-24T05:47:35.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.640+0000 7f6b39600640 1 -- 172.21.15.123:0/947261601 learned_addr learned my addr 172.21.15.123:0/947261601 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:35.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.640+0000 7f6b38c00640 1 --2- 172.21.15.123:0/947261601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b340ad690 0x7f6b340cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.640+0000 7f6b39600640 1 -- 172.21.15.123:0/947261601 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b340af950 msgr2=0x7f6b340ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.640+0000 7f6b39600640 1 --2- 172.21.15.123:0/947261601 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b340af950 0x7f6b340ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.640+0000 7f6b39600640 1 -- 172.21.15.123:0/947261601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b340ad690 msgr2=0x7f6b340cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.640+0000 7f6b39600640 1 --2- 172.21.15.123:0/947261601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b340ad690 0x7f6b340cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.640+0000 7f6b39600640 1 -- 172.21.15.123:0/947261601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b340ca040 con 0x7f6b3415a2a0 2024-04-24T05:47:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.640+0000 7f6b38c00640 1 --2- 172.21.15.123:0/947261601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b340ad690 0x7f6b340cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.640+0000 7f6b39600640 1 --2- 172.21.15.123:0/947261601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3415a2a0 0x7f6b340cd440 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00c7b0 tx=0x7f6b2c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.641+0000 7f6b30c00640 1 -- 172.21.15.123:0/947261601 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c00ce60 con 0x7f6b3415a2a0 2024-04-24T05:47:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.641+0000 7f6b30c00640 1 -- 172.21.15.123:0/947261601 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2c004b10 con 0x7f6b3415a2a0 2024-04-24T05:47:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.641+0000 7f6b30c00640 1 -- 172.21.15.123:0/947261601 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c00a850 con 0x7f6b3415a2a0 2024-04-24T05:47:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.641+0000 7f6b3b600640 1 -- 172.21.15.123:0/947261601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b340ca2e0 con 0x7f6b3415a2a0 2024-04-24T05:47:35.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.641+0000 7f6b3b600640 1 -- 172.21.15.123:0/947261601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b340ca8a0 con 0x7f6b3415a2a0 2024-04-24T05:47:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.642+0000 7f6b3b600640 1 -- 172.21.15.123:0/947261601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b34068a20 con 0x7f6b3415a2a0 2024-04-24T05:47:35.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.643+0000 7f6b30c00640 1 -- 172.21.15.123:0/947261601 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b2c00a9f0 con 0x7f6b3415a2a0 2024-04-24T05:47:35.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.644+0000 7f6b30c00640 1 --2- 172.21.15.123:0/947261601 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6b1406c8f0 0x7f6b1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:35.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.644+0000 7f6b30c00640 1 -- 172.21.15.123:0/947261601 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f6b2c08bc60 con 0x7f6b3415a2a0 2024-04-24T05:47:35.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.644+0000 7f6b38c00640 1 --2- 172.21.15.123:0/947261601 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6b1406c8f0 0x7f6b1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:35.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.645+0000 7f6b38c00640 1 --2- 172.21.15.123:0/947261601 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6b1406c8f0 0x7f6b1406edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6b28034000 tx=0x7f6b2802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:35.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.649+0000 7f6b30c00640 1 -- 172.21.15.123:0/947261601 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b2c020070 con 0x7f6b3415a2a0 2024-04-24T05:47:35.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.920+0000 7f6b3b600640 1 -- 172.21.15.123:0/947261601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6b340ad1f0 con 0x7f6b3415a2a0 2024-04-24T05:47:35.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.921+0000 7f6b30c00640 1 -- 172.21.15.123:0/947261601 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) ==== 80+0+6645 (secure 0 0 0) 0x7f6b2c05a1b0 con 0x7f6b3415a2a0 2024-04-24T05:47:35.923 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:35.924 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-24T05:47:35.924 INFO:teuthology.orchestra.run.smithi123.stdout: "auth_dump": [ 2024-04-24T05:47:35.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.924 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mds.a", 2024-04-24T05:47:35.924 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBYnChmnq3ZCxAA/jPeNwIiPPR/4z1oxDbmrg==", 2024-04-24T05:47:35.924 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.924 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-24T05:47:35.924 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2024-04-24T05:47:35.924 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mds.b", 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBYnChmJV4HKRAA4sb5ium46BxVm8vPrshrHg==", 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mds.c", 2024-04-24T05:47:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBYnChm1Lz0GRAA5I5XFsVJP/B2b5sJ8Xzfaw==", 2024-04-24T05:47:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-24T05:47:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2024-04-24T05:47:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2024-04-24T05:47:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mds.d", 2024-04-24T05:47:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBYnChm6IjANRAAOzvW4AE9/wb+nE7UERpUzg==", 2024-04-24T05:47:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-24T05:47:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2024-04-24T05:47:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2024-04-24T05:47:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.0", 2024-04-24T05:47:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBcnChmstDGMRAAcjdLoMmhVSQxYoQ2DFmwgw==", 2024-04-24T05:47:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-24T05:47:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-24T05:47:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.1", 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBgnChmm/o3ARAAyWAABEnw+Bj0K2Y8C9V4TQ==", 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.2", 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBjnChmxUXyDBAA+Te+IW8KZK9VlA6thlpGZQ==", 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.3", 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBmnChm6amzJhAAAr0M8SMHGIwpyWvaV2zkag==", 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.4", 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBsnChm6D7VMRAAp8Oejg6qgdkGE9scSWWHGw==", 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-24T05:47:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-24T05:47:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.5", 2024-04-24T05:47:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBwnChmJWQbERAA2/c/kpOLl5re5h2XnQehBw==", 2024-04-24T05:47:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-24T05:47:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.6", 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBznChmwGE9GxAAm5XkXoA0a7YHyKL4Dvojbg==", 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.7", 2024-04-24T05:47:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQB2nChmziE2JRAAeU2jmUbvOctMr4oc9FGuEQ==", 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.0", 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBZnChmQ4k9BhAAqlK7y/LktbsFTIUhRQCMYg==", 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow r", 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow rw", 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow rwx" 2024-04-24T05:47:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.1", 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBZnChmVke+GBAAUWUCOE5KNa5ZiOO2QJWQNw==", 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow r", 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow rw", 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow rwx" 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.2", 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBZnChmH8+MDxAA6XyqkcDkR8RW5WwxIeDp1A==", 2024-04-24T05:47:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-24T05:47:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow r", 2024-04-24T05:47:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow rw", 2024-04-24T05:47:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow rwx" 2024-04-24T05:47:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.3", 2024-04-24T05:47:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBZnChmqPdQIRAAzr7jrMslzCOsbGAHQKEH9w==", 2024-04-24T05:47:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2024-04-24T05:47:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow r", 2024-04-24T05:47:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow rw", 2024-04-24T05:47:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow rwx" 2024-04-24T05:47:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.admin", 2024-04-24T05:47:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBXnChmsw/jCBAAnhJKi1uIOmvadAg0OBipng==", 2024-04-24T05:47:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow *", 2024-04-24T05:47:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2024-04-24T05:47:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2024-04-24T05:47:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-mds", 2024-04-24T05:47:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQB/nChm4XGSFhAAeH0e/GDfieQEUu2dpPSg8Q==", 2024-04-24T05:47:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-mds" 2024-04-24T05:47:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-mgr", 2024-04-24T05:47:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQB/nChmSqGSFhAAyqe1x+17tdcLGAGZZLhy8w==", 2024-04-24T05:47:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-24T05:47:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-osd", 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQB/nChmlcuSFhAAVzb3AH30MqHbX+7MFUtZAg==", 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-osd" 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-rbd", 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQB/nChmYPWSFhAA3wusfHQijEQ/Zm/qMsuawQ==", 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-24T05:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQB/nChmQymTFhAACTnGRST/wypelWEbZsHXeA==", 2024-04-24T05:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-24T05:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-rgw", 2024-04-24T05:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQB/nChmhlGTFhAADow+o2ZIv0fRYchUfQP/nw==", 2024-04-24T05:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-24T05:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mgr.x", 2024-04-24T05:47:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBYnChmBMpBAhAAMObhb6ZHg0Zck7O/3zNiew==", 2024-04-24T05:47:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow *", 2024-04-24T05:47:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile mgr", 2024-04-24T05:47:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mgr.y", 2024-04-24T05:47:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBXnChme+AYMxAAbFi71Rqyc9RBMexN54Eumg==", 2024-04-24T05:47:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2024-04-24T05:47:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow *", 2024-04-24T05:47:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile mgr", 2024-04-24T05:47:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2024-04-24T05:47:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-24T05:47:35.940 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-24T05:47:35.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.924+0000 7f6b3b600640 1 -- 172.21.15.123:0/947261601 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6b1406c8f0 msgr2=0x7f6b1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:35.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.924+0000 7f6b3b600640 1 --2- 172.21.15.123:0/947261601 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6b1406c8f0 0x7f6b1406edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6b28034000 tx=0x7f6b2802e000 comp rx=0 tx=0).stop 2024-04-24T05:47:35.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.924+0000 7f6b3b600640 1 -- 172.21.15.123:0/947261601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3415a2a0 msgr2=0x7f6b340cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:35.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.924+0000 7f6b3b600640 1 --2- 172.21.15.123:0/947261601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3415a2a0 0x7f6b340cd440 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00c7b0 tx=0x7f6b2c00cc70 comp rx=0 tx=0).stop 2024-04-24T05:47:35.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.924+0000 7f6b3b600640 1 -- 172.21.15.123:0/947261601 shutdown_connections 2024-04-24T05:47:35.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.924+0000 7f6b3b600640 1 --2- 172.21.15.123:0/947261601 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f6b1406c8f0 0x7f6b1406edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:35.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.924+0000 7f6b3b600640 1 --2- 172.21.15.123:0/947261601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3415a2a0 0x7f6b340cd440 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:35.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.924+0000 7f6b3b600640 1 --2- 172.21.15.123:0/947261601 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b340af950 0x7f6b340ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:35.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.924+0000 7f6b3b600640 1 --2- 172.21.15.123:0/947261601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b340ad690 0x7f6b340cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:35.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.924+0000 7f6b3b600640 1 -- 172.21.15.123:0/947261601 >> 172.21.15.123:0/947261601 conn(0x7f6b340b1e70 msgr2=0x7f6b3415dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:35.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.924+0000 7f6b3b600640 1 -- 172.21.15.123:0/947261601 shutdown_connections 2024-04-24T05:47:35.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:35.924+0000 7f6b3b600640 1 -- 172.21.15.123:0/947261601 wait complete. 2024-04-24T05:47:35.945 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-24T05:47:36.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.138+0000 7f043de00640 1 Processor -- start 2024-04-24T05:47:36.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.138+0000 7f043de00640 1 -- start start 2024-04-24T05:47:36.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.139+0000 7f043de00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f043815b5d0 0x7f04380cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:36.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.139+0000 7f043de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f043815f030 0x7f043815f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:36.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.139+0000 7f043de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f043815f9d0 0x7f043815b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:36.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.139+0000 7f043de00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f04380af850 con 0x7f043815f9d0 2024-04-24T05:47:36.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.139+0000 7f043de00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f043815fe60 con 0x7f043815b5d0 2024-04-24T05:47:36.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.139+0000 7f043de00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f04380aec40 con 0x7f043815f030 2024-04-24T05:47:36.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.139+0000 7f0437400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f043815f9d0 0x7f043815b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:36.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.139+0000 7f0437400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f043815f9d0 0x7f043815b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45180/0 (socket says 172.21.15.123:45180) 2024-04-24T05:47:36.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.139+0000 7f0437400640 1 -- 172.21.15.123:0/2147270545 learned_addr learned my addr 172.21.15.123:0/2147270545 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:36.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.140+0000 7f0436a00640 1 --2- 172.21.15.123:0/2147270545 >> v2:172.21.15.155:3300/0 conn(0x7f043815b5d0 0x7f04380cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:36.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.140+0000 7f0437400640 1 -- 172.21.15.123:0/2147270545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f043815f030 msgr2=0x7f043815f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:36.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.140+0000 7f0437400640 1 --2- 172.21.15.123:0/2147270545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f043815f030 0x7f043815f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.140+0000 7f0437400640 1 -- 172.21.15.123:0/2147270545 >> v2:172.21.15.155:3300/0 conn(0x7f043815b5d0 msgr2=0x7f04380cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:36.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.140+0000 7f0437400640 1 --2- 172.21.15.123:0/2147270545 >> v2:172.21.15.155:3300/0 conn(0x7f043815b5d0 0x7f04380cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.140+0000 7f0437400640 1 -- 172.21.15.123:0/2147270545 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04380aef20 con 0x7f043815f9d0 2024-04-24T05:47:36.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.140+0000 7f0436a00640 1 --2- 172.21.15.123:0/2147270545 >> v2:172.21.15.155:3300/0 conn(0x7f043815b5d0 0x7f04380cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:36.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.141+0000 7f0437400640 1 --2- 172.21.15.123:0/2147270545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f043815f9d0 0x7f043815b9b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f04380685b0 tx=0x7f043002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be68cfb32c97b4e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:36.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.141+0000 7f0435600640 1 -- 172.21.15.123:0/2147270545 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430037070 con 0x7f043815f9d0 2024-04-24T05:47:36.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.141+0000 7f0435600640 1 -- 172.21.15.123:0/2147270545 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0430032440 con 0x7f043815f9d0 2024-04-24T05:47:36.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.141+0000 7f0435600640 1 -- 172.21.15.123:0/2147270545 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430032810 con 0x7f043815f9d0 2024-04-24T05:47:36.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.142+0000 7f043de00640 1 -- 172.21.15.123:0/2147270545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f043815f9d0 msgr2=0x7f043815b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:36.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.142+0000 7f043de00640 1 --2- 172.21.15.123:0/2147270545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f043815f9d0 0x7f043815b9b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f04380685b0 tx=0x7f043002a140 comp rx=0 tx=0).stop 2024-04-24T05:47:36.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.142+0000 7f043de00640 1 -- 172.21.15.123:0/2147270545 shutdown_connections 2024-04-24T05:47:36.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.142+0000 7f043de00640 1 --2- 172.21.15.123:0/2147270545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f043815f9d0 0x7f043815b9b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.142+0000 7f043de00640 1 --2- 172.21.15.123:0/2147270545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f043815f030 0x7f043815f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.142+0000 7f043de00640 1 --2- 172.21.15.123:0/2147270545 >> v2:172.21.15.155:3300/0 conn(0x7f043815b5d0 0x7f04380cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.142+0000 7f043de00640 1 -- 172.21.15.123:0/2147270545 >> 172.21.15.123:0/2147270545 conn(0x7f04380b1e70 msgr2=0x7f04380b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:36.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.142+0000 7f043de00640 1 -- 172.21.15.123:0/2147270545 shutdown_connections 2024-04-24T05:47:36.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.142+0000 7f043de00640 1 -- 172.21.15.123:0/2147270545 wait complete. 2024-04-24T05:47:36.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.144+0000 7f043de00640 1 Processor -- start 2024-04-24T05:47:36.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.144+0000 7f043de00640 1 -- start start 2024-04-24T05:47:36.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.145+0000 7f043de00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f043815b5d0 0x7f04381f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:36.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.145+0000 7f043de00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f043815f030 0x7f0438073460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:36.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.145+0000 7f043de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f043815f9d0 0x7f043806dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:36.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.145+0000 7f043de00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f043815bef0 con 0x7f043815f9d0 2024-04-24T05:47:36.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.145+0000 7f043de00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04380af4e0 con 0x7f043815b5d0 2024-04-24T05:47:36.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.145+0000 7f043de00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f043815c070 con 0x7f043815f030 2024-04-24T05:47:36.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.145+0000 7f0437400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f043815f9d0 0x7f043806dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:36.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.145+0000 7f0437400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f043815f9d0 0x7f043806dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45190/0 (socket says 172.21.15.123:45190) 2024-04-24T05:47:36.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.145+0000 7f0437400640 1 -- 172.21.15.123:0/3137468600 learned_addr learned my addr 172.21.15.123:0/3137468600 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:36.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.146+0000 7f0436a00640 1 --2- 172.21.15.123:0/3137468600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f043815b5d0 0x7f04381f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:36.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.146+0000 7f0437400640 1 -- 172.21.15.123:0/3137468600 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f043815f030 msgr2=0x7f0438073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:36.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.146+0000 7f0437400640 1 --2- 172.21.15.123:0/3137468600 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f043815f030 0x7f0438073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.146+0000 7f0437400640 1 -- 172.21.15.123:0/3137468600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f043815b5d0 msgr2=0x7f04381f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:36.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.146+0000 7f0437400640 1 --2- 172.21.15.123:0/3137468600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f043815b5d0 0x7f04381f5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.146+0000 7f0437400640 1 -- 172.21.15.123:0/3137468600 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043814a330 con 0x7f043815f9d0 2024-04-24T05:47:36.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.146+0000 7f0437400640 1 --2- 172.21.15.123:0/3137468600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f043815f9d0 0x7f043806dc70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f04300090b0 tx=0x7f04300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:36.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.146+0000 7f042f400640 1 -- 172.21.15.123:0/3137468600 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430041020 con 0x7f043815f9d0 2024-04-24T05:47:36.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.146+0000 7f042f400640 1 -- 172.21.15.123:0/3137468600 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0430030760 con 0x7f043815f9d0 2024-04-24T05:47:36.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.146+0000 7f043de00640 1 -- 172.21.15.123:0/3137468600 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04381f4ca0 con 0x7f043815f9d0 2024-04-24T05:47:36.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.146+0000 7f042f400640 1 -- 172.21.15.123:0/3137468600 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430037040 con 0x7f043815f9d0 2024-04-24T05:47:36.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.146+0000 7f043de00640 1 -- 172.21.15.123:0/3137468600 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04381f5260 con 0x7f043815f9d0 2024-04-24T05:47:36.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.148+0000 7f043de00640 1 -- 172.21.15.123:0/3137468600 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0438068a20 con 0x7f043815f9d0 2024-04-24T05:47:36.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.148+0000 7f042f400640 1 -- 172.21.15.123:0/3137468600 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0430044020 con 0x7f043815f9d0 2024-04-24T05:47:36.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.149+0000 7f042f400640 1 --2- 172.21.15.123:0/3137468600 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f041806c8f0 0x7f041806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:36.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.150+0000 7f042f400640 1 -- 172.21.15.123:0/3137468600 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f04300ac330 con 0x7f043815f9d0 2024-04-24T05:47:36.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.150+0000 7f0436a00640 1 --2- 172.21.15.123:0/3137468600 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f041806c8f0 0x7f041806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:36.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.151+0000 7f0436a00640 1 --2- 172.21.15.123:0/3137468600 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f041806c8f0 0x7f041806edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0428008f80 tx=0x7f04280084d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:36.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.155+0000 7f042f400640 1 -- 172.21.15.123:0/3137468600 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f043004b030 con 0x7f043815f9d0 2024-04-24T05:47:36.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.438+0000 7f043de00640 1 -- 172.21.15.123:0/3137468600 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f043806bf10 con 0x7f043815f9d0 2024-04-24T05:47:36.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.447+0000 7f042f400640 1 -- 172.21.15.123:0/3137468600 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f043007a880 con 0x7f043815f9d0 2024-04-24T05:47:36.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.449+0000 7f043de00640 1 -- 172.21.15.123:0/3137468600 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f041806c8f0 msgr2=0x7f041806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:36.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.449+0000 7f043de00640 1 --2- 172.21.15.123:0/3137468600 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f041806c8f0 0x7f041806edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0428008f80 tx=0x7f04280084d0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.449+0000 7f043de00640 1 -- 172.21.15.123:0/3137468600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f043815f9d0 msgr2=0x7f043806dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:36.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.449+0000 7f043de00640 1 --2- 172.21.15.123:0/3137468600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f043815f9d0 0x7f043806dc70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f04300090b0 tx=0x7f04300046c0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.450+0000 7f043de00640 1 -- 172.21.15.123:0/3137468600 shutdown_connections 2024-04-24T05:47:36.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.450+0000 7f043de00640 1 --2- 172.21.15.123:0/3137468600 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f041806c8f0 0x7f041806edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.450+0000 7f043de00640 1 --2- 172.21.15.123:0/3137468600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f043815f9d0 0x7f043806dc70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.450+0000 7f043de00640 1 --2- 172.21.15.123:0/3137468600 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f043815f030 0x7f0438073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.450+0000 7f043de00640 1 --2- 172.21.15.123:0/3137468600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f043815b5d0 0x7f04381f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.450+0000 7f043de00640 1 -- 172.21.15.123:0/3137468600 >> 172.21.15.123:0/3137468600 conn(0x7f04380b1e70 msgr2=0x7f043814eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:36.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.450+0000 7f043de00640 1 -- 172.21.15.123:0/3137468600 shutdown_connections 2024-04-24T05:47:36.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.450+0000 7f043de00640 1 -- 172.21.15.123:0/3137468600 wait complete. 2024-04-24T05:47:36.470 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-24T05:47:36.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.665+0000 7ff619400640 1 Processor -- start 2024-04-24T05:47:36.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.666+0000 7ff619400640 1 -- start start 2024-04-24T05:47:36.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.667+0000 7ff619400640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff61406ca10 0x7ff6140cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:36.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.667+0000 7ff619400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff61406cdf0 0x7ff61415eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:36.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.667+0000 7ff619400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff61415f900 con 0x7ff61415f0b0 2024-04-24T05:47:36.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.667+0000 7ff619400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff6140cba10 con 0x7ff61406cdf0 2024-04-24T05:47:36.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.667+0000 7ff619400640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff6140cbcf0 con 0x7ff61406ca10 2024-04-24T05:47:36.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.667+0000 7ff60b400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff61406cdf0 0x7ff61415eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:36.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.667+0000 7ff618a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff61415f0b0 0x7ff6140ca670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33570/0 (socket says 172.21.15.123:33570) 2024-04-24T05:47:36.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.667+0000 7ff618a00640 1 -- 172.21.15.123:0/1270834380 learned_addr learned my addr 172.21.15.123:0/1270834380 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:36.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.668+0000 7ff613e00640 1 --2- 172.21.15.123:0/1270834380 >> v2:172.21.15.155:3300/0 conn(0x7ff61406ca10 0x7ff6140cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:36.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.668+0000 7ff60b400640 1 -- 172.21.15.123:0/1270834380 >> v2:172.21.15.155:3300/0 conn(0x7ff61406ca10 msgr2=0x7ff6140cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:36.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.668+0000 7ff60b400640 1 --2- 172.21.15.123:0/1270834380 >> v2:172.21.15.155:3300/0 conn(0x7ff61406ca10 0x7ff6140cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.668+0000 7ff60b400640 1 -- 172.21.15.123:0/1270834380 >> v1:172.21.15.123:6789/0 conn(0x7ff61415f0b0 legacy=0x7ff6140ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:36.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.668+0000 7ff60b400640 1 -- 172.21.15.123:0/1270834380 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6140cbfd0 con 0x7ff61406cdf0 2024-04-24T05:47:36.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.669+0000 7ff60b400640 1 --2- 172.21.15.123:0/1270834380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff61406cdf0 0x7ff61415eb70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff6140746f0 tx=0x7ff5fc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c4a01d9cc920e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:36.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.669+0000 7ff613400640 1 -- 172.21.15.123:0/1270834380 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc037070 con 0x7ff61406cdf0 2024-04-24T05:47:36.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.669+0000 7ff613400640 1 -- 172.21.15.123:0/1270834380 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5fc032440 con 0x7ff61406cdf0 2024-04-24T05:47:36.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.669+0000 7ff613400640 1 -- 172.21.15.123:0/1270834380 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc032810 con 0x7ff61406cdf0 2024-04-24T05:47:36.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.670+0000 7ff619400640 1 -- 172.21.15.123:0/1270834380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff61406cdf0 msgr2=0x7ff61415eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:36.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.670+0000 7ff619400640 1 --2- 172.21.15.123:0/1270834380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff61406cdf0 0x7ff61415eb70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff6140746f0 tx=0x7ff5fc02a1a0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.670+0000 7ff619400640 1 -- 172.21.15.123:0/1270834380 shutdown_connections 2024-04-24T05:47:36.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.670+0000 7ff619400640 1 --2- 172.21.15.123:0/1270834380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff61406cdf0 0x7ff61415eb70 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.670+0000 7ff619400640 1 --2- 172.21.15.123:0/1270834380 >> v2:172.21.15.155:3300/0 conn(0x7ff61406ca10 0x7ff6140cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.670+0000 7ff619400640 1 -- 172.21.15.123:0/1270834380 >> 172.21.15.123:0/1270834380 conn(0x7ff6140b1b10 msgr2=0x7ff6140b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:36.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.670+0000 7ff619400640 1 -- 172.21.15.123:0/1270834380 shutdown_connections 2024-04-24T05:47:36.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.670+0000 7ff619400640 1 -- 172.21.15.123:0/1270834380 wait complete. 2024-04-24T05:47:36.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.672+0000 7ff619400640 1 Processor -- start 2024-04-24T05:47:36.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.672+0000 7ff619400640 1 -- start start 2024-04-24T05:47:36.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.673+0000 7ff619400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61406ca10 0x7ff61415d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:36.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.673+0000 7ff619400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff61406cdf0 0x7ff61409fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:36.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.673+0000 7ff619400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff61415f0b0 0x7ff6140a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:36.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.673+0000 7ff619400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff61406d250 con 0x7ff61415f0b0 2024-04-24T05:47:36.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.673+0000 7ff619400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff61415fbe0 con 0x7ff61406ca10 2024-04-24T05:47:36.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.673+0000 7ff619400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff61415f540 con 0x7ff61406cdf0 2024-04-24T05:47:36.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.673+0000 7ff618a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff61415f0b0 0x7ff6140a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:36.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.673+0000 7ff618a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff61415f0b0 0x7ff6140a0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45200/0 (socket says 172.21.15.123:45200) 2024-04-24T05:47:36.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.673+0000 7ff618a00640 1 -- 172.21.15.123:0/1155648090 learned_addr learned my addr 172.21.15.123:0/1155648090 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:36.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.674+0000 7ff613e00640 1 --2- 172.21.15.123:0/1155648090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61406ca10 0x7ff61415d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:36.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.674+0000 7ff60b400640 1 --2- 172.21.15.123:0/1155648090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff61406cdf0 0x7ff61409fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:36.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.674+0000 7ff618a00640 1 -- 172.21.15.123:0/1155648090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff61406cdf0 msgr2=0x7ff61409fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:36.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.674+0000 7ff618a00640 1 --2- 172.21.15.123:0/1155648090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff61406cdf0 0x7ff61409fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.674+0000 7ff618a00640 1 -- 172.21.15.123:0/1155648090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61406ca10 msgr2=0x7ff61415d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:36.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.674+0000 7ff618a00640 1 --2- 172.21.15.123:0/1155648090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61406ca10 0x7ff61415d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.674+0000 7ff618a00640 1 -- 172.21.15.123:0/1155648090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6140c7290 con 0x7ff61415f0b0 2024-04-24T05:47:36.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.674+0000 7ff618a00640 1 --2- 172.21.15.123:0/1155648090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff61415f0b0 0x7ff6140a0370 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00cbc0 tx=0x7ff60c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:36.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.674+0000 7ff610c00640 1 -- 172.21.15.123:0/1155648090 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c017070 con 0x7ff61415f0b0 2024-04-24T05:47:36.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.674+0000 7ff610c00640 1 -- 172.21.15.123:0/1155648090 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff60c00ad40 con 0x7ff61415f0b0 2024-04-24T05:47:36.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.674+0000 7ff610c00640 1 -- 172.21.15.123:0/1155648090 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c012600 con 0x7ff61415f0b0 2024-04-24T05:47:36.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.674+0000 7ff619400640 1 -- 172.21.15.123:0/1155648090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6140c7570 con 0x7ff61415f0b0 2024-04-24T05:47:36.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.675+0000 7ff619400640 1 -- 172.21.15.123:0/1155648090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6140c7b30 con 0x7ff61415f0b0 2024-04-24T05:47:36.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.676+0000 7ff619400640 1 -- 172.21.15.123:0/1155648090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff61409de50 con 0x7ff61415f0b0 2024-04-24T05:47:36.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.677+0000 7ff610c00640 1 -- 172.21.15.123:0/1155648090 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff60c01a070 con 0x7ff61415f0b0 2024-04-24T05:47:36.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.678+0000 7ff610c00640 1 --2- 172.21.15.123:0/1155648090 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5f006c8f0 0x7ff5f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:36.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.678+0000 7ff610c00640 1 -- 172.21.15.123:0/1155648090 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7ff60c08b700 con 0x7ff61415f0b0 2024-04-24T05:47:36.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.678+0000 7ff613e00640 1 --2- 172.21.15.123:0/1155648090 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5f006c8f0 0x7ff5f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:36.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.679+0000 7ff613e00640 1 --2- 172.21.15.123:0/1155648090 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5f006c8f0 0x7ff5f006edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff60400b7c0 tx=0x7ff604008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:36.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.683+0000 7ff610c00640 1 -- 172.21.15.123:0/1155648090 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff61409de50 con 0x7ff61415f0b0 2024-04-24T05:47:36.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.953+0000 7ff619400640 1 -- 172.21.15.123:0/1155648090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7ff614075b70 con 0x7ff61415f0b0 2024-04-24T05:47:36.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.956+0000 7ff610c00640 1 -- 172.21.15.123:0/1155648090 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7ff60c059600 con 0x7ff61415f0b0 2024-04-24T05:47:36.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.958+0000 7ff619400640 1 -- 172.21.15.123:0/1155648090 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5f006c8f0 msgr2=0x7ff5f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:36.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.958+0000 7ff619400640 1 --2- 172.21.15.123:0/1155648090 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5f006c8f0 0x7ff5f006edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff60400b7c0 tx=0x7ff604008040 comp rx=0 tx=0).stop 2024-04-24T05:47:36.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.958+0000 7ff619400640 1 -- 172.21.15.123:0/1155648090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff61415f0b0 msgr2=0x7ff6140a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:36.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.958+0000 7ff619400640 1 --2- 172.21.15.123:0/1155648090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff61415f0b0 0x7ff6140a0370 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00cbc0 tx=0x7ff60c00a430 comp rx=0 tx=0).stop 2024-04-24T05:47:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.958+0000 7ff619400640 1 -- 172.21.15.123:0/1155648090 shutdown_connections 2024-04-24T05:47:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.958+0000 7ff619400640 1 --2- 172.21.15.123:0/1155648090 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff5f006c8f0 0x7ff5f006edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.958+0000 7ff619400640 1 --2- 172.21.15.123:0/1155648090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff61415f0b0 0x7ff6140a0370 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.958+0000 7ff619400640 1 --2- 172.21.15.123:0/1155648090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff61406cdf0 0x7ff61409fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.958+0000 7ff619400640 1 --2- 172.21.15.123:0/1155648090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61406ca10 0x7ff61415d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.958+0000 7ff619400640 1 -- 172.21.15.123:0/1155648090 >> 172.21.15.123:0/1155648090 conn(0x7ff6140b1b10 msgr2=0x7ff6140afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.959+0000 7ff619400640 1 -- 172.21.15.123:0/1155648090 shutdown_connections 2024-04-24T05:47:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:36.959+0000 7ff619400640 1 -- 172.21.15.123:0/1155648090 wait complete. 2024-04-24T05:47:36.976 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-24T05:47:37.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.173+0000 7fe98ac00640 1 Processor -- start 2024-04-24T05:47:37.174 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.174+0000 7fe98ac00640 1 -- start start 2024-04-24T05:47:37.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.175+0000 7fe98ac00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe9840b56e0 0x7fe9840b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:37.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.175+0000 7fe98ac00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe9841622f0 0x7fe984162780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:37.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.175+0000 7fe98ac00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9840ce970 con 0x7fe984161d90 2024-04-24T05:47:37.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.175+0000 7fe98ac00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe9840b5b70 con 0x7fe9841622f0 2024-04-24T05:47:37.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.175+0000 7fe98ac00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe9840b3d40 con 0x7fe9840b56e0 2024-04-24T05:47:37.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.176+0000 7fe98a200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe9841622f0 0x7fe984162780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:37.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.176+0000 7fe98a200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe9841622f0 0x7fe984162780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45214/0 (socket says 172.21.15.123:45214) 2024-04-24T05:47:37.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.176+0000 7fe98a200640 1 -- 172.21.15.123:0/2190702510 learned_addr learned my addr 172.21.15.123:0/2190702510 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:37.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.176+0000 7fe988e00640 1 --2- 172.21.15.123:0/2190702510 >> v2:172.21.15.155:3300/0 conn(0x7fe9840b56e0 0x7fe9840b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:37.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.176+0000 7fe98a200640 1 -- 172.21.15.123:0/2190702510 >> v2:172.21.15.155:3300/0 conn(0x7fe9840b56e0 msgr2=0x7fe9840b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:37.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.176+0000 7fe98a200640 1 --2- 172.21.15.123:0/2190702510 >> v2:172.21.15.155:3300/0 conn(0x7fe9840b56e0 0x7fe9840b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.176+0000 7fe98a200640 1 -- 172.21.15.123:0/2190702510 >> v1:172.21.15.155:6789/0 conn(0x7fe984161d90 legacy=0x7fe98406e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:37.177 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.176+0000 7fe98a200640 1 -- 172.21.15.123:0/2190702510 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9840cec50 con 0x7fe9841622f0 2024-04-24T05:47:37.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.177+0000 7fe98a200640 1 --2- 172.21.15.123:0/2190702510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9841622f0 0x7fe984162780 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009e50 tx=0x7fe97c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94c110d3cba45c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:37.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.177+0000 7fe983e00640 1 -- 172.21.15.123:0/2190702510 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c036070 con 0x7fe9841622f0 2024-04-24T05:47:37.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.178+0000 7fe983e00640 1 -- 172.21.15.123:0/2190702510 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe97c031600 con 0x7fe9841622f0 2024-04-24T05:47:37.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.178+0000 7fe983e00640 1 -- 172.21.15.123:0/2190702510 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c0319d0 con 0x7fe9841622f0 2024-04-24T05:47:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.178+0000 7fe98ac00640 1 -- 172.21.15.123:0/2190702510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9841622f0 msgr2=0x7fe984162780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.178+0000 7fe98ac00640 1 --2- 172.21.15.123:0/2190702510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9841622f0 0x7fe984162780 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009e50 tx=0x7fe97c02a710 comp rx=0 tx=0).stop 2024-04-24T05:47:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.178+0000 7fe98ac00640 1 -- 172.21.15.123:0/2190702510 shutdown_connections 2024-04-24T05:47:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.178+0000 7fe98ac00640 1 --2- 172.21.15.123:0/2190702510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9841622f0 0x7fe984162780 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.178+0000 7fe98ac00640 1 --2- 172.21.15.123:0/2190702510 >> v2:172.21.15.155:3300/0 conn(0x7fe9840b56e0 0x7fe9840b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.178+0000 7fe98ac00640 1 -- 172.21.15.123:0/2190702510 >> 172.21.15.123:0/2190702510 conn(0x7fe9840cd370 msgr2=0x7fe9840cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.178+0000 7fe98ac00640 1 -- 172.21.15.123:0/2190702510 shutdown_connections 2024-04-24T05:47:37.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.178+0000 7fe98ac00640 1 -- 172.21.15.123:0/2190702510 wait complete. 2024-04-24T05:47:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.180+0000 7fe98ac00640 1 Processor -- start 2024-04-24T05:47:37.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.180+0000 7fe98ac00640 1 -- start start 2024-04-24T05:47:37.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.181+0000 7fe98ac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9840b56e0 0x7fe9841f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:37.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.181+0000 7fe98ac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe984161d90 0x7fe98414e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:37.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.181+0000 7fe98ac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9841622f0 0x7fe98415d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:37.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.181+0000 7fe98ac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9840cf210 con 0x7fe984161d90 2024-04-24T05:47:37.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.181+0000 7fe98ac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9840b3f70 con 0x7fe9841622f0 2024-04-24T05:47:37.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.181+0000 7fe98ac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe9840cf730 con 0x7fe9840b56e0 2024-04-24T05:47:37.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.181+0000 7fe988e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe984161d90 0x7fe98414e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:37.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.181+0000 7fe988e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe984161d90 0x7fe98414e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45230/0 (socket says 172.21.15.123:45230) 2024-04-24T05:47:37.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.181+0000 7fe988e00640 1 -- 172.21.15.123:0/1219637937 learned_addr learned my addr 172.21.15.123:0/1219637937 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:37.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.182+0000 7fe988e00640 1 -- 172.21.15.123:0/1219637937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9840b56e0 msgr2=0x7fe9841f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:37.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.182+0000 7fe988e00640 1 --2- 172.21.15.123:0/1219637937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9840b56e0 0x7fe9841f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.182+0000 7fe988e00640 1 -- 172.21.15.123:0/1219637937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9841622f0 msgr2=0x7fe98415d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:37.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.182+0000 7fe98a200640 1 --2- 172.21.15.123:0/1219637937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9841622f0 0x7fe98415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:37.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.182+0000 7fe988e00640 1 --2- 172.21.15.123:0/1219637937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9841622f0 0x7fe98415d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.182+0000 7fe988e00640 1 -- 172.21.15.123:0/1219637937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98415e1b0 con 0x7fe984161d90 2024-04-24T05:47:37.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.182+0000 7fe988e00640 1 --2- 172.21.15.123:0/1219637937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe984161d90 0x7fe98414e460 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe97000d830 tx=0x7fe97000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:37.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.182+0000 7fe981600640 1 -- 172.21.15.123:0/1219637937 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97000b6c0 con 0x7fe984161d90 2024-04-24T05:47:37.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.182+0000 7fe981600640 1 -- 172.21.15.123:0/1219637937 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe970019070 con 0x7fe984161d90 2024-04-24T05:47:37.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.182+0000 7fe981600640 1 -- 172.21.15.123:0/1219637937 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97000ba80 con 0x7fe984161d90 2024-04-24T05:47:37.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.182+0000 7fe98ac00640 1 -- 172.21.15.123:0/1219637937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98415e490 con 0x7fe984161d90 2024-04-24T05:47:37.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.183+0000 7fe98ac00640 1 -- 172.21.15.123:0/1219637937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98415bf80 con 0x7fe984161d90 2024-04-24T05:47:37.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.184+0000 7fe98ac00640 1 -- 172.21.15.123:0/1219637937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe984075b70 con 0x7fe984161d90 2024-04-24T05:47:37.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.185+0000 7fe981600640 1 -- 172.21.15.123:0/1219637937 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe97001e460 con 0x7fe984161d90 2024-04-24T05:47:37.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.185+0000 7fe981600640 1 --2- 172.21.15.123:0/1219637937 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe96406c920 0x7fe96406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:37.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.186+0000 7fe981600640 1 -- 172.21.15.123:0/1219637937 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fe97008c130 con 0x7fe984161d90 2024-04-24T05:47:37.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.186+0000 7fe989800640 1 --2- 172.21.15.123:0/1219637937 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe96406c920 0x7fe96406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:37.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.186+0000 7fe989800640 1 --2- 172.21.15.123:0/1219637937 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe96406c920 0x7fe96406ede0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe978003cc0 tx=0x7fe9780079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:37.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.190+0000 7fe981600640 1 -- 172.21.15.123:0/1219637937 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe97005a680 con 0x7fe984161d90 2024-04-24T05:47:37.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.459+0000 7fe98ac00640 1 -- 172.21.15.123:0/1219637937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fe9841630d0 con 0x7fe984161d90 2024-04-24T05:47:37.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.463+0000 7fe981600640 1 -- 172.21.15.123:0/1219637937 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7fe970059f80 con 0x7fe984161d90 2024-04-24T05:47:37.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.465+0000 7fe98ac00640 1 -- 172.21.15.123:0/1219637937 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe96406c920 msgr2=0x7fe96406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:37.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.465+0000 7fe98ac00640 1 --2- 172.21.15.123:0/1219637937 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe96406c920 0x7fe96406ede0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe978003cc0 tx=0x7fe9780079b0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.465+0000 7fe98ac00640 1 -- 172.21.15.123:0/1219637937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe984161d90 msgr2=0x7fe98414e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:37.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.465+0000 7fe98ac00640 1 --2- 172.21.15.123:0/1219637937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe984161d90 0x7fe98414e460 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe97000d830 tx=0x7fe97000dcf0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.465+0000 7fe98ac00640 1 -- 172.21.15.123:0/1219637937 shutdown_connections 2024-04-24T05:47:37.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.465+0000 7fe98ac00640 1 --2- 172.21.15.123:0/1219637937 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe96406c920 0x7fe96406ede0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.465+0000 7fe98ac00640 1 --2- 172.21.15.123:0/1219637937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9841622f0 0x7fe98415d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.465+0000 7fe98ac00640 1 --2- 172.21.15.123:0/1219637937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe984161d90 0x7fe98414e460 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.465+0000 7fe98ac00640 1 --2- 172.21.15.123:0/1219637937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9840b56e0 0x7fe9841f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.465+0000 7fe98ac00640 1 -- 172.21.15.123:0/1219637937 >> 172.21.15.123:0/1219637937 conn(0x7fe9840cd370 msgr2=0x7fe98415a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:37.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.465+0000 7fe98ac00640 1 -- 172.21.15.123:0/1219637937 shutdown_connections 2024-04-24T05:47:37.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.465+0000 7fe98ac00640 1 -- 172.21.15.123:0/1219637937 wait complete. 2024-04-24T05:47:37.484 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-24T05:47:37.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.672+0000 7faeb4a00640 1 Processor -- start 2024-04-24T05:47:37.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.673+0000 7faeb4a00640 1 -- start start 2024-04-24T05:47:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.674+0000 7faeb4a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faeb006f760 0x7faeb015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.674+0000 7faeb4a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7faeb00afa00 0x7faeb00ad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.674+0000 7faeb4a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeb00b64b0 con 0x7faeb006fb40 2024-04-24T05:47:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.674+0000 7faeb4a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7faeb00b41d0 con 0x7faeb00afa00 2024-04-24T05:47:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.674+0000 7faeb4a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faeb00adf30 con 0x7faeb006f760 2024-04-24T05:47:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.675+0000 7faeaf200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7faeb00afa00 0x7faeb00ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.675+0000 7faeaf200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7faeb00afa00 0x7faeb00ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45238/0 (socket says 172.21.15.123:45238) 2024-04-24T05:47:37.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.675+0000 7faeaf200640 1 -- 172.21.15.123:0/1927595971 learned_addr learned my addr 172.21.15.123:0/1927595971 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:37.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.675+0000 7faeae800640 1 --2- 172.21.15.123:0/1927595971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faeb006f760 0x7faeb015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:37.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.675+0000 7faeaf200640 1 -- 172.21.15.123:0/1927595971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faeb006f760 msgr2=0x7faeb015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:37.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.675+0000 7faeaf200640 1 --2- 172.21.15.123:0/1927595971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faeb006f760 0x7faeb015eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.675+0000 7faeaf200640 1 -- 172.21.15.123:0/1927595971 >> v1:172.21.15.155:6789/0 conn(0x7faeb006fb40 legacy=0x7faeb00af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:37.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.675+0000 7faeaf200640 1 -- 172.21.15.123:0/1927595971 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb00ae210 con 0x7faeb00afa00 2024-04-24T05:47:37.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.676+0000 7faeaf200640 1 --2- 172.21.15.123:0/1927595971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb00afa00 0x7faeb00ad850 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7faea8009e50 tx=0x7faea802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405caf16086426fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:37.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.677+0000 7faead400640 1 -- 172.21.15.123:0/1927595971 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea8036070 con 0x7faeb00afa00 2024-04-24T05:47:37.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.677+0000 7faead400640 1 -- 172.21.15.123:0/1927595971 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea8031600 con 0x7faeb00afa00 2024-04-24T05:47:37.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.677+0000 7faead400640 1 -- 172.21.15.123:0/1927595971 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea80319d0 con 0x7faeb00afa00 2024-04-24T05:47:37.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.678+0000 7faeb4a00640 1 -- 172.21.15.123:0/1927595971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb00afa00 msgr2=0x7faeb00ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:37.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.678+0000 7faeb4a00640 1 --2- 172.21.15.123:0/1927595971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb00afa00 0x7faeb00ad850 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7faea8009e50 tx=0x7faea802a6c0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.678+0000 7faeb4a00640 1 -- 172.21.15.123:0/1927595971 shutdown_connections 2024-04-24T05:47:37.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.678+0000 7faeb4a00640 1 --2- 172.21.15.123:0/1927595971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb00afa00 0x7faeb00ad850 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.678+0000 7faeb4a00640 1 --2- 172.21.15.123:0/1927595971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faeb006f760 0x7faeb015eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.678+0000 7faeb4a00640 1 -- 172.21.15.123:0/1927595971 >> 172.21.15.123:0/1927595971 conn(0x7faeb00d11c0 msgr2=0x7faeb00d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:37.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.678+0000 7faeb4a00640 1 -- 172.21.15.123:0/1927595971 shutdown_connections 2024-04-24T05:47:37.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.678+0000 7faeb4a00640 1 -- 172.21.15.123:0/1927595971 wait complete. 2024-04-24T05:47:37.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.679+0000 7faeb4a00640 1 Processor -- start 2024-04-24T05:47:37.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.680+0000 7faeb4a00640 1 -- start start 2024-04-24T05:47:37.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.680+0000 7faeb4a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faeb006f760 0x7faeb014dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:37.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.680+0000 7faeb4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb006fb40 0x7faeb014ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:37.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.680+0000 7faeb4a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faeb00afa00 0x7faeb014f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:37.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.680+0000 7faeb4a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faeb0158470 con 0x7faeb006fb40 2024-04-24T05:47:37.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.680+0000 7faeb4a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faeb00ae7d0 con 0x7faeb00afa00 2024-04-24T05:47:37.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.680+0000 7faeb4a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faeb01585f0 con 0x7faeb006f760 2024-04-24T05:47:37.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.681+0000 7faeade00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb006fb40 0x7faeb014ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:37.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.681+0000 7faeade00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb006fb40 0x7faeb014ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45246/0 (socket says 172.21.15.123:45246) 2024-04-24T05:47:37.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.681+0000 7faeade00640 1 -- 172.21.15.123:0/848735129 learned_addr learned my addr 172.21.15.123:0/848735129 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:37.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.681+0000 7faeade00640 1 -- 172.21.15.123:0/848735129 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faeb006f760 msgr2=0x7faeb014dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:37.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.681+0000 7faeade00640 1 --2- 172.21.15.123:0/848735129 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faeb006f760 0x7faeb014dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.681+0000 7faeade00640 1 -- 172.21.15.123:0/848735129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faeb00afa00 msgr2=0x7faeb014f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:37.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.681+0000 7faeade00640 1 --2- 172.21.15.123:0/848735129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faeb00afa00 0x7faeb014f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.681+0000 7faeade00640 1 -- 172.21.15.123:0/848735129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb006dd00 con 0x7faeb006fb40 2024-04-24T05:47:37.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.681+0000 7faeade00640 1 --2- 172.21.15.123:0/848735129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb006fb40 0x7faeb014ef00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c460 tx=0x7fae9800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:37.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.682+0000 7faea6a00640 1 -- 172.21.15.123:0/848735129 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98016020 con 0x7faeb006fb40 2024-04-24T05:47:37.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.682+0000 7faea6a00640 1 -- 172.21.15.123:0/848735129 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae98012870 con 0x7faeb006fb40 2024-04-24T05:47:37.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.682+0000 7faea6a00640 1 -- 172.21.15.123:0/848735129 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98012c60 con 0x7faeb006fb40 2024-04-24T05:47:37.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.682+0000 7faeb4a00640 1 -- 172.21.15.123:0/848735129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb014fa70 con 0x7faeb006fb40 2024-04-24T05:47:37.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.682+0000 7faeb4a00640 1 -- 172.21.15.123:0/848735129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb0150030 con 0x7faeb006fb40 2024-04-24T05:47:37.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.683+0000 7faeb4a00640 1 -- 172.21.15.123:0/848735129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb0075b70 con 0x7faeb006fb40 2024-04-24T05:47:37.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.684+0000 7faea6a00640 1 -- 172.21.15.123:0/848735129 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae98019020 con 0x7faeb006fb40 2024-04-24T05:47:37.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.685+0000 7faea6a00640 1 --2- 172.21.15.123:0/848735129 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fae8c06c8f0 0x7fae8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:37.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.685+0000 7faea6a00640 1 -- 172.21.15.123:0/848735129 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fae9808b6f0 con 0x7faeb006fb40 2024-04-24T05:47:37.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.686+0000 7faeae800640 1 --2- 172.21.15.123:0/848735129 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fae8c06c8f0 0x7fae8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:37.686 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.686+0000 7faeae800640 1 --2- 172.21.15.123:0/848735129 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fae8c06c8f0 0x7fae8c06edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faea00098f0 tx=0x7faea0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:37.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.690+0000 7faea6a00640 1 -- 172.21.15.123:0/848735129 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae98020030 con 0x7faeb006fb40 2024-04-24T05:47:37.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.962+0000 7faeb4a00640 1 -- 172.21.15.123:0/848735129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7faeb00d1f00 con 0x7faeb006fb40 2024-04-24T05:47:37.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.966+0000 7faea6a00640 1 -- 172.21.15.123:0/848735129 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7fae98059cc0 con 0x7faeb006fb40 2024-04-24T05:47:37.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.969+0000 7faeb4a00640 1 -- 172.21.15.123:0/848735129 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fae8c06c8f0 msgr2=0x7fae8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:37.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.969+0000 7faeb4a00640 1 --2- 172.21.15.123:0/848735129 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fae8c06c8f0 0x7fae8c06edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faea00098f0 tx=0x7faea0008040 comp rx=0 tx=0).stop 2024-04-24T05:47:37.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.969+0000 7faeb4a00640 1 -- 172.21.15.123:0/848735129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb006fb40 msgr2=0x7faeb014ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:37.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.969+0000 7faeb4a00640 1 --2- 172.21.15.123:0/848735129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb006fb40 0x7faeb014ef00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c460 tx=0x7fae9800c920 comp rx=0 tx=0).stop 2024-04-24T05:47:37.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.969+0000 7faeb4a00640 1 -- 172.21.15.123:0/848735129 shutdown_connections 2024-04-24T05:47:37.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.969+0000 7faeb4a00640 1 --2- 172.21.15.123:0/848735129 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fae8c06c8f0 0x7fae8c06edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.969+0000 7faeb4a00640 1 --2- 172.21.15.123:0/848735129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faeb00afa00 0x7faeb014f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.969+0000 7faeb4a00640 1 --2- 172.21.15.123:0/848735129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb006fb40 0x7faeb014ef00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.969+0000 7faeb4a00640 1 --2- 172.21.15.123:0/848735129 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faeb006f760 0x7faeb014dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:37.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.969+0000 7faeb4a00640 1 -- 172.21.15.123:0/848735129 >> 172.21.15.123:0/848735129 conn(0x7faeb00d11c0 msgr2=0x7faeb01601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:37.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.969+0000 7faeb4a00640 1 -- 172.21.15.123:0/848735129 shutdown_connections 2024-04-24T05:47:37.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:37.970+0000 7faeb4a00640 1 -- 172.21.15.123:0/848735129 wait complete. 2024-04-24T05:47:37.988 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-24T05:47:38.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.179+0000 7f96bea00640 1 Processor -- start 2024-04-24T05:47:38.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.180+0000 7f96bea00640 1 -- start start 2024-04-24T05:47:38.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.181+0000 7f96bea00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f96b815f280 0x7f96b81695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:38.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.181+0000 7f96bea00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b815f900 con 0x7f96b80cc4f0 2024-04-24T05:47:38.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.181+0000 7f96bea00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b80cba10 con 0x7f96b806ca10 2024-04-24T05:47:38.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.181+0000 7f96bea00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f96b80cbcf0 con 0x7f96b815f280 2024-04-24T05:47:38.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.181+0000 7f96be000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f96b815f280 0x7f96b81695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:38.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.181+0000 7f96bcc00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f96b80cc4f0 0x7f96b815eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33572/0 (socket says 172.21.15.123:33572) 2024-04-24T05:47:38.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.181+0000 7f96bcc00640 1 -- 172.21.15.123:0/619493605 learned_addr learned my addr 172.21.15.123:0/619493605 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:38.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.182+0000 7f96be000640 1 -- 172.21.15.123:0/619493605 >> v1:172.21.15.155:6789/0 conn(0x7f96b806ca10 legacy=0x7f96b806cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:38.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.182+0000 7f96be000640 1 -- 172.21.15.123:0/619493605 >> v1:172.21.15.123:6789/0 conn(0x7f96b80cc4f0 legacy=0x7f96b815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:38.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.182+0000 7f96b7e00640 1 -- 172.21.15.123:0/619493605 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1114178175 0 0) 0x7f96b815f900 con 0x7f96b80cc4f0 2024-04-24T05:47:38.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.182+0000 7f96be000640 1 -- 172.21.15.123:0/619493605 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b80cbfd0 con 0x7f96b815f280 2024-04-24T05:47:38.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.183+0000 7f96be000640 1 --2- 172.21.15.123:0/619493605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b815f280 0x7f96b81695c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009d90 tx=0x7f96b002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55e9b67d6e1eb4b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:38.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.183+0000 7f96b7e00640 1 -- 172.21.15.123:0/619493605 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0007740 con 0x7f96b815f280 2024-04-24T05:47:38.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.183+0000 7f96b7e00640 1 -- 172.21.15.123:0/619493605 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b00078e0 con 0x7f96b815f280 2024-04-24T05:47:38.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.183+0000 7f96b7e00640 1 -- 172.21.15.123:0/619493605 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0007cb0 con 0x7f96b815f280 2024-04-24T05:47:38.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.184+0000 7f96bea00640 1 -- 172.21.15.123:0/619493605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b815f280 msgr2=0x7f96b81695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:38.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.184+0000 7f96bea00640 1 --2- 172.21.15.123:0/619493605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b815f280 0x7f96b81695c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009d90 tx=0x7f96b002a6a0 comp rx=0 tx=0).stop 2024-04-24T05:47:38.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.184+0000 7f96bea00640 1 -- 172.21.15.123:0/619493605 shutdown_connections 2024-04-24T05:47:38.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.184+0000 7f96bea00640 1 --2- 172.21.15.123:0/619493605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b815f280 0x7f96b81695c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:38.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.184+0000 7f96bea00640 1 -- 172.21.15.123:0/619493605 >> 172.21.15.123:0/619493605 conn(0x7f96b80b1b10 msgr2=0x7f96b80b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:38.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.184+0000 7f96bea00640 1 -- 172.21.15.123:0/619493605 shutdown_connections 2024-04-24T05:47:38.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.184+0000 7f96bea00640 1 -- 172.21.15.123:0/619493605 wait complete. 2024-04-24T05:47:38.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.186+0000 7f96bea00640 1 Processor -- start 2024-04-24T05:47:38.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.186+0000 7f96bea00640 1 -- start start 2024-04-24T05:47:38.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.187+0000 7f96bea00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b806ca10 0x7f96b80c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:38.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.187+0000 7f96bea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b80cc4f0 0x7f96b8155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:38.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.187+0000 7f96bea00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b815f280 0x7f96b81558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:38.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.187+0000 7f96bea00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96b8169b00 con 0x7f96b80cc4f0 2024-04-24T05:47:38.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.187+0000 7f96bea00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96b815fbe0 con 0x7f96b806ca10 2024-04-24T05:47:38.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.187+0000 7f96bea00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f96b8169c80 con 0x7f96b815f280 2024-04-24T05:47:38.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.187+0000 7f96bcc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b80cc4f0 0x7f96b8155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:38.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.187+0000 7f96bcc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b80cc4f0 0x7f96b8155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45266/0 (socket says 172.21.15.123:45266) 2024-04-24T05:47:38.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.187+0000 7f96bcc00640 1 -- 172.21.15.123:0/3205284708 learned_addr learned my addr 172.21.15.123:0/3205284708 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:38.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.187+0000 7f96be000640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b815f280 0x7f96b81558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:38.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.187+0000 7f96bd600640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b806ca10 0x7f96b80c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:38.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.188+0000 7f96bcc00640 1 -- 172.21.15.123:0/3205284708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b815f280 msgr2=0x7f96b81558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:38.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.188+0000 7f96bcc00640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b815f280 0x7f96b81558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:38.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.188+0000 7f96bcc00640 1 -- 172.21.15.123:0/3205284708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b806ca10 msgr2=0x7f96b80c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:38.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.188+0000 7f96bcc00640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b806ca10 0x7f96b80c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:38.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.188+0000 7f96bcc00640 1 -- 172.21.15.123:0/3205284708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b8161d60 con 0x7f96b80cc4f0 2024-04-24T05:47:38.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.188+0000 7f96bd600640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b806ca10 0x7f96b80c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:38.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.188+0000 7f96bcc00640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b80cc4f0 0x7f96b8155390 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c4d0 tx=0x7f96a400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:38.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.188+0000 7f96b5600640 1 -- 172.21.15.123:0/3205284708 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4018070 con 0x7f96b80cc4f0 2024-04-24T05:47:38.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.188+0000 7f96b5600640 1 -- 172.21.15.123:0/3205284708 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a4007500 con 0x7f96b80cc4f0 2024-04-24T05:47:38.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.188+0000 7f96b5600640 1 -- 172.21.15.123:0/3205284708 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4004a20 con 0x7f96b80cc4f0 2024-04-24T05:47:38.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.188+0000 7f96bea00640 1 -- 172.21.15.123:0/3205284708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b81610e0 con 0x7f96b80cc4f0 2024-04-24T05:47:38.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.188+0000 7f96bea00640 1 -- 172.21.15.123:0/3205284708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b81616a0 con 0x7f96b80cc4f0 2024-04-24T05:47:38.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.189+0000 7f96bea00640 1 -- 172.21.15.123:0/3205284708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b80cfbc0 con 0x7f96b80cc4f0 2024-04-24T05:47:38.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.190+0000 7f96b5600640 1 -- 172.21.15.123:0/3205284708 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96a4004bc0 con 0x7f96b80cc4f0 2024-04-24T05:47:38.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.191+0000 7f96b5600640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f969806c8f0 0x7f969806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:38.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.191+0000 7f96b5600640 1 -- 172.21.15.123:0/3205284708 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f96a408ac40 con 0x7f96b80cc4f0 2024-04-24T05:47:38.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.192+0000 7f96bd600640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f969806c8f0 0x7f969806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:38.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.192+0000 7f96bd600640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f969806c8f0 0x7f969806edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f96b81f8350 tx=0x7f96ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:38.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.196+0000 7f96b5600640 1 -- 172.21.15.123:0/3205284708 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96b80cfbc0 con 0x7f96b80cc4f0 2024-04-24T05:47:38.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.480+0000 7f96bea00640 1 -- 172.21.15.123:0/3205284708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f96b815f0a0 con 0x7f96b80cc4f0 2024-04-24T05:47:38.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.485+0000 7f96b5600640 1 -- 172.21.15.123:0/3205284708 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) ==== 97+0+0 (secure 0 0 0) 0x7f96b815f0a0 con 0x7f96b80cc4f0 2024-04-24T05:47:38.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.488+0000 7f96bea00640 1 -- 172.21.15.123:0/3205284708 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f969806c8f0 msgr2=0x7f969806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:38.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.488+0000 7f96bea00640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f969806c8f0 0x7f969806edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f96b81f8350 tx=0x7f96ac009040 comp rx=0 tx=0).stop 2024-04-24T05:47:38.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.488+0000 7f96bea00640 1 -- 172.21.15.123:0/3205284708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b80cc4f0 msgr2=0x7f96b8155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:38.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.488+0000 7f96bea00640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b80cc4f0 0x7f96b8155390 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c4d0 tx=0x7f96a400c990 comp rx=0 tx=0).stop 2024-04-24T05:47:38.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.488+0000 7f96bea00640 1 -- 172.21.15.123:0/3205284708 shutdown_connections 2024-04-24T05:47:38.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.488+0000 7f96bea00640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f969806c8f0 0x7f969806edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:38.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.488+0000 7f96bea00640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b815f280 0x7f96b81558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:38.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.488+0000 7f96bea00640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96b80cc4f0 0x7f96b8155390 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:38.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.488+0000 7f96bea00640 1 --2- 172.21.15.123:0/3205284708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b806ca10 0x7f96b80c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:38.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.488+0000 7f96bea00640 1 -- 172.21.15.123:0/3205284708 >> 172.21.15.123:0/3205284708 conn(0x7f96b80b1b10 msgr2=0x7f96b81560c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:38.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.488+0000 7f96bea00640 1 -- 172.21.15.123:0/3205284708 shutdown_connections 2024-04-24T05:47:38.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.488+0000 7f96bea00640 1 -- 172.21.15.123:0/3205284708 wait complete. 2024-04-24T05:47:38.509 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-24T05:47:38.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.707+0000 7f6328a00640 1 Processor -- start 2024-04-24T05:47:38.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.707+0000 7f6328a00640 1 -- start start 2024-04-24T05:47:38.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.708+0000 7f6328a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f63240ad150 0x7f632406cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:38.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.709+0000 7f6328a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f632406ba00 0x7f632415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:38.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.709+0000 7f6328a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f632406be90 con 0x7f63240ad530 2024-04-24T05:47:38.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.709+0000 7f6328a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f632406c170 con 0x7f63240ad150 2024-04-24T05:47:38.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.709+0000 7f6328a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f632406c450 con 0x7f632406ba00 2024-04-24T05:47:38.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.709+0000 7f6321e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f63240ad150 0x7f632406cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:38.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.709+0000 7f6321e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f63240ad150 0x7f632406cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45268/0 (socket says 172.21.15.123:45268) 2024-04-24T05:47:38.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.709+0000 7f6321e00640 1 -- 172.21.15.123:0/952245655 learned_addr learned my addr 172.21.15.123:0/952245655 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:38.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.709+0000 7f6321400640 1 --2- 172.21.15.123:0/952245655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f632406ba00 0x7f632415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:38.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.710+0000 7f6321e00640 1 -- 172.21.15.123:0/952245655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f632406ba00 msgr2=0x7f632415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:38.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.710+0000 7f6321e00640 1 --2- 172.21.15.123:0/952245655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f632406ba00 0x7f632415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:38.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.710+0000 7f6321e00640 1 -- 172.21.15.123:0/952245655 >> v1:172.21.15.155:6789/0 conn(0x7f63240ad530 legacy=0x7f632406dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:38.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.710+0000 7f6321e00640 1 -- 172.21.15.123:0/952245655 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632406c730 con 0x7f63240ad150 2024-04-24T05:47:38.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.710+0000 7f6321400640 1 --2- 172.21.15.123:0/952245655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f632406ba00 0x7f632415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:38.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.711+0000 7f6321e00640 1 --2- 172.21.15.123:0/952245655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63240ad150 0x7f632406cf30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6314009870 tx=0x7f631402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee4ca31419c97cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:38.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.711+0000 7f6320a00640 1 -- 172.21.15.123:0/952245655 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314037070 con 0x7f63240ad150 2024-04-24T05:47:38.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.711+0000 7f6320a00640 1 -- 172.21.15.123:0/952245655 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6314032440 con 0x7f63240ad150 2024-04-24T05:47:38.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.711+0000 7f6320a00640 1 -- 172.21.15.123:0/952245655 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314032810 con 0x7f63240ad150 2024-04-24T05:47:38.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.712+0000 7f6328a00640 1 -- 172.21.15.123:0/952245655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63240ad150 msgr2=0x7f632406cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:38.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.712+0000 7f6328a00640 1 --2- 172.21.15.123:0/952245655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63240ad150 0x7f632406cf30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6314009870 tx=0x7f631402a210 comp rx=0 tx=0).stop 2024-04-24T05:47:38.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.712+0000 7f6328a00640 1 -- 172.21.15.123:0/952245655 shutdown_connections 2024-04-24T05:47:38.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.712+0000 7f6328a00640 1 --2- 172.21.15.123:0/952245655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f632406ba00 0x7f632415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:38.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.712+0000 7f6328a00640 1 --2- 172.21.15.123:0/952245655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63240ad150 0x7f632406cf30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:38.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.712+0000 7f6328a00640 1 -- 172.21.15.123:0/952245655 >> 172.21.15.123:0/952245655 conn(0x7f63240b1e70 msgr2=0x7f63240b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:38.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.712+0000 7f6328a00640 1 -- 172.21.15.123:0/952245655 shutdown_connections 2024-04-24T05:47:38.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.712+0000 7f6328a00640 1 -- 172.21.15.123:0/952245655 wait complete. 2024-04-24T05:47:38.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.713+0000 7f6328a00640 1 Processor -- start 2024-04-24T05:47:38.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.714+0000 7f6328a00640 1 -- start start 2024-04-24T05:47:38.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.714+0000 7f6328a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63240ad150 0x7f63240c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:38.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.714+0000 7f6328a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63240ad530 0x7f63240c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:38.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.715+0000 7f6328a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63241522d0 0x7f63240c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:38.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.715+0000 7f6328a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63240af360 con 0x7f63240ad150 2024-04-24T05:47:38.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.715+0000 7f6328a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f63240ad9e0 con 0x7f63240ad530 2024-04-24T05:47:38.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.715+0000 7f6328a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f63241f0150 con 0x7f63241522d0 2024-04-24T05:47:38.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.715+0000 7f6321e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63240ad150 0x7f63240c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:38.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.715+0000 7f6321e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63240ad150 0x7f63240c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45270/0 (socket says 172.21.15.123:45270) 2024-04-24T05:47:38.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.715+0000 7f6321e00640 1 -- 172.21.15.123:0/2431437394 learned_addr learned my addr 172.21.15.123:0/2431437394 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:38.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.715+0000 7f6321e00640 1 -- 172.21.15.123:0/2431437394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63241522d0 msgr2=0x7f63240c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:38.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.715+0000 7f6321e00640 1 --2- 172.21.15.123:0/2431437394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63241522d0 0x7f63240c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:38.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.715+0000 7f6321400640 1 --2- 172.21.15.123:0/2431437394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63240ad530 0x7f63240c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:38.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.715+0000 7f6321e00640 1 -- 172.21.15.123:0/2431437394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63240ad530 msgr2=0x7f63240c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:38.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.715+0000 7f6321e00640 1 --2- 172.21.15.123:0/2431437394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63240ad530 0x7f63240c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:38.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.715+0000 7f6321e00640 1 -- 172.21.15.123:0/2431437394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632409de90 con 0x7f63240ad150 2024-04-24T05:47:38.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.716+0000 7f6321e00640 1 --2- 172.21.15.123:0/2431437394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63240ad150 0x7f63240c6b00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6314000c00 tx=0x7f63140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:38.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.716+0000 7f631a000640 1 -- 172.21.15.123:0/2431437394 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314037040 con 0x7f63240ad150 2024-04-24T05:47:38.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.716+0000 7f631a000640 1 -- 172.21.15.123:0/2431437394 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6314042070 con 0x7f63240ad150 2024-04-24T05:47:38.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.716+0000 7f631a000640 1 -- 172.21.15.123:0/2431437394 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314030bd0 con 0x7f63240ad150 2024-04-24T05:47:38.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.716+0000 7f6328a00640 1 -- 172.21.15.123:0/2431437394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6324071ea0 con 0x7f63240ad150 2024-04-24T05:47:38.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.716+0000 7f6328a00640 1 -- 172.21.15.123:0/2431437394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63240cede0 con 0x7f63240ad150 2024-04-24T05:47:38.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.717+0000 7f6328a00640 1 -- 172.21.15.123:0/2431437394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6324068a20 con 0x7f63240ad150 2024-04-24T05:47:38.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.718+0000 7f631a000640 1 -- 172.21.15.123:0/2431437394 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6314047460 con 0x7f63240ad150 2024-04-24T05:47:38.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.719+0000 7f631a000640 1 --2- 172.21.15.123:0/2431437394 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f630406c920 0x7f630406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:38.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.719+0000 7f631a000640 1 -- 172.21.15.123:0/2431437394 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f63140ac5d0 con 0x7f63240ad150 2024-04-24T05:47:38.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.719+0000 7f6321400640 1 --2- 172.21.15.123:0/2431437394 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f630406c920 0x7f630406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:38.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.720+0000 7f6321400640 1 --2- 172.21.15.123:0/2431437394 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f630406c920 0x7f630406ede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f63240cc290 tx=0x7f6310008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:38.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.724+0000 7f631a000640 1 -- 172.21.15.123:0/2431437394 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6314045360 con 0x7f63240ad150 2024-04-24T05:47:38.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:38.995+0000 7f6328a00640 1 -- 172.21.15.123:0/2431437394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f632409e2b0 con 0x7f63240ad150 2024-04-24T05:47:39.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.000+0000 7f631a000640 1 -- 172.21.15.123:0/2431437394 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) ==== 90+0+0 (secure 0 0 0) 0x7f631407ab20 con 0x7f63240ad150 2024-04-24T05:47:39.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.002+0000 7f6328a00640 1 -- 172.21.15.123:0/2431437394 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f630406c920 msgr2=0x7f630406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:39.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.002+0000 7f6328a00640 1 --2- 172.21.15.123:0/2431437394 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f630406c920 0x7f630406ede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f63240cc290 tx=0x7f6310008040 comp rx=0 tx=0).stop 2024-04-24T05:47:39.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.002+0000 7f6328a00640 1 -- 172.21.15.123:0/2431437394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63240ad150 msgr2=0x7f63240c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:39.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.002+0000 7f6328a00640 1 --2- 172.21.15.123:0/2431437394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63240ad150 0x7f63240c6b00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6314000c00 tx=0x7f63140046c0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.002+0000 7f6328a00640 1 -- 172.21.15.123:0/2431437394 shutdown_connections 2024-04-24T05:47:39.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.002+0000 7f6328a00640 1 --2- 172.21.15.123:0/2431437394 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f630406c920 0x7f630406ede0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.002+0000 7f6328a00640 1 --2- 172.21.15.123:0/2431437394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63241522d0 0x7f63240c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.002+0000 7f6328a00640 1 --2- 172.21.15.123:0/2431437394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63240ad530 0x7f63240c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.002+0000 7f6328a00640 1 --2- 172.21.15.123:0/2431437394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63240ad150 0x7f63240c6b00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.002+0000 7f6328a00640 1 -- 172.21.15.123:0/2431437394 >> 172.21.15.123:0/2431437394 conn(0x7f63240b1e70 msgr2=0x7f63240b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:39.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.003+0000 7f6328a00640 1 -- 172.21.15.123:0/2431437394 shutdown_connections 2024-04-24T05:47:39.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.003+0000 7f6328a00640 1 -- 172.21.15.123:0/2431437394 wait complete. 2024-04-24T05:47:39.022 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:39.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.223+0000 7fa632a00640 1 Processor -- start 2024-04-24T05:47:39.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.223+0000 7fa632a00640 1 -- start start 2024-04-24T05:47:39.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.225+0000 7fa632a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa62c15b5c0 0x7fa62c0cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:39.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.225+0000 7fa632a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa62c06dbe0 0x7fa62c06e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:39.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.225+0000 7fa632a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa62c0b3000 con 0x7fa62c0acf40 2024-04-24T05:47:39.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.225+0000 7fa632a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa62c06e580 con 0x7fa62c06dbe0 2024-04-24T05:47:39.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.225+0000 7fa632a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa62c0ce4b0 con 0x7fa62c15b5c0 2024-04-24T05:47:39.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.225+0000 7fa630c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa62c06dbe0 0x7fa62c06e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:39.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.225+0000 7fa630c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa62c06dbe0 0x7fa62c06e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38716/0 (socket says 172.21.15.123:38716) 2024-04-24T05:47:39.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.225+0000 7fa630c00640 1 -- 172.21.15.123:0/704951014 learned_addr learned my addr 172.21.15.123:0/704951014 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:39.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.225+0000 7fa632000640 1 --1- 172.21.15.123:0/704951014 >> v1:172.21.15.123:6789/0 conn(0x7fa62c0acf40 0x7fa62c0ad3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:51090/0 (socket says 172.21.15.123:51090) 2024-04-24T05:47:39.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.226+0000 7fa631600640 1 --2- 172.21.15.123:0/704951014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa62c15b5c0 0x7fa62c0cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:39.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.226+0000 7fa630c00640 1 -- 172.21.15.123:0/704951014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa62c15b5c0 msgr2=0x7fa62c0cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:39.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.226+0000 7fa630c00640 1 --2- 172.21.15.123:0/704951014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa62c15b5c0 0x7fa62c0cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.226+0000 7fa630c00640 1 -- 172.21.15.123:0/704951014 >> v1:172.21.15.123:6789/0 conn(0x7fa62c0acf40 legacy=0x7fa62c0ad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:39.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.226+0000 7fa630c00640 1 -- 172.21.15.123:0/704951014 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c06d0c0 con 0x7fa62c06dbe0 2024-04-24T05:47:39.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.227+0000 7fa630c00640 1 --2- 172.21.15.123:0/704951014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c06dbe0 0x7fa62c06e040 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa61800bb10 tx=0x7fa61802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211145f565d05f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:39.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.227+0000 7fa62be00640 1 -- 172.21.15.123:0/704951014 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618036020 con 0x7fa62c06dbe0 2024-04-24T05:47:39.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.227+0000 7fa62be00640 1 -- 172.21.15.123:0/704951014 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa618032820 con 0x7fa62c06dbe0 2024-04-24T05:47:39.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.227+0000 7fa62be00640 1 -- 172.21.15.123:0/704951014 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618032bf0 con 0x7fa62c06dbe0 2024-04-24T05:47:39.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.228+0000 7fa632a00640 1 -- 172.21.15.123:0/704951014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c06dbe0 msgr2=0x7fa62c06e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:39.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.228+0000 7fa632a00640 1 --2- 172.21.15.123:0/704951014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c06dbe0 0x7fa62c06e040 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa61800bb10 tx=0x7fa61802c920 comp rx=0 tx=0).stop 2024-04-24T05:47:39.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.228+0000 7fa632a00640 1 -- 172.21.15.123:0/704951014 shutdown_connections 2024-04-24T05:47:39.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.228+0000 7fa632a00640 1 --2- 172.21.15.123:0/704951014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c06dbe0 0x7fa62c06e040 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.228+0000 7fa632a00640 1 --2- 172.21.15.123:0/704951014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa62c15b5c0 0x7fa62c0cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.228+0000 7fa632a00640 1 -- 172.21.15.123:0/704951014 >> 172.21.15.123:0/704951014 conn(0x7fa62c0d0070 msgr2=0x7fa62c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:39.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.228+0000 7fa632a00640 1 -- 172.21.15.123:0/704951014 shutdown_connections 2024-04-24T05:47:39.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.228+0000 7fa632a00640 1 -- 172.21.15.123:0/704951014 wait complete. 2024-04-24T05:47:39.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.229+0000 7fa632a00640 1 Processor -- start 2024-04-24T05:47:39.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.230+0000 7fa632a00640 1 -- start start 2024-04-24T05:47:39.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.230+0000 7fa632a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa62c06dbe0 0x7fa62c09d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:39.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.230+0000 7fa632a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa62c0acf40 0x7fa62c073790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:39.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.230+0000 7fa632a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c15b5c0 0x7fa62c15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:39.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.231+0000 7fa632a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c06cd20 con 0x7fa62c15b5c0 2024-04-24T05:47:39.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.231+0000 7fa632a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c0ae510 con 0x7fa62c06dbe0 2024-04-24T05:47:39.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.231+0000 7fa632a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa62c06cea0 con 0x7fa62c0acf40 2024-04-24T05:47:39.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.231+0000 7fa632000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c15b5c0 0x7fa62c15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:39.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.231+0000 7fa632000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c15b5c0 0x7fa62c15dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38718/0 (socket says 172.21.15.123:38718) 2024-04-24T05:47:39.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.231+0000 7fa632000640 1 -- 172.21.15.123:0/1666335914 learned_addr learned my addr 172.21.15.123:0/1666335914 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:39.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.231+0000 7fa631600640 1 --2- 172.21.15.123:0/1666335914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa62c06dbe0 0x7fa62c09d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:39.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.231+0000 7fa630c00640 1 --2- 172.21.15.123:0/1666335914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa62c0acf40 0x7fa62c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:39.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.231+0000 7fa632000640 1 -- 172.21.15.123:0/1666335914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa62c0acf40 msgr2=0x7fa62c073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:39.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.231+0000 7fa632000640 1 --2- 172.21.15.123:0/1666335914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa62c0acf40 0x7fa62c073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.231+0000 7fa632000640 1 -- 172.21.15.123:0/1666335914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa62c06dbe0 msgr2=0x7fa62c09d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:39.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.231+0000 7fa632000640 1 --2- 172.21.15.123:0/1666335914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa62c06dbe0 0x7fa62c09d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.231+0000 7fa632000640 1 -- 172.21.15.123:0/1666335914 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c09d690 con 0x7fa62c15b5c0 2024-04-24T05:47:39.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.232+0000 7fa632000640 1 --2- 172.21.15.123:0/1666335914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c15b5c0 0x7fa62c15dcd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa62400c710 tx=0x7fa62400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:39.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.232+0000 7fa629600640 1 -- 172.21.15.123:0/1666335914 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624017070 con 0x7fa62c15b5c0 2024-04-24T05:47:39.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.232+0000 7fa629600640 1 -- 172.21.15.123:0/1666335914 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa62400ce20 con 0x7fa62c15b5c0 2024-04-24T05:47:39.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.232+0000 7fa629600640 1 -- 172.21.15.123:0/1666335914 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62400a860 con 0x7fa62c15b5c0 2024-04-24T05:47:39.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.232+0000 7fa632a00640 1 -- 172.21.15.123:0/1666335914 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62c072cb0 con 0x7fa62c15b5c0 2024-04-24T05:47:39.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.232+0000 7fa632a00640 1 -- 172.21.15.123:0/1666335914 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62c15e210 con 0x7fa62c15b5c0 2024-04-24T05:47:39.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.233+0000 7fa632a00640 1 -- 172.21.15.123:0/1666335914 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62c075b70 con 0x7fa62c15b5c0 2024-04-24T05:47:39.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.234+0000 7fa629600640 1 -- 172.21.15.123:0/1666335914 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa62401a070 con 0x7fa62c15b5c0 2024-04-24T05:47:39.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.235+0000 7fa629600640 1 --2- 172.21.15.123:0/1666335914 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa60c06c920 0x7fa60c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:39.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.235+0000 7fa629600640 1 -- 172.21.15.123:0/1666335914 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fa62408b080 con 0x7fa62c15b5c0 2024-04-24T05:47:39.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.235+0000 7fa631600640 1 --2- 172.21.15.123:0/1666335914 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa60c06c920 0x7fa60c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:39.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.236+0000 7fa631600640 1 --2- 172.21.15.123:0/1666335914 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa60c06c920 0x7fa60c06ede0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa620006fd0 tx=0x7fa620008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:39.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.240+0000 7fa629600640 1 -- 172.21.15.123:0/1666335914 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6240595d0 con 0x7fa62c15b5c0 2024-04-24T05:47:39.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.443+0000 7fa632a00640 1 -- 172.21.15.123:0/1666335914 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa62c06ef20 con 0x7fa62c15b5c0 2024-04-24T05:47:39.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.445+0000 7fa629600640 1 -- 172.21.15.123:0/1666335914 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13200 (secure 0 0 0) 0x7fa624058ed0 con 0x7fa62c15b5c0 2024-04-24T05:47:39.445 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:39.446 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":35,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:47:35.428935+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:39.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.447+0000 7fa632a00640 1 -- 172.21.15.123:0/1666335914 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa60c06c920 msgr2=0x7fa60c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:39.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.447+0000 7fa632a00640 1 --2- 172.21.15.123:0/1666335914 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa60c06c920 0x7fa60c06ede0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa620006fd0 tx=0x7fa620008040 comp rx=0 tx=0).stop 2024-04-24T05:47:39.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.447+0000 7fa632a00640 1 -- 172.21.15.123:0/1666335914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c15b5c0 msgr2=0x7fa62c15dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:39.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.447+0000 7fa632a00640 1 --2- 172.21.15.123:0/1666335914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c15b5c0 0x7fa62c15dcd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa62400c710 tx=0x7fa62400cbd0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.448+0000 7fa632a00640 1 -- 172.21.15.123:0/1666335914 shutdown_connections 2024-04-24T05:47:39.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.448+0000 7fa632a00640 1 --2- 172.21.15.123:0/1666335914 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa60c06c920 0x7fa60c06ede0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.448+0000 7fa632a00640 1 --2- 172.21.15.123:0/1666335914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c15b5c0 0x7fa62c15dcd0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.448+0000 7fa632a00640 1 --2- 172.21.15.123:0/1666335914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa62c0acf40 0x7fa62c073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.448+0000 7fa632a00640 1 --2- 172.21.15.123:0/1666335914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa62c06dbe0 0x7fa62c09d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.448+0000 7fa632a00640 1 -- 172.21.15.123:0/1666335914 >> 172.21.15.123:0/1666335914 conn(0x7fa62c0d0070 msgr2=0x7fa62c0ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:39.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.448+0000 7fa632a00640 1 -- 172.21.15.123:0/1666335914 shutdown_connections 2024-04-24T05:47:39.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.448+0000 7fa632a00640 1 -- 172.21.15.123:0/1666335914 wait complete. 2024-04-24T05:47:39.468 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T05:45:48.542834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T05:47:39.468 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T05:47:39.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.670+0000 7ff1a3200640 1 Processor -- start 2024-04-24T05:47:39.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.671+0000 7ff1a3200640 1 -- start start 2024-04-24T05:47:39.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.672+0000 7ff1a3200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff1a415c470 0x7ff1a40aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:39.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.672+0000 7ff1a3200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1a406dbc0 con 0x7ff1a415b980 2024-04-24T05:47:39.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.672+0000 7ff1a3200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1a406de30 con 0x7ff1a40af040 2024-04-24T05:47:39.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.672+0000 7ff1a3200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff1a4071b80 con 0x7ff1a415c470 2024-04-24T05:47:39.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.672+0000 7ff199400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff1a415c470 0x7ff1a40aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:39.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.673+0000 7ff199400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff1a415c470 0x7ff1a40aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38726/0 (socket says 172.21.15.123:38726) 2024-04-24T05:47:39.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.673+0000 7ff199400640 1 -- 172.21.15.123:0/3828562681 learned_addr learned my addr 172.21.15.123:0/3828562681 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:39.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.673+0000 7ff1a1e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff1a415b980 0x7ff1a415bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:51104/0 (socket says 172.21.15.123:51104) 2024-04-24T05:47:39.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.673+0000 7ff199400640 1 -- 172.21.15.123:0/3828562681 >> v1:172.21.15.155:6789/0 conn(0x7ff1a40af040 legacy=0x7ff1a4164f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:39.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.673+0000 7ff199400640 1 -- 172.21.15.123:0/3828562681 >> v1:172.21.15.123:6789/0 conn(0x7ff1a415b980 legacy=0x7ff1a415bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:39.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.673+0000 7ff199400640 1 -- 172.21.15.123:0/3828562681 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a406e110 con 0x7ff1a415c470 2024-04-24T05:47:39.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.673+0000 7ff1a1400640 1 -- 172.21.15.123:0/3828562681 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 875685460 0 0) 0x7ff1a406dbc0 con 0x7ff1a415b980 2024-04-24T05:47:39.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.674+0000 7ff199400640 1 --2- 172.21.15.123:0/3828562681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a415c470 0x7ff1a40aeb00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009870 tx=0x7ff18c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c85372689bb5598 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:39.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.674+0000 7ff1a1400640 1 -- 172.21.15.123:0/3828562681 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c037070 con 0x7ff1a415c470 2024-04-24T05:47:39.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.674+0000 7ff1a1400640 1 -- 172.21.15.123:0/3828562681 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff18c032440 con 0x7ff1a415c470 2024-04-24T05:47:39.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.674+0000 7ff1a1400640 1 -- 172.21.15.123:0/3828562681 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c032810 con 0x7ff1a415c470 2024-04-24T05:47:39.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.675+0000 7ff1a3200640 1 -- 172.21.15.123:0/3828562681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a415c470 msgr2=0x7ff1a40aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:39.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.675+0000 7ff1a3200640 1 --2- 172.21.15.123:0/3828562681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a415c470 0x7ff1a40aeb00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009870 tx=0x7ff18c02a210 comp rx=0 tx=0).stop 2024-04-24T05:47:39.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.675+0000 7ff1a3200640 1 -- 172.21.15.123:0/3828562681 shutdown_connections 2024-04-24T05:47:39.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.675+0000 7ff1a3200640 1 --2- 172.21.15.123:0/3828562681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a415c470 0x7ff1a40aeb00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.675+0000 7ff1a3200640 1 -- 172.21.15.123:0/3828562681 >> 172.21.15.123:0/3828562681 conn(0x7ff1a40b1cf0 msgr2=0x7ff1a40b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:39.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.675+0000 7ff1a3200640 1 -- 172.21.15.123:0/3828562681 shutdown_connections 2024-04-24T05:47:39.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.675+0000 7ff1a3200640 1 -- 172.21.15.123:0/3828562681 wait complete. 2024-04-24T05:47:39.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.677+0000 7ff1a3200640 1 Processor -- start 2024-04-24T05:47:39.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.677+0000 7ff1a3200640 1 -- start start 2024-04-24T05:47:39.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.678+0000 7ff1a3200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a40af040 0x7ff1a414f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:39.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.678+0000 7ff1a3200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1a415b980 0x7ff1a41578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:39.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.678+0000 7ff1a3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a415c470 0x7ff1a409a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:39.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.678+0000 7ff1a3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a40af4d0 con 0x7ff1a415c470 2024-04-24T05:47:39.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.678+0000 7ff1a3200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a40af830 con 0x7ff1a40af040 2024-04-24T05:47:39.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.678+0000 7ff1a3200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a406f920 con 0x7ff1a415b980 2024-04-24T05:47:39.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.678+0000 7ff1a2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a415c470 0x7ff1a409a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:39.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.678+0000 7ff1a2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a415c470 0x7ff1a409a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38738/0 (socket says 172.21.15.123:38738) 2024-04-24T05:47:39.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.678+0000 7ff1a2800640 1 -- 172.21.15.123:0/1309266188 learned_addr learned my addr 172.21.15.123:0/1309266188 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:39.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.679+0000 7ff1a1e00640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a40af040 0x7ff1a414f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:39.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.679+0000 7ff1a2800640 1 -- 172.21.15.123:0/1309266188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1a415b980 msgr2=0x7ff1a41578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:39.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.679+0000 7ff199400640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1a415b980 0x7ff1a41578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:39.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.679+0000 7ff1a2800640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1a415b980 0x7ff1a41578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.679+0000 7ff1a2800640 1 -- 172.21.15.123:0/1309266188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a40af040 msgr2=0x7ff1a414f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:39.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.679+0000 7ff1a2800640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a40af040 0x7ff1a414f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.679+0000 7ff1a2800640 1 -- 172.21.15.123:0/1309266188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a41582c0 con 0x7ff1a415c470 2024-04-24T05:47:39.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.679+0000 7ff1a1e00640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a40af040 0x7ff1a414f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:39.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.679+0000 7ff1a2800640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a415c470 0x7ff1a409a150 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00c780 tx=0x7ff19c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:39.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.679+0000 7ff19aa00640 1 -- 172.21.15.123:0/1309266188 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c004b10 con 0x7ff1a415c470 2024-04-24T05:47:39.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.679+0000 7ff19aa00640 1 -- 172.21.15.123:0/1309266188 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff19c004cb0 con 0x7ff1a415c470 2024-04-24T05:47:39.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.679+0000 7ff1a3200640 1 -- 172.21.15.123:0/1309266188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a41585a0 con 0x7ff1a415c470 2024-04-24T05:47:39.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.680+0000 7ff19aa00640 1 -- 172.21.15.123:0/1309266188 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c00aa90 con 0x7ff1a415c470 2024-04-24T05:47:39.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.680+0000 7ff1a3200640 1 -- 172.21.15.123:0/1309266188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a414f870 con 0x7ff1a415c470 2024-04-24T05:47:39.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.681+0000 7ff1a3200640 1 -- 172.21.15.123:0/1309266188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a4068a20 con 0x7ff1a415c470 2024-04-24T05:47:39.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.682+0000 7ff19aa00640 1 -- 172.21.15.123:0/1309266188 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff19c00ac30 con 0x7ff1a415c470 2024-04-24T05:47:39.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.683+0000 7ff19aa00640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff18006c8f0 0x7ff18006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:39.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.683+0000 7ff19aa00640 1 -- 172.21.15.123:0/1309266188 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7ff19c08bbc0 con 0x7ff1a415c470 2024-04-24T05:47:39.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.683+0000 7ff1a1e00640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff18006c8f0 0x7ff18006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:39.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.684+0000 7ff1a1e00640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff18006c8f0 0x7ff18006edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff194000f60 tx=0x7ff194009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:39.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.687+0000 7ff19aa00640 1 -- 172.21.15.123:0/1309266188 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff19c020070 con 0x7ff1a415c470 2024-04-24T05:47:39.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.884+0000 7ff1a3200640 1 -- 172.21.15.123:0/1309266188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff1a415b760 con 0x7ff1a415c470 2024-04-24T05:47:39.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.885+0000 7ff19aa00640 1 -- 172.21.15.123:0/1309266188 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7ff19c05a1a0 con 0x7ff1a415c470 2024-04-24T05:47:39.885 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-04-24T05:47:39.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.887+0000 7ff1a3200640 1 -- 172.21.15.123:0/1309266188 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff18006c8f0 msgr2=0x7ff18006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:39.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.887+0000 7ff1a3200640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff18006c8f0 0x7ff18006edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff194000f60 tx=0x7ff194009040 comp rx=0 tx=0).stop 2024-04-24T05:47:39.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.887+0000 7ff1a3200640 1 -- 172.21.15.123:0/1309266188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a415c470 msgr2=0x7ff1a409a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:39.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.887+0000 7ff1a3200640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a415c470 0x7ff1a409a150 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00c780 tx=0x7ff19c00cc40 comp rx=0 tx=0).stop 2024-04-24T05:47:39.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.888+0000 7ff1a3200640 1 -- 172.21.15.123:0/1309266188 shutdown_connections 2024-04-24T05:47:39.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.888+0000 7ff1a3200640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff18006c8f0 0x7ff18006edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.888+0000 7ff1a3200640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a415c470 0x7ff1a409a150 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.888+0000 7ff1a3200640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1a415b980 0x7ff1a41578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.888+0000 7ff1a3200640 1 --2- 172.21.15.123:0/1309266188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a40af040 0x7ff1a414f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:39.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.888+0000 7ff1a3200640 1 -- 172.21.15.123:0/1309266188 >> 172.21.15.123:0/1309266188 conn(0x7ff1a40b1cf0 msgr2=0x7ff1a40b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:39.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.888+0000 7ff1a3200640 1 -- 172.21.15.123:0/1309266188 shutdown_connections 2024-04-24T05:47:39.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:39.888+0000 7ff1a3200640 1 -- 172.21.15.123:0/1309266188 wait complete. 2024-04-24T05:47:39.909 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T05:47:40.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.102+0000 7ff753800640 1 Processor -- start 2024-04-24T05:47:40.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.102+0000 7ff753800640 1 -- start start 2024-04-24T05:47:40.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.104+0000 7ff753800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff74c0af950 0x7ff74c167580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.104+0000 7ff753800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff74c15cdf0 con 0x7ff74c0ad860 2024-04-24T05:47:40.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.104+0000 7ff753800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff74c15d0d0 con 0x7ff74c15a2a0 2024-04-24T05:47:40.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.104+0000 7ff753800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff74c15a680 con 0x7ff74c0af950 2024-04-24T05:47:40.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.104+0000 7ff74be00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff74c0ad860 0x7ff74c0af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:51112/0 (socket says 172.21.15.123:51112) 2024-04-24T05:47:40.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.104+0000 7ff74be00640 1 -- 172.21.15.123:0/25705913 learned_addr learned my addr 172.21.15.123:0/25705913 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:40.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.104+0000 7ff751800640 1 --2- 172.21.15.123:0/25705913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff74c0af950 0x7ff74c167580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:40.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 -- 172.21.15.123:0/25705913 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2365929389 0 0) 0x7ff74c15cdf0 con 0x7ff74c0ad860 2024-04-24T05:47:40.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 -- 172.21.15.123:0/25705913 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff734003520 con 0x7ff74c0ad860 2024-04-24T05:47:40.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 -- 172.21.15.123:0/25705913 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3094492302 0 0) 0x7ff734003520 con 0x7ff74c0ad860 2024-04-24T05:47:40.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 -- 172.21.15.123:0/25705913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff74c0af950 msgr2=0x7ff74c167580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:40.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 --2- 172.21.15.123:0/25705913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff74c0af950 0x7ff74c167580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 -- 172.21.15.123:0/25705913 >> v1:172.21.15.155:6789/0 conn(0x7ff74c15a2a0 legacy=0x7ff74c0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:40.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 -- 172.21.15.123:0/25705913 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c15d3b0 con 0x7ff74c0ad860 2024-04-24T05:47:40.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 -- 172.21.15.123:0/25705913 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7ff738001670 con 0x7ff74c0ad860 2024-04-24T05:47:40.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 -- 172.21.15.123:0/25705913 >> v1:172.21.15.123:6789/0 conn(0x7ff74c0ad860 legacy=0x7ff74c0af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:40.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 --2- 172.21.15.123:0/25705913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff74c0af950 0x7ff7340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 --2- 172.21.15.123:0/25705913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff734003f60 0x7ff734004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 --2- 172.21.15.123:0/25705913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff734004d40 0x7ff7340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 -- 172.21.15.123:0/25705913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c15a680 con 0x7ff734004d40 2024-04-24T05:47:40.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 -- 172.21.15.123:0/25705913 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff73400bb50 con 0x7ff734003f60 2024-04-24T05:47:40.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.105+0000 7ff74b400640 1 -- 172.21.15.123:0/25705913 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff73400bd50 con 0x7ff74c0af950 2024-04-24T05:47:40.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.106+0000 7ff751800640 1 --2- 172.21.15.123:0/25705913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff734004d40 0x7ff7340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:40.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.106+0000 7ff74be00640 1 --2- 172.21.15.123:0/25705913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff734003f60 0x7ff734004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:40.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.106+0000 7ff750e00640 1 --2- 172.21.15.123:0/25705913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff74c0af950 0x7ff7340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:40.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.106+0000 7ff751800640 1 -- 172.21.15.123:0/25705913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff74c0af950 msgr2=0x7ff7340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:40.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.106+0000 7ff751800640 1 --2- 172.21.15.123:0/25705913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff74c0af950 0x7ff7340039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.106+0000 7ff751800640 1 -- 172.21.15.123:0/25705913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff734003f60 msgr2=0x7ff734004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:40.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.106+0000 7ff751800640 1 --2- 172.21.15.123:0/25705913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff734003f60 0x7ff734004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.106+0000 7ff751800640 1 -- 172.21.15.123:0/25705913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff734003520 con 0x7ff734004d40 2024-04-24T05:47:40.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.106+0000 7ff74be00640 1 --2- 172.21.15.123:0/25705913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff734003f60 0x7ff734004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:40.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.106+0000 7ff750e00640 1 --2- 172.21.15.123:0/25705913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff74c0af950 0x7ff7340039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:40.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.107+0000 7ff751800640 1 --2- 172.21.15.123:0/25705913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff734004d40 0x7ff7340051c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff744009fc0 tx=0x7ff734005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9622b36ba7da86c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:40.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.107+0000 7ff74b400640 1 -- 172.21.15.123:0/25705913 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744009c10 con 0x7ff734004d40 2024-04-24T05:47:40.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.107+0000 7ff74b400640 1 -- 172.21.15.123:0/25705913 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff744036070 con 0x7ff734004d40 2024-04-24T05:47:40.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.107+0000 7ff74b400640 1 -- 172.21.15.123:0/25705913 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744031880 con 0x7ff734004d40 2024-04-24T05:47:40.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.108+0000 7ff753800640 1 -- 172.21.15.123:0/25705913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff734004d40 msgr2=0x7ff7340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:40.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.108+0000 7ff753800640 1 --2- 172.21.15.123:0/25705913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff734004d40 0x7ff7340051c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff744009fc0 tx=0x7ff734005830 comp rx=0 tx=0).stop 2024-04-24T05:47:40.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.108+0000 7ff751800640 1 -- 172.21.15.123:0/25705913 reap_dead start 2024-04-24T05:47:40.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.108+0000 7ff753800640 1 -- 172.21.15.123:0/25705913 shutdown_connections 2024-04-24T05:47:40.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.108+0000 7ff753800640 1 -- 172.21.15.123:0/25705913 >> 172.21.15.123:0/25705913 conn(0x7ff74c0b1e70 msgr2=0x7ff74c0b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:40.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.108+0000 7ff753800640 1 -- 172.21.15.123:0/25705913 shutdown_connections 2024-04-24T05:47:40.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.108+0000 7ff753800640 1 -- 172.21.15.123:0/25705913 wait complete. 2024-04-24T05:47:40.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.110+0000 7ff753800640 1 Processor -- start 2024-04-24T05:47:40.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.110+0000 7ff753800640 1 -- start start 2024-04-24T05:47:40.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.111+0000 7ff753800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff734004d40 0x7ff74c15e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.111+0000 7ff753800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff74c08bf70 0x7ff74c1f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.111+0000 7ff753800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff74c072300 0x7ff74c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.111+0000 7ff753800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c06cb10 con 0x7ff74c08bf70 2024-04-24T05:47:40.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.111+0000 7ff753800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c15da00 con 0x7ff734004d40 2024-04-24T05:47:40.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.111+0000 7ff753800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff74c06cc90 con 0x7ff74c072300 2024-04-24T05:47:40.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.111+0000 7ff74be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff74c08bf70 0x7ff74c1f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:40.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.111+0000 7ff74be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff74c08bf70 0x7ff74c1f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38760/0 (socket says 172.21.15.123:38760) 2024-04-24T05:47:40.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.111+0000 7ff74be00640 1 -- 172.21.15.123:0/3696679066 learned_addr learned my addr 172.21.15.123:0/3696679066 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:40.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.111+0000 7ff750e00640 1 --2- 172.21.15.123:0/3696679066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff734004d40 0x7ff74c15e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:40.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.111+0000 7ff751800640 1 --2- 172.21.15.123:0/3696679066 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff74c072300 0x7ff74c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:40.117 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.111+0000 7ff74be00640 1 -- 172.21.15.123:0/3696679066 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff74c072300 msgr2=0x7ff74c06ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:40.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.111+0000 7ff74be00640 1 --2- 172.21.15.123:0/3696679066 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff74c072300 0x7ff74c06ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.112+0000 7ff74be00640 1 -- 172.21.15.123:0/3696679066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff734004d40 msgr2=0x7ff74c15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:40.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.112+0000 7ff74be00640 1 --2- 172.21.15.123:0/3696679066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff734004d40 0x7ff74c15e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.112+0000 7ff74be00640 1 -- 172.21.15.123:0/3696679066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c08c3f0 con 0x7ff74c08bf70 2024-04-24T05:47:40.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.112+0000 7ff74be00640 1 --2- 172.21.15.123:0/3696679066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff74c08bf70 0x7ff74c1f3be0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff73800d8f0 tx=0x7ff73800ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:40.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.112+0000 7ff748c00640 1 -- 172.21.15.123:0/3696679066 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738019070 con 0x7ff74c08bf70 2024-04-24T05:47:40.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.112+0000 7ff748c00640 1 -- 172.21.15.123:0/3696679066 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7380074e0 con 0x7ff74c08bf70 2024-04-24T05:47:40.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.112+0000 7ff748c00640 1 -- 172.21.15.123:0/3696679066 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7380049e0 con 0x7ff74c08bf70 2024-04-24T05:47:40.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.112+0000 7ff753800640 1 -- 172.21.15.123:0/3696679066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74c0983a0 con 0x7ff74c08bf70 2024-04-24T05:47:40.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.112+0000 7ff753800640 1 -- 172.21.15.123:0/3696679066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74c1f4630 con 0x7ff74c08bf70 2024-04-24T05:47:40.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.113+0000 7ff753800640 1 -- 172.21.15.123:0/3696679066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff74c068a20 con 0x7ff74c08bf70 2024-04-24T05:47:40.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.114+0000 7ff748c00640 1 -- 172.21.15.123:0/3696679066 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff73801c070 con 0x7ff74c08bf70 2024-04-24T05:47:40.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.115+0000 7ff748c00640 1 --2- 172.21.15.123:0/3696679066 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff72c06c8f0 0x7ff72c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.115+0000 7ff748c00640 1 -- 172.21.15.123:0/3696679066 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7ff73808b560 con 0x7ff74c08bf70 2024-04-24T05:47:40.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.115+0000 7ff750e00640 1 --2- 172.21.15.123:0/3696679066 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff72c06c8f0 0x7ff72c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:40.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.116+0000 7ff750e00640 1 --2- 172.21.15.123:0/3696679066 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff72c06c8f0 0x7ff72c06edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff740001630 tx=0x7ff740009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:40.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.119+0000 7ff748c00640 1 -- 172.21.15.123:0/3696679066 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff74c08c3f0 con 0x7ff74c08bf70 2024-04-24T05:47:40.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.321+0000 7ff753800640 1 -- 172.21.15.123:0/3696679066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff74c15f6b0 con 0x7ff74c08bf70 2024-04-24T05:47:40.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.322+0000 7ff748c00640 1 -- 172.21.15.123:0/3696679066 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7ff738059b40 con 0x7ff74c08bf70 2024-04-24T05:47:40.322 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:40.323 INFO:teuthology.orchestra.run.smithi123.stdout:[ 2024-04-24T05:47:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-24T05:47:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: "poolnum": 1, 2024-04-24T05:47:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: "poolname": ".mgr" 2024-04-24T05:47:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:40.323 INFO:teuthology.orchestra.run.smithi123.stdout:] 2024-04-24T05:47:40.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.324+0000 7ff753800640 1 -- 172.21.15.123:0/3696679066 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff72c06c8f0 msgr2=0x7ff72c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:40.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.324+0000 7ff753800640 1 --2- 172.21.15.123:0/3696679066 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff72c06c8f0 0x7ff72c06edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff740001630 tx=0x7ff740009040 comp rx=0 tx=0).stop 2024-04-24T05:47:40.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.324+0000 7ff753800640 1 -- 172.21.15.123:0/3696679066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff74c08bf70 msgr2=0x7ff74c1f3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:40.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.324+0000 7ff753800640 1 --2- 172.21.15.123:0/3696679066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff74c08bf70 0x7ff74c1f3be0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff73800d8f0 tx=0x7ff73800ddb0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.324+0000 7ff753800640 1 -- 172.21.15.123:0/3696679066 shutdown_connections 2024-04-24T05:47:40.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.325+0000 7ff753800640 1 --2- 172.21.15.123:0/3696679066 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7ff72c06c8f0 0x7ff72c06edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.325+0000 7ff753800640 1 --2- 172.21.15.123:0/3696679066 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff74c072300 0x7ff74c06ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.325+0000 7ff753800640 1 --2- 172.21.15.123:0/3696679066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff74c08bf70 0x7ff74c1f3be0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.325+0000 7ff753800640 1 --2- 172.21.15.123:0/3696679066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff734004d40 0x7ff74c15e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.325+0000 7ff753800640 1 -- 172.21.15.123:0/3696679066 >> 172.21.15.123:0/3696679066 conn(0x7ff74c0b1e70 msgr2=0x7ff74c15a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.325+0000 7ff753800640 1 -- 172.21.15.123:0/3696679066 shutdown_connections 2024-04-24T05:47:40.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.325+0000 7ff753800640 1 -- 172.21.15.123:0/3696679066 wait complete. 2024-04-24T05:47:40.344 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T05:47:40.344 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T05:47:40.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.544+0000 7f288be00640 1 Processor -- start 2024-04-24T05:47:40.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.544+0000 7f288be00640 1 -- start start 2024-04-24T05:47:40.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.546+0000 7f288be00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f288c15ebd0 0x7f288c162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.546+0000 7f288be00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288c0b6cf0 con 0x7f288c0d04b0 2024-04-24T05:47:40.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.546+0000 7f288be00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288c0cee10 con 0x7f288c15e7f0 2024-04-24T05:47:40.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.546+0000 7f288be00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f288c163480 con 0x7f288c15ebd0 2024-04-24T05:47:40.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.546+0000 7f2883e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f288c0d04b0 0x7f288c166850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:51124/0 (socket says 172.21.15.123:51124) 2024-04-24T05:47:40.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.546+0000 7f2883e00640 1 -- 172.21.15.123:0/1723743635 learned_addr learned my addr 172.21.15.123:0/1723743635 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 -- 172.21.15.123:0/1723743635 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4180367432 0 0) 0x7f288c0b6cf0 con 0x7f288c0d04b0 2024-04-24T05:47:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 -- 172.21.15.123:0/1723743635 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2870003520 con 0x7f288c0d04b0 2024-04-24T05:47:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288b400640 1 --2- 172.21.15.123:0/1723743635 >> v2:172.21.15.155:3300/0 conn(0x7f288c15ebd0 0x7f288c162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 -- 172.21.15.123:0/1723743635 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3022211620 0 0) 0x7f2870003520 con 0x7f288c0d04b0 2024-04-24T05:47:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 -- 172.21.15.123:0/1723743635 >> v2:172.21.15.155:3300/0 conn(0x7f288c15ebd0 msgr2=0x7f288c162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 --2- 172.21.15.123:0/1723743635 >> v2:172.21.15.155:3300/0 conn(0x7f288c15ebd0 0x7f288c162f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 -- 172.21.15.123:0/1723743635 >> v1:172.21.15.155:6789/0 conn(0x7f288c15e7f0 legacy=0x7f288c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 -- 172.21.15.123:0/1723743635 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c0cf0f0 con 0x7f288c0d04b0 2024-04-24T05:47:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 -- 172.21.15.123:0/1723743635 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f2874001670 con 0x7f288c0d04b0 2024-04-24T05:47:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 -- 172.21.15.123:0/1723743635 >> v1:172.21.15.123:6789/0 conn(0x7f288c0d04b0 legacy=0x7f288c166850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 --2- 172.21.15.123:0/1723743635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28700039b0 0x7f2870003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 --2- 172.21.15.123:0/1723743635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2870004800 0x7f2870004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 --2- 172.21.15.123:0/1723743635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28700052e0 0x7f287000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 -- 172.21.15.123:0/1723743635 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f288c163480 con 0x7f2870004800 2024-04-24T05:47:40.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 -- 172.21.15.123:0/1723743635 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f287000c190 con 0x7f28700052e0 2024-04-24T05:47:40.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f288a000640 1 -- 172.21.15.123:0/1723743635 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f287000c390 con 0x7f28700039b0 2024-04-24T05:47:40.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.547+0000 7f2883e00640 1 --2- 172.21.15.123:0/1723743635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2870004800 0x7f2870004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:40.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.548+0000 7f2883e00640 1 -- 172.21.15.123:0/1723743635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28700039b0 msgr2=0x7f2870003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:40.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.548+0000 7f2883e00640 1 --2- 172.21.15.123:0/1723743635 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28700039b0 0x7f2870003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.548+0000 7f2883e00640 1 -- 172.21.15.123:0/1723743635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28700052e0 msgr2=0x7f287000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:40.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.548+0000 7f2883e00640 1 --2- 172.21.15.123:0/1723743635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28700052e0 0x7f287000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.548+0000 7f2883e00640 1 -- 172.21.15.123:0/1723743635 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2870003520 con 0x7f2870004800 2024-04-24T05:47:40.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.548+0000 7f288b400640 1 -- 172.21.15.123:0/1723743635 reap_dead start 2024-04-24T05:47:40.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.549+0000 7f2883e00640 1 --2- 172.21.15.123:0/1723743635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2870004800 0x7f2870004c80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2874007fd0 tx=0x7f2870005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994f59a1ebd037a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:40.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.549+0000 7f288a000640 1 -- 172.21.15.123:0/1723743635 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874007d50 con 0x7f2870004800 2024-04-24T05:47:40.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.549+0000 7f288a000640 1 -- 172.21.15.123:0/1723743635 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2874005af0 con 0x7f2870004800 2024-04-24T05:47:40.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.549+0000 7f288a000640 1 -- 172.21.15.123:0/1723743635 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874007070 con 0x7f2870004800 2024-04-24T05:47:40.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.550+0000 7f288be00640 1 -- 172.21.15.123:0/1723743635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2870004800 msgr2=0x7f2870004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:40.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.550+0000 7f288be00640 1 --2- 172.21.15.123:0/1723743635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2870004800 0x7f2870004c80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2874007fd0 tx=0x7f2870005830 comp rx=0 tx=0).stop 2024-04-24T05:47:40.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.550+0000 7f288be00640 1 -- 172.21.15.123:0/1723743635 shutdown_connections 2024-04-24T05:47:40.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.550+0000 7f288be00640 1 --2- 172.21.15.123:0/1723743635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2870004800 0x7f2870004c80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.550+0000 7f288be00640 1 -- 172.21.15.123:0/1723743635 >> 172.21.15.123:0/1723743635 conn(0x7f288c0b9530 msgr2=0x7f288c0b9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:40.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.550+0000 7f288be00640 1 -- 172.21.15.123:0/1723743635 shutdown_connections 2024-04-24T05:47:40.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.550+0000 7f288be00640 1 -- 172.21.15.123:0/1723743635 wait complete. 2024-04-24T05:47:40.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.552+0000 7f288be00640 1 Processor -- start 2024-04-24T05:47:40.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.552+0000 7f288be00640 1 -- start start 2024-04-24T05:47:40.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.553+0000 7f288be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2870004800 0x7f288c1f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.553+0000 7f288be00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f288c15a090 0x7f288c15bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.553+0000 7f288be00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f288c06c6b0 0x7f288c15c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.553+0000 7f288be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f288c0cf8f0 con 0x7f2870004800 2024-04-24T05:47:40.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.553+0000 7f288be00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f288c0cebc0 con 0x7f288c15a090 2024-04-24T05:47:40.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.553+0000 7f288be00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f288c0cfa70 con 0x7f288c06c6b0 2024-04-24T05:47:40.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.553+0000 7f288aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2870004800 0x7f288c1f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:40.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.553+0000 7f288aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2870004800 0x7f288c1f4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38778/0 (socket says 172.21.15.123:38778) 2024-04-24T05:47:40.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.553+0000 7f288aa00640 1 -- 172.21.15.123:0/273456804 learned_addr learned my addr 172.21.15.123:0/273456804 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:40.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.553+0000 7f288aa00640 1 -- 172.21.15.123:0/273456804 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f288c06c6b0 msgr2=0x7f288c15c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:40.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.553+0000 7f288aa00640 1 --2- 172.21.15.123:0/273456804 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f288c06c6b0 0x7f288c15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.553+0000 7f288aa00640 1 -- 172.21.15.123:0/273456804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f288c15a090 msgr2=0x7f288c15bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:40.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.553+0000 7f288aa00640 1 --2- 172.21.15.123:0/273456804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f288c15a090 0x7f288c15bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:40.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.553+0000 7f288aa00640 1 -- 172.21.15.123:0/273456804 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c160570 con 0x7f2870004800 2024-04-24T05:47:40.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.554+0000 7f288aa00640 1 --2- 172.21.15.123:0/273456804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2870004800 0x7f288c1f4e50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f287c00c3c0 tx=0x7f287c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:40.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.554+0000 7f2882a00640 1 -- 172.21.15.123:0/273456804 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c017070 con 0x7f2870004800 2024-04-24T05:47:40.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.554+0000 7f2882a00640 1 -- 172.21.15.123:0/273456804 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f287c0035f0 con 0x7f2870004800 2024-04-24T05:47:40.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.554+0000 7f2882a00640 1 -- 172.21.15.123:0/273456804 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c003a20 con 0x7f2870004800 2024-04-24T05:47:40.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.554+0000 7f288be00640 1 -- 172.21.15.123:0/273456804 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288c06cc00 con 0x7f2870004800 2024-04-24T05:47:40.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.554+0000 7f288be00640 1 -- 172.21.15.123:0/273456804 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f288c160840 con 0x7f2870004800 2024-04-24T05:47:40.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.555+0000 7f288be00640 1 -- 172.21.15.123:0/273456804 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f288c075b70 con 0x7f2870004800 2024-04-24T05:47:40.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.556+0000 7f2882a00640 1 -- 172.21.15.123:0/273456804 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f287c01a070 con 0x7f2870004800 2024-04-24T05:47:40.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.557+0000 7f2882a00640 1 --2- 172.21.15.123:0/273456804 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f286806c920 0x7f286806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:40.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.557+0000 7f2882a00640 1 -- 172.21.15.123:0/273456804 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f287c08ad70 con 0x7f2870004800 2024-04-24T05:47:40.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.557+0000 7f2883e00640 1 --2- 172.21.15.123:0/273456804 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f286806c920 0x7f286806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:40.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.558+0000 7f2883e00640 1 --2- 172.21.15.123:0/273456804 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f286806c920 0x7f286806ede0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2874033000 tx=0x7f2874009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:40.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.561+0000 7f2882a00640 1 -- 172.21.15.123:0/273456804 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f287c0592c0 con 0x7f2870004800 2024-04-24T05:47:40.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:40.746+0000 7f288be00640 1 -- 172.21.15.123:0/273456804 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f288c0ce9a0 con 0x7f2870004800 2024-04-24T05:47:41.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.474+0000 7f2882a00640 1 -- 172.21.15.123:0/273456804 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7f287c058bc0 con 0x7f2870004800 2024-04-24T05:47:41.475 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' created 2024-04-24T05:47:41.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.477+0000 7f288be00640 1 -- 172.21.15.123:0/273456804 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f286806c920 msgr2=0x7f286806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:41.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.477+0000 7f288be00640 1 --2- 172.21.15.123:0/273456804 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f286806c920 0x7f286806ede0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2874033000 tx=0x7f2874009030 comp rx=0 tx=0).stop 2024-04-24T05:47:41.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.477+0000 7f288be00640 1 -- 172.21.15.123:0/273456804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2870004800 msgr2=0x7f288c1f4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:41.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.477+0000 7f288be00640 1 --2- 172.21.15.123:0/273456804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2870004800 0x7f288c1f4e50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f287c00c3c0 tx=0x7f287c00c880 comp rx=0 tx=0).stop 2024-04-24T05:47:41.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.478+0000 7f288be00640 1 -- 172.21.15.123:0/273456804 shutdown_connections 2024-04-24T05:47:41.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.478+0000 7f288be00640 1 --2- 172.21.15.123:0/273456804 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f286806c920 0x7f286806ede0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:41.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.478+0000 7f288be00640 1 --2- 172.21.15.123:0/273456804 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f288c06c6b0 0x7f288c15c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:41.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.478+0000 7f288be00640 1 --2- 172.21.15.123:0/273456804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f288c15a090 0x7f288c15bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:41.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.478+0000 7f288be00640 1 --2- 172.21.15.123:0/273456804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2870004800 0x7f288c1f4e50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:41.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.478+0000 7f288be00640 1 -- 172.21.15.123:0/273456804 >> 172.21.15.123:0/273456804 conn(0x7f288c0b9530 msgr2=0x7f288c0cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:41.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.478+0000 7f288be00640 1 -- 172.21.15.123:0/273456804 shutdown_connections 2024-04-24T05:47:41.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.478+0000 7f288be00640 1 -- 172.21.15.123:0/273456804 wait complete. 2024-04-24T05:47:41.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T05:47:41.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.697+0000 7f1b85800640 1 Processor -- start 2024-04-24T05:47:41.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.697+0000 7f1b85800640 1 -- start start 2024-04-24T05:47:41.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.698+0000 7f1b85800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8015e970 0x7f1b800d44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:41.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.698+0000 7f1b85800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b800d3050 con 0x7f1b800b39b0 2024-04-24T05:47:41.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.699+0000 7f1b85800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b800cfdf0 con 0x7f1b800d3e10 2024-04-24T05:47:41.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.699+0000 7f1b85800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1b800d09c0 con 0x7f1b8015e970 2024-04-24T05:47:41.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.699+0000 7f1b7f400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1b800b39b0 0x7f1b800d2940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:51132/0 (socket says 172.21.15.123:51132) 2024-04-24T05:47:41.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.699+0000 7f1b7f400640 1 -- 172.21.15.123:0/2299217020 learned_addr learned my addr 172.21.15.123:0/2299217020 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:41.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.699+0000 7f1b7ea00640 1 --2- 172.21.15.123:0/2299217020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8015e970 0x7f1b800d44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:41.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 -- 172.21.15.123:0/2299217020 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 98015339 0 0) 0x7f1b800d3050 con 0x7f1b800b39b0 2024-04-24T05:47:41.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 -- 172.21.15.123:0/2299217020 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b60003520 con 0x7f1b800b39b0 2024-04-24T05:47:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 -- 172.21.15.123:0/2299217020 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 509735853 0 0) 0x7f1b60003520 con 0x7f1b800b39b0 2024-04-24T05:47:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 -- 172.21.15.123:0/2299217020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8015e970 msgr2=0x7f1b800d44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 --2- 172.21.15.123:0/2299217020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8015e970 0x7f1b800d44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 -- 172.21.15.123:0/2299217020 >> v1:172.21.15.155:6789/0 conn(0x7f1b800d3e10 legacy=0x7f1b800d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 -- 172.21.15.123:0/2299217020 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b800d1e60 con 0x7f1b800b39b0 2024-04-24T05:47:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 -- 172.21.15.123:0/2299217020 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f1b78001670 con 0x7f1b800b39b0 2024-04-24T05:47:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 -- 172.21.15.123:0/2299217020 >> v1:172.21.15.123:6789/0 conn(0x7f1b800b39b0 legacy=0x7f1b800d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 --2- 172.21.15.123:0/2299217020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b600039e0 0x7f1b60003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 --2- 172.21.15.123:0/2299217020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b60004800 0x7f1b60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 --2- 172.21.15.123:0/2299217020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8015e970 0x7f1b60005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 -- 172.21.15.123:0/2299217020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b800d09c0 con 0x7f1b60004800 2024-04-24T05:47:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 -- 172.21.15.123:0/2299217020 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6000baf0 con 0x7f1b600039e0 2024-04-24T05:47:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.700+0000 7f1b7d600640 1 -- 172.21.15.123:0/2299217020 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1b6000bcf0 con 0x7f1b8015e970 2024-04-24T05:47:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.701+0000 7f1b7e000640 1 --2- 172.21.15.123:0/2299217020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b60004800 0x7f1b60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.701+0000 7f1b7ea00640 1 --2- 172.21.15.123:0/2299217020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b600039e0 0x7f1b60003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.701+0000 7f1b7f400640 1 --2- 172.21.15.123:0/2299217020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8015e970 0x7f1b60005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.701+0000 7f1b7e000640 1 -- 172.21.15.123:0/2299217020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8015e970 msgr2=0x7f1b60005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.701+0000 7f1b7e000640 1 --2- 172.21.15.123:0/2299217020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8015e970 0x7f1b60005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.701+0000 7f1b7e000640 1 -- 172.21.15.123:0/2299217020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b600039e0 msgr2=0x7f1b60003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.701+0000 7f1b7e000640 1 --2- 172.21.15.123:0/2299217020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b600039e0 0x7f1b60003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.701+0000 7f1b7e000640 1 -- 172.21.15.123:0/2299217020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b60003520 con 0x7f1b60004800 2024-04-24T05:47:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.701+0000 7f1b7f400640 1 --2- 172.21.15.123:0/2299217020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8015e970 0x7f1b60005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T05:47:41.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.703+0000 7f1b7e000640 1 --2- 172.21.15.123:0/2299217020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b60004800 0x7f1b60004c80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1b68001e20 tx=0x7f1b60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219f8546d9503faa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:41.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.703+0000 7f1b7d600640 1 -- 172.21.15.123:0/2299217020 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68001a70 con 0x7f1b60004800 2024-04-24T05:47:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.703+0000 7f1b7d600640 1 -- 172.21.15.123:0/2299217020 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6802ae40 con 0x7f1b60004800 2024-04-24T05:47:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.703+0000 7f1b7d600640 1 -- 172.21.15.123:0/2299217020 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68003bb0 con 0x7f1b60004800 2024-04-24T05:47:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.703+0000 7f1b85800640 1 -- 172.21.15.123:0/2299217020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b60004800 msgr2=0x7f1b60004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.703+0000 7f1b85800640 1 --2- 172.21.15.123:0/2299217020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b60004800 0x7f1b60004c80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1b68001e20 tx=0x7f1b60005830 comp rx=0 tx=0).stop 2024-04-24T05:47:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.703+0000 7f1b7f400640 1 -- 172.21.15.123:0/2299217020 reap_dead start 2024-04-24T05:47:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.704+0000 7f1b85800640 1 -- 172.21.15.123:0/2299217020 shutdown_connections 2024-04-24T05:47:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.704+0000 7f1b85800640 1 -- 172.21.15.123:0/2299217020 >> 172.21.15.123:0/2299217020 conn(0x7f1b800b9a40 msgr2=0x7f1b800b75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.704+0000 7f1b85800640 1 -- 172.21.15.123:0/2299217020 shutdown_connections 2024-04-24T05:47:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.704+0000 7f1b85800640 1 -- 172.21.15.123:0/2299217020 wait complete. 2024-04-24T05:47:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.705+0000 7f1b85800640 1 Processor -- start 2024-04-24T05:47:41.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.706+0000 7f1b85800640 1 -- start start 2024-04-24T05:47:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b85800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b60004800 0x7f1b80157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b85800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b80158e90 0x7f1b8015ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b85800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b80159310 0x7f1b80160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b85800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b800d0650 con 0x7f1b60004800 2024-04-24T05:47:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b85800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b800d0370 con 0x7f1b80159310 2024-04-24T05:47:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b85800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8015ed50 con 0x7f1b80158e90 2024-04-24T05:47:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b7ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b60004800 0x7f1b80157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b7ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b60004800 0x7f1b80157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38792/0 (socket says 172.21.15.123:38792) 2024-04-24T05:47:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b7ea00640 1 -- 172.21.15.123:0/1893534295 learned_addr learned my addr 172.21.15.123:0/1893534295 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b7e000640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b80158e90 0x7f1b8015ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b7f400640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b80159310 0x7f1b80160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b7ea00640 1 -- 172.21.15.123:0/1893534295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b80158e90 msgr2=0x7f1b8015ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b7ea00640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b80158e90 0x7f1b8015ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:41.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b7ea00640 1 -- 172.21.15.123:0/1893534295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b80159310 msgr2=0x7f1b80160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:41.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b7ea00640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b80159310 0x7f1b80160480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:41.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b7ea00640 1 -- 172.21.15.123:0/1893534295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b801f7e10 con 0x7f1b60004800 2024-04-24T05:47:41.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.707+0000 7f1b7e000640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b80158e90 0x7f1b8015ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:41.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.708+0000 7f1b7ea00640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b60004800 0x7f1b80157ee0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1b700119d0 tx=0x7f1b70011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:41.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.708+0000 7f1b76a00640 1 -- 172.21.15.123:0/1893534295 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7000faf0 con 0x7f1b60004800 2024-04-24T05:47:41.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.708+0000 7f1b76a00640 1 -- 172.21.15.123:0/1893534295 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b7000fc90 con 0x7f1b60004800 2024-04-24T05:47:41.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.708+0000 7f1b85800640 1 -- 172.21.15.123:0/1893534295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b80158420 con 0x7f1b60004800 2024-04-24T05:47:41.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.708+0000 7f1b76a00640 1 -- 172.21.15.123:0/1893534295 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70014440 con 0x7f1b60004800 2024-04-24T05:47:41.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.708+0000 7f1b85800640 1 -- 172.21.15.123:0/1893534295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b801f8360 con 0x7f1b60004800 2024-04-24T05:47:41.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.709+0000 7f1b85800640 1 -- 172.21.15.123:0/1893534295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b80074350 con 0x7f1b60004800 2024-04-24T05:47:41.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.710+0000 7f1b76a00640 1 -- 172.21.15.123:0/1893534295 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b70012040 con 0x7f1b60004800 2024-04-24T05:47:41.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.711+0000 7f1b76a00640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1b5c06c8f0 0x7f1b5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:41.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.711+0000 7f1b76a00640 1 -- 172.21.15.123:0/1893534295 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7f1b7008c810 con 0x7f1b60004800 2024-04-24T05:47:41.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.712+0000 7f1b7e000640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1b5c06c8f0 0x7f1b5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:41.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.712+0000 7f1b7e000640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1b5c06c8f0 0x7f1b5c06edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1b68032000 tx=0x7f1b6802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:41.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.715+0000 7f1b76a00640 1 -- 172.21.15.123:0/1893534295 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b80074350 con 0x7f1b60004800 2024-04-24T05:47:41.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:41.902+0000 7f1b85800640 1 -- 172.21.15.123:0/1893534295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1b8015d900 con 0x7f1b60004800 2024-04-24T05:47:42.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.490+0000 7f1b76a00640 1 -- 172.21.15.123:0/1893534295 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7f1b8015d900 con 0x7f1b60004800 2024-04-24T05:47:42.492 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' created 2024-04-24T05:47:42.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.494+0000 7f1b85800640 1 -- 172.21.15.123:0/1893534295 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1b5c06c8f0 msgr2=0x7f1b5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:42.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.494+0000 7f1b85800640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1b5c06c8f0 0x7f1b5c06edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1b68032000 tx=0x7f1b6802e000 comp rx=0 tx=0).stop 2024-04-24T05:47:42.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.494+0000 7f1b85800640 1 -- 172.21.15.123:0/1893534295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b60004800 msgr2=0x7f1b80157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:42.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.494+0000 7f1b85800640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b60004800 0x7f1b80157ee0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1b700119d0 tx=0x7f1b70011e90 comp rx=0 tx=0).stop 2024-04-24T05:47:42.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.495+0000 7f1b85800640 1 -- 172.21.15.123:0/1893534295 shutdown_connections 2024-04-24T05:47:42.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.495+0000 7f1b85800640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1b5c06c8f0 0x7f1b5c06edb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:42.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.495+0000 7f1b85800640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b80159310 0x7f1b80160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:42.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.495+0000 7f1b85800640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b80158e90 0x7f1b8015ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:42.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.495+0000 7f1b85800640 1 --2- 172.21.15.123:0/1893534295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b60004800 0x7f1b80157ee0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:42.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.495+0000 7f1b85800640 1 -- 172.21.15.123:0/1893534295 >> 172.21.15.123:0/1893534295 conn(0x7f1b800b9a40 msgr2=0x7f1b8015dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:42.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.495+0000 7f1b85800640 1 -- 172.21.15.123:0/1893534295 shutdown_connections 2024-04-24T05:47:42.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.495+0000 7f1b85800640 1 -- 172.21.15.123:0/1893534295 wait complete. 2024-04-24T05:47:42.532 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T05:47:42.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.706+0000 7fdab7200640 1 Processor -- start 2024-04-24T05:47:42.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.706+0000 7fdab7200640 1 -- start start 2024-04-24T05:47:42.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.707+0000 7fdab7200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdab815c2f0 0x7fdab8141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:42.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.707+0000 7fdab7200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdab80cc5d0 0x7fdab80acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:42.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.707+0000 7fdab7200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdab80ad510 0x7fdab815ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:42.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.707+0000 7fdab7200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdab80ad980 con 0x7fdab80cc5d0 2024-04-24T05:47:42.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.707+0000 7fdab7200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdab806bd30 con 0x7fdab815c2f0 2024-04-24T05:47:42.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.707+0000 7fdab7200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdab806ba00 con 0x7fdab80ad510 2024-04-24T05:47:42.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.708+0000 7fdaad400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdab80cc5d0 0x7fdab80acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:42.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.708+0000 7fdaad400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdab80cc5d0 0x7fdab80acfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38800/0 (socket says 172.21.15.123:38800) 2024-04-24T05:47:42.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.708+0000 7fdaad400640 1 -- 172.21.15.123:0/2906917761 learned_addr learned my addr 172.21.15.123:0/2906917761 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:42.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.708+0000 7fdaad400640 1 -- 172.21.15.123:0/2906917761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdab80ad510 msgr2=0x7fdab815ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:42.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.708+0000 7fdaad400640 1 --2- 172.21.15.123:0/2906917761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdab80ad510 0x7fdab815ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:42.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.708+0000 7fdab5e00640 1 --2- 172.21.15.123:0/2906917761 >> v2:172.21.15.155:3300/0 conn(0x7fdab815c2f0 0x7fdab8141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:42.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.708+0000 7fdaad400640 1 -- 172.21.15.123:0/2906917761 >> v2:172.21.15.155:3300/0 conn(0x7fdab815c2f0 msgr2=0x7fdab8141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:42.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.708+0000 7fdaad400640 1 --2- 172.21.15.123:0/2906917761 >> v2:172.21.15.155:3300/0 conn(0x7fdab815c2f0 0x7fdab8141ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:42.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.708+0000 7fdaad400640 1 -- 172.21.15.123:0/2906917761 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab8142530 con 0x7fdab80cc5d0 2024-04-24T05:47:42.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.709+0000 7fdaad400640 1 --2- 172.21.15.123:0/2906917761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab80cc5d0 0x7fdab80acfd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdab80afb30 tx=0x7fdaa002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74945fba9fe1db3c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:42.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.709+0000 7fdab5400640 1 -- 172.21.15.123:0/2906917761 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0037070 con 0x7fdab80cc5d0 2024-04-24T05:47:42.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.710+0000 7fdab5400640 1 -- 172.21.15.123:0/2906917761 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa0032440 con 0x7fdab80cc5d0 2024-04-24T05:47:42.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.710+0000 7fdab5400640 1 -- 172.21.15.123:0/2906917761 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0032810 con 0x7fdab80cc5d0 2024-04-24T05:47:42.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.710+0000 7fdab7200640 1 -- 172.21.15.123:0/2906917761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab80cc5d0 msgr2=0x7fdab80acfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:42.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.710+0000 7fdab7200640 1 --2- 172.21.15.123:0/2906917761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab80cc5d0 0x7fdab80acfd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdab80afb30 tx=0x7fdaa002a140 comp rx=0 tx=0).stop 2024-04-24T05:47:42.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.710+0000 7fdab7200640 1 -- 172.21.15.123:0/2906917761 shutdown_connections 2024-04-24T05:47:42.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.710+0000 7fdab7200640 1 --2- 172.21.15.123:0/2906917761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdab80ad510 0x7fdab815ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:42.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.710+0000 7fdab7200640 1 --2- 172.21.15.123:0/2906917761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab80cc5d0 0x7fdab80acfd0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:42.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.710+0000 7fdab7200640 1 --2- 172.21.15.123:0/2906917761 >> v2:172.21.15.155:3300/0 conn(0x7fdab815c2f0 0x7fdab8141ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:42.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.710+0000 7fdab7200640 1 -- 172.21.15.123:0/2906917761 >> 172.21.15.123:0/2906917761 conn(0x7fdab80b1cf0 msgr2=0x7fdab80b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:42.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.710+0000 7fdab7200640 1 -- 172.21.15.123:0/2906917761 shutdown_connections 2024-04-24T05:47:42.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.710+0000 7fdab7200640 1 -- 172.21.15.123:0/2906917761 wait complete. 2024-04-24T05:47:42.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.712+0000 7fdab7200640 1 Processor -- start 2024-04-24T05:47:42.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.712+0000 7fdab7200640 1 -- start start 2024-04-24T05:47:42.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.713+0000 7fdab7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab80ad510 0x7fdab8142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:42.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.713+0000 7fdab7200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdab80cc5d0 0x7fdab80b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:42.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.713+0000 7fdab7200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdab815c2f0 0x7fdab81f4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:42.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.713+0000 7fdab7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdab8142220 con 0x7fdab80ad510 2024-04-24T05:47:42.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.713+0000 7fdab7200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdab8142af0 con 0x7fdab815c2f0 2024-04-24T05:47:42.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.713+0000 7fdab7200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdab815c6d0 con 0x7fdab80cc5d0 2024-04-24T05:47:42.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.713+0000 7fdab5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab80ad510 0x7fdab8142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:42.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.713+0000 7fdab5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab80ad510 0x7fdab8142fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38812/0 (socket says 172.21.15.123:38812) 2024-04-24T05:47:42.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.713+0000 7fdab5e00640 1 -- 172.21.15.123:0/1897643154 learned_addr learned my addr 172.21.15.123:0/1897643154 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:42.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.714+0000 7fdaad400640 1 --2- 172.21.15.123:0/1897643154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdab80cc5d0 0x7fdab80b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:42.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.714+0000 7fdab6800640 1 --2- 172.21.15.123:0/1897643154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdab815c2f0 0x7fdab81f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:42.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.714+0000 7fdab5e00640 1 -- 172.21.15.123:0/1897643154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdab80cc5d0 msgr2=0x7fdab80b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:42.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.714+0000 7fdab5e00640 1 --2- 172.21.15.123:0/1897643154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdab80cc5d0 0x7fdab80b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:42.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.714+0000 7fdab5e00640 1 -- 172.21.15.123:0/1897643154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdab815c2f0 msgr2=0x7fdab81f4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:42.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.714+0000 7fdab5e00640 1 --2- 172.21.15.123:0/1897643154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdab815c2f0 0x7fdab81f4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:42.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.714+0000 7fdab5e00640 1 -- 172.21.15.123:0/1897643154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab80cca30 con 0x7fdab80ad510 2024-04-24T05:47:42.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.714+0000 7fdab5e00640 1 --2- 172.21.15.123:0/1897643154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab80ad510 0x7fdab8142fd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800ce10 tx=0x7fdaa800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:42.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.714+0000 7fdaaea00640 1 -- 172.21.15.123:0/1897643154 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8003b90 con 0x7fdab80ad510 2024-04-24T05:47:42.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.714+0000 7fdaaea00640 1 -- 172.21.15.123:0/1897643154 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa8003d30 con 0x7fdab80ad510 2024-04-24T05:47:42.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.714+0000 7fdaaea00640 1 -- 172.21.15.123:0/1897643154 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8004240 con 0x7fdab80ad510 2024-04-24T05:47:42.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.714+0000 7fdab7200640 1 -- 172.21.15.123:0/1897643154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab80ccd10 con 0x7fdab80ad510 2024-04-24T05:47:42.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.715+0000 7fdab7200640 1 -- 172.21.15.123:0/1897643154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab80cd2d0 con 0x7fdab80ad510 2024-04-24T05:47:42.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.716+0000 7fdab7200640 1 -- 172.21.15.123:0/1897643154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab8068a20 con 0x7fdab80ad510 2024-04-24T05:47:42.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.717+0000 7fdaaea00640 1 -- 172.21.15.123:0/1897643154 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdaa80043e0 con 0x7fdab80ad510 2024-04-24T05:47:42.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.718+0000 7fdaaea00640 1 --2- 172.21.15.123:0/1897643154 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda9406c8f0 0x7fda9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:42.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.718+0000 7fdaaea00640 1 -- 172.21.15.123:0/1897643154 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7fdaa808c7e0 con 0x7fdab80ad510 2024-04-24T05:47:42.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.718+0000 7fdaad400640 1 --2- 172.21.15.123:0/1897643154 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda9406c8f0 0x7fda9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:42.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.719+0000 7fdaad400640 1 --2- 172.21.15.123:0/1897643154 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda9406c8f0 0x7fda9406edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0034000 tx=0x7fdaa002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:42.724 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.723+0000 7fdaaea00640 1 -- 172.21.15.123:0/1897643154 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdaa801d340 con 0x7fdab80ad510 2024-04-24T05:47:42.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:42.951+0000 7fdab7200640 1 -- 172.21.15.123:0/1897643154 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fdab80cd670 con 0x7fdab80ad510 2024-04-24T05:47:43.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.510+0000 7fdaaea00640 1 -- 172.21.15.123:0/1897643154 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7fdaa805aae0 con 0x7fdab80ad510 2024-04-24T05:47:43.513 INFO:teuthology.orchestra.run.smithi123.stderr:new fs with metadata pool 5 and data pool 6 2024-04-24T05:47:43.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.519+0000 7fdab7200640 1 -- 172.21.15.123:0/1897643154 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda9406c8f0 msgr2=0x7fda9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:43.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.519+0000 7fdab7200640 1 --2- 172.21.15.123:0/1897643154 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda9406c8f0 0x7fda9406edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0034000 tx=0x7fdaa002e000 comp rx=0 tx=0).stop 2024-04-24T05:47:43.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.519+0000 7fdab7200640 1 -- 172.21.15.123:0/1897643154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab80ad510 msgr2=0x7fdab8142fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:43.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.519+0000 7fdab7200640 1 --2- 172.21.15.123:0/1897643154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab80ad510 0x7fdab8142fd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800ce10 tx=0x7fdaa800a660 comp rx=0 tx=0).stop 2024-04-24T05:47:43.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.523+0000 7fdab7200640 1 -- 172.21.15.123:0/1897643154 shutdown_connections 2024-04-24T05:47:43.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.523+0000 7fdab7200640 1 --2- 172.21.15.123:0/1897643154 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda9406c8f0 0x7fda9406edb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:43.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.523+0000 7fdab7200640 1 --2- 172.21.15.123:0/1897643154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdab815c2f0 0x7fdab81f4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:43.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.523+0000 7fdab7200640 1 --2- 172.21.15.123:0/1897643154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdab80cc5d0 0x7fdab80b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:43.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.523+0000 7fdab7200640 1 --2- 172.21.15.123:0/1897643154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab80ad510 0x7fdab8142fd0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:43.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.523+0000 7fdab7200640 1 -- 172.21.15.123:0/1897643154 >> 172.21.15.123:0/1897643154 conn(0x7fdab80b1cf0 msgr2=0x7fdab806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:43.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.524+0000 7fdab7200640 1 -- 172.21.15.123:0/1897643154 shutdown_connections 2024-04-24T05:47:43.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.524+0000 7fdab7200640 1 -- 172.21.15.123:0/1897643154 wait complete. 2024-04-24T05:47:43.559 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:43.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.727+0000 7f1e10a00640 1 Processor -- start 2024-04-24T05:47:43.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.727+0000 7f1e10a00640 1 -- start start 2024-04-24T05:47:43.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.729+0000 7f1e10a00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e0c161ba0 0x7f1e0c1623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:43.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.729+0000 7f1e10a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e0c161f80 0x7f1e0c162930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:43.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.729+0000 7f1e10a00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e0c06fb40 con 0x7f1e0c0ada30 2024-04-24T05:47:43.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.729+0000 7f1e10a00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1e0c06fe20 con 0x7f1e0c161ba0 2024-04-24T05:47:43.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.729+0000 7f1e10a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0c06c220 con 0x7f1e0c161f80 2024-04-24T05:47:43.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.729+0000 7f1e09e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e0c161ba0 0x7f1e0c1623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:43.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.730+0000 7f1e09e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e0c161ba0 0x7f1e0c1623f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:40966/0 (socket says 172.21.15.123:40966) 2024-04-24T05:47:43.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.730+0000 7f1e09e00640 1 -- 172.21.15.123:0/1126702289 learned_addr learned my addr 172.21.15.123:0/1126702289 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:43.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.730+0000 7f1e09400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e0c161f80 0x7f1e0c162930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:43.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.730+0000 7f1e09e00640 1 -- 172.21.15.123:0/1126702289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e0c161f80 msgr2=0x7f1e0c162930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:43.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.730+0000 7f1e09e00640 1 --2- 172.21.15.123:0/1126702289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e0c161f80 0x7f1e0c162930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:43.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.730+0000 7f1e09e00640 1 -- 172.21.15.123:0/1126702289 >> v1:172.21.15.155:6789/0 conn(0x7f1e0c0ada30 legacy=0x7f1e0c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:43.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.730+0000 7f1e09e00640 1 -- 172.21.15.123:0/1126702289 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0c06c500 con 0x7f1e0c161ba0 2024-04-24T05:47:43.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.732+0000 7f1e09e00640 1 --2- 172.21.15.123:0/1126702289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e0c161ba0 0x7f1e0c1623f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009e50 tx=0x7f1dfc02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc7271418912bad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:43.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.732+0000 7f1e08a00640 1 -- 172.21.15.123:0/1126702289 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc036070 con 0x7f1e0c161ba0 2024-04-24T05:47:43.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.732+0000 7f1e08a00640 1 -- 172.21.15.123:0/1126702289 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dfc031600 con 0x7f1e0c161ba0 2024-04-24T05:47:43.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.732+0000 7f1e08a00640 1 -- 172.21.15.123:0/1126702289 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0319d0 con 0x7f1e0c161ba0 2024-04-24T05:47:43.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.733+0000 7f1e10a00640 1 -- 172.21.15.123:0/1126702289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e0c161ba0 msgr2=0x7f1e0c1623f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:43.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.733+0000 7f1e10a00640 1 --2- 172.21.15.123:0/1126702289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e0c161ba0 0x7f1e0c1623f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009e50 tx=0x7f1dfc02a680 comp rx=0 tx=0).stop 2024-04-24T05:47:43.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.733+0000 7f1e10a00640 1 -- 172.21.15.123:0/1126702289 shutdown_connections 2024-04-24T05:47:43.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.733+0000 7f1e10a00640 1 --2- 172.21.15.123:0/1126702289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e0c161f80 0x7f1e0c162930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:43.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.733+0000 7f1e10a00640 1 --2- 172.21.15.123:0/1126702289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e0c161ba0 0x7f1e0c1623f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:43.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.733+0000 7f1e10a00640 1 -- 172.21.15.123:0/1126702289 >> 172.21.15.123:0/1126702289 conn(0x7f1e0c0d0b70 msgr2=0x7f1e0c0d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:43.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.733+0000 7f1e10a00640 1 -- 172.21.15.123:0/1126702289 shutdown_connections 2024-04-24T05:47:43.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.733+0000 7f1e10a00640 1 -- 172.21.15.123:0/1126702289 wait complete. 2024-04-24T05:47:43.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.735+0000 7f1e10a00640 1 Processor -- start 2024-04-24T05:47:43.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.735+0000 7f1e10a00640 1 -- start start 2024-04-24T05:47:43.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e10a00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e0c0ada30 0x7f1e0c140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:43.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e10a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e0c161ba0 0x7f1e0c1f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:43.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e10a00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e0c161f80 0x7f1e0c141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:43.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e10a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c0b3ee0 con 0x7f1e0c161ba0 2024-04-24T05:47:43.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e10a00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c0b3d60 con 0x7f1e0c0ada30 2024-04-24T05:47:43.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e10a00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0c0b4060 con 0x7f1e0c161f80 2024-04-24T05:47:43.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e09400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e0c161ba0 0x7f1e0c1f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:43.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e09400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e0c161ba0 0x7f1e0c1f87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38814/0 (socket says 172.21.15.123:38814) 2024-04-24T05:47:43.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e09400640 1 -- 172.21.15.123:0/3407016742 learned_addr learned my addr 172.21.15.123:0/3407016742 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:43.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e09e00640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e0c0ada30 0x7f1e0c140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:43.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e0a800640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e0c161f80 0x7f1e0c141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:43.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e09400640 1 -- 172.21.15.123:0/3407016742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e0c161f80 msgr2=0x7f1e0c141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:43.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e09400640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e0c161f80 0x7f1e0c141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:43.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e09400640 1 -- 172.21.15.123:0/3407016742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e0c0ada30 msgr2=0x7f1e0c140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:43.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e09400640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e0c0ada30 0x7f1e0c140380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:43.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e09400640 1 -- 172.21.15.123:0/3407016742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0c1f8d20 con 0x7f1e0c161ba0 2024-04-24T05:47:43.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.736+0000 7f1e0a800640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e0c161f80 0x7f1e0c141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:43.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.737+0000 7f1e09400640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e0c161ba0 0x7f1e0c1f87e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9b0 tx=0x7f1df800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:43.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.737+0000 7f1e02000640 1 -- 172.21.15.123:0/3407016742 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8017070 con 0x7f1e0c161ba0 2024-04-24T05:47:43.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.737+0000 7f1e02000640 1 -- 172.21.15.123:0/3407016742 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df800c9a0 con 0x7f1e0c161ba0 2024-04-24T05:47:43.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.737+0000 7f1e10a00640 1 -- 172.21.15.123:0/3407016742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0c1426b0 con 0x7f1e0c161ba0 2024-04-24T05:47:43.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.737+0000 7f1e02000640 1 -- 172.21.15.123:0/3407016742 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df800cd90 con 0x7f1e0c161ba0 2024-04-24T05:47:43.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.737+0000 7f1e10a00640 1 -- 172.21.15.123:0/3407016742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e0c142c70 con 0x7f1e0c161ba0 2024-04-24T05:47:43.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.738+0000 7f1e10a00640 1 -- 172.21.15.123:0/3407016742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e0c069130 con 0x7f1e0c161ba0 2024-04-24T05:47:43.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.739+0000 7f1e02000640 1 -- 172.21.15.123:0/3407016742 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1df801a070 con 0x7f1e0c161ba0 2024-04-24T05:47:43.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.740+0000 7f1e02000640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1dec06c8f0 0x7f1dec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:43.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.741+0000 7f1e02000640 1 -- 172.21.15.123:0/3407016742 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1df808bc80 con 0x7f1e0c161ba0 2024-04-24T05:47:43.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.741+0000 7f1e09e00640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1dec06c8f0 0x7f1dec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:43.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.742+0000 7f1e09e00640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1dec06c8f0 0x7f1dec06edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc033000 tx=0x7f1dfc009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:43.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.745+0000 7f1e02000640 1 -- 172.21.15.123:0/3407016742 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1df8091020 con 0x7f1e0c161ba0 2024-04-24T05:47:43.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.934+0000 7f1e10a00640 1 -- 172.21.15.123:0/3407016742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e0c06bca0 con 0x7f1e0c161ba0 2024-04-24T05:47:43.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.935+0000 7f1e02000640 1 -- 172.21.15.123:0/3407016742 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16991 (secure 0 0 0) 0x7f1e0c1f8d20 con 0x7f1e0c161ba0 2024-04-24T05:47:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:43.938 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":38,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:47:42.953598+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:47:40.759785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-24T05:47:41.914036+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:43.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.938+0000 7f1e10a00640 1 -- 172.21.15.123:0/3407016742 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1dec06c8f0 msgr2=0x7f1dec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:43.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.938+0000 7f1e10a00640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1dec06c8f0 0x7f1dec06edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc033000 tx=0x7f1dfc009000 comp rx=0 tx=0).stop 2024-04-24T05:47:43.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.938+0000 7f1e10a00640 1 -- 172.21.15.123:0/3407016742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e0c161ba0 msgr2=0x7f1e0c1f87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:43.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.938+0000 7f1e10a00640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e0c161ba0 0x7f1e0c1f87e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9b0 tx=0x7f1df800ee70 comp rx=0 tx=0).stop 2024-04-24T05:47:43.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.938+0000 7f1e10a00640 1 -- 172.21.15.123:0/3407016742 shutdown_connections 2024-04-24T05:47:43.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.938+0000 7f1e10a00640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f1dec06c8f0 0x7f1dec06edb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:43.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.938+0000 7f1e10a00640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e0c161f80 0x7f1e0c141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:43.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.938+0000 7f1e10a00640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e0c161ba0 0x7f1e0c1f87e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:43.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.938+0000 7f1e10a00640 1 --2- 172.21.15.123:0/3407016742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e0c0ada30 0x7f1e0c140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:43.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.938+0000 7f1e10a00640 1 -- 172.21.15.123:0/3407016742 >> 172.21.15.123:0/3407016742 conn(0x7f1e0c0d0b70 msgr2=0x7f1e0c06e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:43.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.939+0000 7f1e10a00640 1 -- 172.21.15.123:0/3407016742 shutdown_connections 2024-04-24T05:47:43.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:43.939+0000 7f1e10a00640 1 -- 172.21.15.123:0/3407016742 wait complete. 2024-04-24T05:47:43.957 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:44.145 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.144+0000 7f954b200640 1 Processor -- start 2024-04-24T05:47:44.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.144+0000 7f954b200640 1 -- start start 2024-04-24T05:47:44.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.146+0000 7f954b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954c06ea40 0x7f954c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:44.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.146+0000 7f954b200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f954c0d0de0 con 0x7f954c0ad950 2024-04-24T05:47:44.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.146+0000 7f954b200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f954c06dc80 con 0x7f954c06e660 2024-04-24T05:47:44.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.146+0000 7f954b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f954c15ec00 con 0x7f954c06ea40 2024-04-24T05:47:44.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.146+0000 7f9549400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f954c0ad950 0x7f954c06c110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:51144/0 (socket says 172.21.15.123:51144) 2024-04-24T05:47:44.148 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.146+0000 7f9549400640 1 -- 172.21.15.123:0/3490936865 learned_addr learned my addr 172.21.15.123:0/3490936865 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:44.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.146+0000 7f954a800640 1 --2- 172.21.15.123:0/3490936865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954c06ea40 0x7f954c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:44.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.146+0000 7f9548a00640 1 -- 172.21.15.123:0/3490936865 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2247564831 0 0) 0x7f954c0d0de0 con 0x7f954c0ad950 2024-04-24T05:47:44.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9548a00640 1 -- 172.21.15.123:0/3490936865 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9530003520 con 0x7f954c0ad950 2024-04-24T05:47:44.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9548a00640 1 -- 172.21.15.123:0/3490936865 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 982106365 0 0) 0x7f9530003520 con 0x7f954c0ad950 2024-04-24T05:47:44.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9548a00640 1 -- 172.21.15.123:0/3490936865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954c06ea40 msgr2=0x7f954c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:44.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9548a00640 1 --2- 172.21.15.123:0/3490936865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954c06ea40 0x7f954c0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:44.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9548a00640 1 -- 172.21.15.123:0/3490936865 >> v1:172.21.15.155:6789/0 conn(0x7f954c06e660 legacy=0x7f954c06ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:44.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9548a00640 1 -- 172.21.15.123:0/3490936865 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c0ae0a0 con 0x7f954c0ad950 2024-04-24T05:47:44.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9548a00640 1 -- 172.21.15.123:0/3490936865 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1774909351 0 0) 0x7f9538001670 con 0x7f954c0ad950 2024-04-24T05:47:44.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9548a00640 1 -- 172.21.15.123:0/3490936865 >> v1:172.21.15.123:6789/0 conn(0x7f954c0ad950 legacy=0x7f954c06c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:44.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9548a00640 1 --2- 172.21.15.123:0/3490936865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300039e0 0x7f9530003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:44.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9548a00640 1 --2- 172.21.15.123:0/3490936865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954c06ea40 0x7f9530004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:44.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9548a00640 1 --2- 172.21.15.123:0/3490936865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9530004d40 0x7f95300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:44.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9548a00640 1 -- 172.21.15.123:0/3490936865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f953000ba70 con 0x7f95300039e0 2024-04-24T05:47:44.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9548a00640 1 -- 172.21.15.123:0/3490936865 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f953000bbf0 con 0x7f9530004d40 2024-04-24T05:47:44.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9548a00640 1 -- 172.21.15.123:0/3490936865 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f953000bdf0 con 0x7f954c06ea40 2024-04-24T05:47:44.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.147+0000 7f9549e00640 1 --2- 172.21.15.123:0/3490936865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300039e0 0x7f9530003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:44.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.148+0000 7f9549400640 1 --2- 172.21.15.123:0/3490936865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954c06ea40 0x7f9530004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:44.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.148+0000 7f954a800640 1 --2- 172.21.15.123:0/3490936865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9530004d40 0x7f95300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:44.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.148+0000 7f9549e00640 1 -- 172.21.15.123:0/3490936865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954c06ea40 msgr2=0x7f9530004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:44.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.148+0000 7f9549e00640 1 --2- 172.21.15.123:0/3490936865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954c06ea40 0x7f9530004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:44.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.148+0000 7f9549e00640 1 -- 172.21.15.123:0/3490936865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9530004d40 msgr2=0x7f95300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:44.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.148+0000 7f9549e00640 1 --2- 172.21.15.123:0/3490936865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9530004d40 0x7f95300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:44.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.148+0000 7f9549e00640 1 -- 172.21.15.123:0/3490936865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9530003520 con 0x7f95300039e0 2024-04-24T05:47:44.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.148+0000 7f954a800640 1 --2- 172.21.15.123:0/3490936865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9530004d40 0x7f95300051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:44.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.149+0000 7f9549e00640 1 --2- 172.21.15.123:0/3490936865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300039e0 0x7f9530003e00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f953c001720 tx=0x7f9530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ddc08cb1a1cc8bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:44.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.149+0000 7f9548a00640 1 -- 172.21.15.123:0/3490936865 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c001370 con 0x7f95300039e0 2024-04-24T05:47:44.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.149+0000 7f9548a00640 1 -- 172.21.15.123:0/3490936865 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953c0052d0 con 0x7f95300039e0 2024-04-24T05:47:44.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.149+0000 7f9548a00640 1 -- 172.21.15.123:0/3490936865 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c02abc0 con 0x7f95300039e0 2024-04-24T05:47:44.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.149+0000 7f954b200640 1 -- 172.21.15.123:0/3490936865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300039e0 msgr2=0x7f9530003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:44.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.149+0000 7f954b200640 1 --2- 172.21.15.123:0/3490936865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300039e0 0x7f9530003e00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f953c001720 tx=0x7f9530005830 comp rx=0 tx=0).stop 2024-04-24T05:47:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.149+0000 7f954a800640 1 -- 172.21.15.123:0/3490936865 reap_dead start 2024-04-24T05:47:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.150+0000 7f954b200640 1 -- 172.21.15.123:0/3490936865 shutdown_connections 2024-04-24T05:47:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.150+0000 7f954b200640 1 -- 172.21.15.123:0/3490936865 >> 172.21.15.123:0/3490936865 conn(0x7f954c0d0070 msgr2=0x7f954c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.150+0000 7f954b200640 1 -- 172.21.15.123:0/3490936865 shutdown_connections 2024-04-24T05:47:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.150+0000 7f954b200640 1 -- 172.21.15.123:0/3490936865 wait complete. 2024-04-24T05:47:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.151+0000 7f954b200640 1 Processor -- start 2024-04-24T05:47:44.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.151+0000 7f954b200640 1 -- start start 2024-04-24T05:47:44.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.152+0000 7f954b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300039e0 0x7f954c14e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:44.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.152+0000 7f954b200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954c156420 0x7f954c1568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:44.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.152+0000 7f954b200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f954c06ba00 0x7f954c06be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:44.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.152+0000 7f954b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f954c0cf7b0 con 0x7f95300039e0 2024-04-24T05:47:44.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.152+0000 7f954b200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f954c0ade50 con 0x7f954c06ba00 2024-04-24T05:47:44.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.152+0000 7f954b200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f954c0cf930 con 0x7f954c156420 2024-04-24T05:47:44.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.153+0000 7f9549e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300039e0 0x7f954c14e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:44.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.153+0000 7f9549e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300039e0 0x7f954c14e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38830/0 (socket says 172.21.15.123:38830) 2024-04-24T05:47:44.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.153+0000 7f9549e00640 1 -- 172.21.15.123:0/2902553849 learned_addr learned my addr 172.21.15.123:0/2902553849 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:44.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.153+0000 7f9549e00640 1 -- 172.21.15.123:0/2902553849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954c156420 msgr2=0x7f954c1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:44.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.153+0000 7f9549e00640 1 --2- 172.21.15.123:0/2902553849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954c156420 0x7f954c1568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:44.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.153+0000 7f9549e00640 1 -- 172.21.15.123:0/2902553849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f954c06ba00 msgr2=0x7f954c06be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:44.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.153+0000 7f9549e00640 1 --2- 172.21.15.123:0/2902553849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f954c06ba00 0x7f954c06be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:44.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.153+0000 7f9549e00640 1 -- 172.21.15.123:0/2902553849 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c156ee0 con 0x7f95300039e0 2024-04-24T05:47:44.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.153+0000 7f9549e00640 1 --2- 172.21.15.123:0/2902553849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300039e0 0x7f954c14e2c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f953c001840 tx=0x7f953c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:44.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.154+0000 7f9542000640 1 -- 172.21.15.123:0/2902553849 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c037040 con 0x7f95300039e0 2024-04-24T05:47:44.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.154+0000 7f9542000640 1 -- 172.21.15.123:0/2902553849 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953c005480 con 0x7f95300039e0 2024-04-24T05:47:44.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.154+0000 7f954b200640 1 -- 172.21.15.123:0/2902553849 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954c1571c0 con 0x7f95300039e0 2024-04-24T05:47:44.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.154+0000 7f9542000640 1 -- 172.21.15.123:0/2902553849 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c001370 con 0x7f95300039e0 2024-04-24T05:47:44.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.154+0000 7f954b200640 1 -- 172.21.15.123:0/2902553849 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954c157780 con 0x7f95300039e0 2024-04-24T05:47:44.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.155+0000 7f954b200640 1 -- 172.21.15.123:0/2902553849 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f954c075b70 con 0x7f95300039e0 2024-04-24T05:47:44.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.156+0000 7f9542000640 1 -- 172.21.15.123:0/2902553849 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f953c044020 con 0x7f95300039e0 2024-04-24T05:47:44.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.157+0000 7f9542000640 1 --2- 172.21.15.123:0/2902553849 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f952c06c8f0 0x7f952c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:44.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.157+0000 7f9542000640 1 -- 172.21.15.123:0/2902553849 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f953c007640 con 0x7f95300039e0 2024-04-24T05:47:44.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.157+0000 7f9549400640 1 --2- 172.21.15.123:0/2902553849 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f952c06c8f0 0x7f952c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:44.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.157+0000 7f9549400640 1 --2- 172.21.15.123:0/2902553849 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f952c06c8f0 0x7f952c06edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9538000c00 tx=0x7f9538009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:44.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.161+0000 7f9542000640 1 -- 172.21.15.123:0/2902553849 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f953c079ce0 con 0x7f95300039e0 2024-04-24T05:47:44.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.347+0000 7f954b200640 1 -- 172.21.15.123:0/2902553849 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f954c0b0900 con 0x7f95300039e0 2024-04-24T05:47:44.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.350+0000 7f9542000640 1 -- 172.21.15.123:0/2902553849 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16991 (secure 0 0 0) 0x7f953c0795e0 con 0x7f95300039e0 2024-04-24T05:47:44.352 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:44.352 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":38,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:47:42.953598+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:47:40.759785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-24T05:47:41.914036+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:44.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.352+0000 7f954b200640 1 -- 172.21.15.123:0/2902553849 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f952c06c8f0 msgr2=0x7f952c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:44.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.352+0000 7f954b200640 1 --2- 172.21.15.123:0/2902553849 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f952c06c8f0 0x7f952c06edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9538000c00 tx=0x7f9538009040 comp rx=0 tx=0).stop 2024-04-24T05:47:44.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.352+0000 7f954b200640 1 -- 172.21.15.123:0/2902553849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300039e0 msgr2=0x7f954c14e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:44.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.352+0000 7f954b200640 1 --2- 172.21.15.123:0/2902553849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300039e0 0x7f954c14e2c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f953c001840 tx=0x7f953c001120 comp rx=0 tx=0).stop 2024-04-24T05:47:44.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.352+0000 7f954b200640 1 -- 172.21.15.123:0/2902553849 shutdown_connections 2024-04-24T05:47:44.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.352+0000 7f954b200640 1 --2- 172.21.15.123:0/2902553849 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f952c06c8f0 0x7f952c06edb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:44.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.352+0000 7f954b200640 1 --2- 172.21.15.123:0/2902553849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f954c06ba00 0x7f954c06be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:44.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.352+0000 7f954b200640 1 --2- 172.21.15.123:0/2902553849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f954c156420 0x7f954c1568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:44.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.352+0000 7f954b200640 1 --2- 172.21.15.123:0/2902553849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300039e0 0x7f954c14e2c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:44.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.353+0000 7f954b200640 1 -- 172.21.15.123:0/2902553849 >> 172.21.15.123:0/2902553849 conn(0x7f954c0d0070 msgr2=0x7f954c155c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:44.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.353+0000 7f954b200640 1 -- 172.21.15.123:0/2902553849 shutdown_connections 2024-04-24T05:47:44.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.353+0000 7f954b200640 1 -- 172.21.15.123:0/2902553849 wait complete. 2024-04-24T05:47:44.371 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T05:47:44.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.567+0000 7f45fe400640 1 Processor -- start 2024-04-24T05:47:44.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.568+0000 7f45fe400640 1 -- start start 2024-04-24T05:47:44.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.569+0000 7f45fe400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f45f8162ee0 0x7f45f80ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:44.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.569+0000 7f45fe400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45f80c38b0 0x7f45f806ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:44.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.569+0000 7f45fe400640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45f80cea60 con 0x7f45f806dcb0 2024-04-24T05:47:44.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.569+0000 7f45fe400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f45f80afa00 con 0x7f45f8162ee0 2024-04-24T05:47:44.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.569+0000 7f45fe400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f45f81632c0 con 0x7f45f80c38b0 2024-04-24T05:47:44.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.569+0000 7f45f7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f45f8162ee0 0x7f45f80ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:44.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.569+0000 7f45f7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f45f8162ee0 0x7f45f80ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38846/0 (socket says 172.21.15.123:38846) 2024-04-24T05:47:44.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.569+0000 7f45f7400640 1 -- 172.21.15.123:0/3450910832 learned_addr learned my addr 172.21.15.123:0/3450910832 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:44.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.569+0000 7f45f7400640 1 -- 172.21.15.123:0/3450910832 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45f80c38b0 msgr2=0x7f45f806ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:44.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.569+0000 7f45f7400640 1 --2- 172.21.15.123:0/3450910832 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45f80c38b0 0x7f45f806ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:44.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.570+0000 7f45f7400640 1 -- 172.21.15.123:0/3450910832 >> v1:172.21.15.155:6789/0 conn(0x7f45f806dcb0 legacy=0x7f45f80c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:44.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.570+0000 7f45f7400640 1 -- 172.21.15.123:0/3450910832 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f80ced40 con 0x7f45f8162ee0 2024-04-24T05:47:44.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.570+0000 7f45f7400640 1 --2- 172.21.15.123:0/3450910832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f8162ee0 0x7f45f80ce2d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009870 tx=0x7f45e802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3614519807cbaa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:44.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.571+0000 7f45f6a00640 1 -- 172.21.15.123:0/3450910832 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8037070 con 0x7f45f8162ee0 2024-04-24T05:47:44.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.571+0000 7f45f6a00640 1 -- 172.21.15.123:0/3450910832 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e8032440 con 0x7f45f8162ee0 2024-04-24T05:47:44.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.571+0000 7f45f6a00640 1 -- 172.21.15.123:0/3450910832 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8032810 con 0x7f45f8162ee0 2024-04-24T05:47:44.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.571+0000 7f45fe400640 1 -- 172.21.15.123:0/3450910832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f8162ee0 msgr2=0x7f45f80ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:44.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.571+0000 7f45fe400640 1 --2- 172.21.15.123:0/3450910832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f8162ee0 0x7f45f80ce2d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009870 tx=0x7f45e802a210 comp rx=0 tx=0).stop 2024-04-24T05:47:44.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.572+0000 7f45fe400640 1 -- 172.21.15.123:0/3450910832 shutdown_connections 2024-04-24T05:47:44.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.572+0000 7f45fe400640 1 --2- 172.21.15.123:0/3450910832 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45f80c38b0 0x7f45f806ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:44.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.572+0000 7f45fe400640 1 --2- 172.21.15.123:0/3450910832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f8162ee0 0x7f45f80ce2d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:44.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.572+0000 7f45fe400640 1 -- 172.21.15.123:0/3450910832 >> 172.21.15.123:0/3450910832 conn(0x7f45f80b51f0 msgr2=0x7f45f80b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:44.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.572+0000 7f45fe400640 1 -- 172.21.15.123:0/3450910832 shutdown_connections 2024-04-24T05:47:44.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.572+0000 7f45fe400640 1 -- 172.21.15.123:0/3450910832 wait complete. 2024-04-24T05:47:44.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.573+0000 7f45fe400640 1 Processor -- start 2024-04-24T05:47:44.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.574+0000 7f45fe400640 1 -- start start 2024-04-24T05:47:44.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.574+0000 7f45fe400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45f806dcb0 0x7f45f80c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:44.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.574+0000 7f45fe400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f80c38b0 0x7f45f81f9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:44.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45fe400640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45f8162ee0 0x7f45f80b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:44.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45fe400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45f80cf300 con 0x7f45f80c38b0 2024-04-24T05:47:44.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45fe400640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f45f80ce810 con 0x7f45f8162ee0 2024-04-24T05:47:44.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45fe400640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f45f80cf5e0 con 0x7f45f806dcb0 2024-04-24T05:47:44.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45efe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f80c38b0 0x7f45f81f9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:44.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45efe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f80c38b0 0x7f45f81f9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38858/0 (socket says 172.21.15.123:38858) 2024-04-24T05:47:44.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45efe00640 1 -- 172.21.15.123:0/1076352460 learned_addr learned my addr 172.21.15.123:0/1076352460 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:44.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45f7400640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45f806dcb0 0x7f45f80c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:44.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45efe00640 1 -- 172.21.15.123:0/1076352460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45f806dcb0 msgr2=0x7f45f80c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45efe00640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45f806dcb0 0x7f45f80c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45efe00640 1 -- 172.21.15.123:0/1076352460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45f8162ee0 msgr2=0x7f45f80b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45f7e00640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45f8162ee0 0x7f45f80b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45efe00640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45f8162ee0 0x7f45f80b0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45efe00640 1 -- 172.21.15.123:0/1076352460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f80cc030 con 0x7f45f80c38b0 2024-04-24T05:47:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.575+0000 7f45f7400640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45f806dcb0 0x7f45f80c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:44.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.576+0000 7f45efe00640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f80c38b0 0x7f45f81f9fb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f45e000c410 tx=0x7f45e000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:44.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.576+0000 7f45ef400640 1 -- 172.21.15.123:0/1076352460 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0017070 con 0x7f45f80c38b0 2024-04-24T05:47:44.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.576+0000 7f45ef400640 1 -- 172.21.15.123:0/1076352460 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e00035f0 con 0x7f45f80c38b0 2024-04-24T05:47:44.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.576+0000 7f45ef400640 1 -- 172.21.15.123:0/1076352460 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0003a20 con 0x7f45f80c38b0 2024-04-24T05:47:44.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.576+0000 7f45fe400640 1 -- 172.21.15.123:0/1076352460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f8160b80 con 0x7f45f80c38b0 2024-04-24T05:47:44.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.576+0000 7f45fe400640 1 -- 172.21.15.123:0/1076352460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f8161140 con 0x7f45f80c38b0 2024-04-24T05:47:44.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.577+0000 7f45fe400640 1 -- 172.21.15.123:0/1076352460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f8068a20 con 0x7f45f80c38b0 2024-04-24T05:47:44.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.578+0000 7f45ef400640 1 -- 172.21.15.123:0/1076352460 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45e0003bc0 con 0x7f45f80c38b0 2024-04-24T05:47:44.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.579+0000 7f45ef400640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f45d406c8f0 0x7f45d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:44.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.579+0000 7f45ef400640 1 -- 172.21.15.123:0/1076352460 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f45e008aff0 con 0x7f45f80c38b0 2024-04-24T05:47:44.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.579+0000 7f45f7400640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f45d406c8f0 0x7f45d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:44.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.580+0000 7f45f7400640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f45d406c8f0 0x7f45d406edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f45e8034000 tx=0x7f45e802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:44.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.583+0000 7f45ef400640 1 -- 172.21.15.123:0/1076352460 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f45e00592a0 con 0x7f45f80c38b0 2024-04-24T05:47:44.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:44.821+0000 7f45fe400640 1 -- 172.21.15.123:0/1076352460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f45f8161310 con 0x7f45f80c38b0 2024-04-24T05:47:45.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.547+0000 7f45ef400640 1 -- 172.21.15.123:0/1076352460 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f45e0058ba0 con 0x7f45f80c38b0 2024-04-24T05:47:45.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.551+0000 7f45fe400640 1 -- 172.21.15.123:0/1076352460 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f45d406c8f0 msgr2=0x7f45d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:45.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.551+0000 7f45fe400640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f45d406c8f0 0x7f45d406edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f45e8034000 tx=0x7f45e802e000 comp rx=0 tx=0).stop 2024-04-24T05:47:45.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.551+0000 7f45fe400640 1 -- 172.21.15.123:0/1076352460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f80c38b0 msgr2=0x7f45f81f9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:45.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.551+0000 7f45fe400640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f80c38b0 0x7f45f81f9fb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f45e000c410 tx=0x7f45e000c8d0 comp rx=0 tx=0).stop 2024-04-24T05:47:45.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.551+0000 7f45fe400640 1 -- 172.21.15.123:0/1076352460 shutdown_connections 2024-04-24T05:47:45.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.551+0000 7f45fe400640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f45d406c8f0 0x7f45d406edb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:45.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.551+0000 7f45fe400640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45f8162ee0 0x7f45f80b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:45.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.551+0000 7f45fe400640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45f80c38b0 0x7f45f81f9fb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:45.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.551+0000 7f45fe400640 1 --2- 172.21.15.123:0/1076352460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45f806dcb0 0x7f45f80c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:45.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.551+0000 7f45fe400640 1 -- 172.21.15.123:0/1076352460 >> 172.21.15.123:0/1076352460 conn(0x7f45f80b51f0 msgr2=0x7f45f815f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:45.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.552+0000 7f45fe400640 1 -- 172.21.15.123:0/1076352460 shutdown_connections 2024-04-24T05:47:45.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.552+0000 7f45fe400640 1 -- 172.21.15.123:0/1076352460 wait complete. 2024-04-24T05:47:45.572 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-24T05:47:45.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-24T05:47:45.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.772+0000 7f75d7600640 1 Processor -- start 2024-04-24T05:47:45.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.772+0000 7f75d7600640 1 -- start start 2024-04-24T05:47:45.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.773+0000 7f75d7600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f75d0071c50 0x7f75d00ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:45.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.773+0000 7f75d7600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d00cf170 con 0x7f75d006c510 2024-04-24T05:47:45.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.773+0000 7f75d7600640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d00cf450 con 0x7f75d00cd0f0 2024-04-24T05:47:45.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.773+0000 7f75d7600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f75d00cd520 con 0x7f75d0071c50 2024-04-24T05:47:45.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.774+0000 7f75d4c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f75d006c510 0x7f75d006c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:51156/0 (socket says 172.21.15.123:51156) 2024-04-24T05:47:45.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.774+0000 7f75d4c00640 1 -- 172.21.15.123:0/654051492 learned_addr learned my addr 172.21.15.123:0/654051492 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:45.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.774+0000 7f75cfe00640 1 --2- 172.21.15.123:0/654051492 >> v2:172.21.15.123:3300/0 conn(0x7f75d0071c50 0x7f75d00ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:45.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.774+0000 7f75cf400640 1 -- 172.21.15.123:0/654051492 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1794793204 0 0) 0x7f75d00cf170 con 0x7f75d006c510 2024-04-24T05:47:45.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.774+0000 7f75cf400640 1 -- 172.21.15.123:0/654051492 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75b8003520 con 0x7f75d006c510 2024-04-24T05:47:45.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.774+0000 7f75cfe00640 1 -- 172.21.15.123:0/654051492 >> v1:172.21.15.155:6789/0 conn(0x7f75d00cd0f0 legacy=0x7f75d00d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:45.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.774+0000 7f75cfe00640 1 -- 172.21.15.123:0/654051492 >> v1:172.21.15.123:6789/0 conn(0x7f75d006c510 legacy=0x7f75d006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:45.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.774+0000 7f75cfe00640 1 -- 172.21.15.123:0/654051492 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d00cf730 con 0x7f75d0071c50 2024-04-24T05:47:45.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.775+0000 7f75cfe00640 1 --2- 172.21.15.123:0/654051492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d0071c50 0x7f75d00ccbb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0097a0 tx=0x7f75bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718eab2454d2e2c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:45.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.776+0000 7f75cf400640 1 -- 172.21.15.123:0/654051492 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc037070 con 0x7f75d0071c50 2024-04-24T05:47:45.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.776+0000 7f75cf400640 1 -- 172.21.15.123:0/654051492 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75bc032440 con 0x7f75d0071c50 2024-04-24T05:47:45.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.776+0000 7f75cf400640 1 -- 172.21.15.123:0/654051492 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc032810 con 0x7f75d0071c50 2024-04-24T05:47:45.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.776+0000 7f75d7600640 1 -- 172.21.15.123:0/654051492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d0071c50 msgr2=0x7f75d00ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:45.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.776+0000 7f75d7600640 1 --2- 172.21.15.123:0/654051492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d0071c50 0x7f75d00ccbb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0097a0 tx=0x7f75bc02a1e0 comp rx=0 tx=0).stop 2024-04-24T05:47:45.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.776+0000 7f75d7600640 1 -- 172.21.15.123:0/654051492 shutdown_connections 2024-04-24T05:47:45.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.776+0000 7f75d7600640 1 --2- 172.21.15.123:0/654051492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d0071c50 0x7f75d00ccbb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:45.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.776+0000 7f75d7600640 1 -- 172.21.15.123:0/654051492 >> 172.21.15.123:0/654051492 conn(0x7f75d00b51f0 msgr2=0x7f75d00b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:45.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.777+0000 7f75d7600640 1 -- 172.21.15.123:0/654051492 shutdown_connections 2024-04-24T05:47:45.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.777+0000 7f75d7600640 1 -- 172.21.15.123:0/654051492 wait complete. 2024-04-24T05:47:45.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.778+0000 7f75d7600640 1 Processor -- start 2024-04-24T05:47:45.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.778+0000 7f75d7600640 1 -- start start 2024-04-24T05:47:45.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.779+0000 7f75d7600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d006c510 0x7f75d01f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:45.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.779+0000 7f75d7600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75d0071c50 0x7f75d006e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:45.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.779+0000 7f75d7600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75d00cd0f0 0x7f75d006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:45.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.779+0000 7f75d7600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f75d006d000 con 0x7f75d006c510 2024-04-24T05:47:45.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.779+0000 7f75d7600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75d00cfcf0 con 0x7f75d00cd0f0 2024-04-24T05:47:45.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.779+0000 7f75d7600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f75d006d180 con 0x7f75d0071c50 2024-04-24T05:47:45.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.779+0000 7f75d4c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d006c510 0x7f75d01f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:45.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.779+0000 7f75d4c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d006c510 0x7f75d01f4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38878/0 (socket says 172.21.15.123:38878) 2024-04-24T05:47:45.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.779+0000 7f75d4c00640 1 -- 172.21.15.123:0/2170509755 learned_addr learned my addr 172.21.15.123:0/2170509755 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:45.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.780+0000 7f75d5600640 1 --2- 172.21.15.123:0/2170509755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75d00cd0f0 0x7f75d006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:45.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.780+0000 7f75d4c00640 1 -- 172.21.15.123:0/2170509755 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75d0071c50 msgr2=0x7f75d006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:45.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.780+0000 7f75d4c00640 1 --2- 172.21.15.123:0/2170509755 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75d0071c50 0x7f75d006e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:45.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.780+0000 7f75d4c00640 1 -- 172.21.15.123:0/2170509755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75d00cd0f0 msgr2=0x7f75d006eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:45.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.780+0000 7f75d4c00640 1 --2- 172.21.15.123:0/2170509755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75d00cd0f0 0x7f75d006eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:45.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.780+0000 7f75d4c00640 1 -- 172.21.15.123:0/2170509755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d01f53b0 con 0x7f75d006c510 2024-04-24T05:47:45.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.780+0000 7f75d5600640 1 --2- 172.21.15.123:0/2170509755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75d00cd0f0 0x7f75d006eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:45.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.780+0000 7f75d4c00640 1 --2- 172.21.15.123:0/2170509755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d006c510 0x7f75d01f4e70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f75c400c430 tx=0x7f75c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:45.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.780+0000 7f75ccc00640 1 -- 172.21.15.123:0/2170509755 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4017070 con 0x7f75d006c510 2024-04-24T05:47:45.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.780+0000 7f75ccc00640 1 -- 172.21.15.123:0/2170509755 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c4005ce0 con 0x7f75d006c510 2024-04-24T05:47:45.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.780+0000 7f75d7600640 1 -- 172.21.15.123:0/2170509755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d00cac30 con 0x7f75d006c510 2024-04-24T05:47:45.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.780+0000 7f75ccc00640 1 -- 172.21.15.123:0/2170509755 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c40072f0 con 0x7f75d006c510 2024-04-24T05:47:45.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.781+0000 7f75d7600640 1 -- 172.21.15.123:0/2170509755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d00cb1f0 con 0x7f75d006c510 2024-04-24T05:47:45.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.782+0000 7f75d7600640 1 -- 172.21.15.123:0/2170509755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d0068a20 con 0x7f75d006c510 2024-04-24T05:47:45.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.783+0000 7f75ccc00640 1 -- 172.21.15.123:0/2170509755 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75c401a070 con 0x7f75d006c510 2024-04-24T05:47:45.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.784+0000 7f75ccc00640 1 --2- 172.21.15.123:0/2170509755 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f75b006c8f0 0x7f75b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:45.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.784+0000 7f75ccc00640 1 -- 172.21.15.123:0/2170509755 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f75c408b240 con 0x7f75d006c510 2024-04-24T05:47:45.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.784+0000 7f75cfe00640 1 --2- 172.21.15.123:0/2170509755 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f75b006c8f0 0x7f75b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:45.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.785+0000 7f75cfe00640 1 --2- 172.21.15.123:0/2170509755 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f75b006c8f0 0x7f75b006edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f75bc034000 tx=0x7f75bc02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:45.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:45.789+0000 7f75ccc00640 1 -- 172.21.15.123:0/2170509755 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f75d01f53b0 con 0x7f75d006c510 2024-04-24T05:47:46.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.029+0000 7f75d7600640 1 -- 172.21.15.123:0/2170509755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f75d00cc5f0 con 0x7f75d006c510 2024-04-24T05:47:46.426 INFO:tasks.ceph.mds.a.smithi123.stderr:2024-04-24T05:47:46.424+0000 7f7cd7c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T05:47:46.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.551+0000 7f75ccc00640 1 -- 172.21.15.123:0/2170509755 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f75c4059440 con 0x7f75d006c510 2024-04-24T05:47:46.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.555+0000 7f75d7600640 1 -- 172.21.15.123:0/2170509755 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f75b006c8f0 msgr2=0x7f75b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:46.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.555+0000 7f75d7600640 1 --2- 172.21.15.123:0/2170509755 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f75b006c8f0 0x7f75b006edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f75bc034000 tx=0x7f75bc02c000 comp rx=0 tx=0).stop 2024-04-24T05:47:46.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.555+0000 7f75d7600640 1 -- 172.21.15.123:0/2170509755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d006c510 msgr2=0x7f75d01f4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:46.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.555+0000 7f75d7600640 1 --2- 172.21.15.123:0/2170509755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d006c510 0x7f75d01f4e70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f75c400c430 tx=0x7f75c400c8f0 comp rx=0 tx=0).stop 2024-04-24T05:47:46.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.555+0000 7f75d7600640 1 -- 172.21.15.123:0/2170509755 shutdown_connections 2024-04-24T05:47:46.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.555+0000 7f75d7600640 1 --2- 172.21.15.123:0/2170509755 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f75b006c8f0 0x7f75b006edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:46.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.555+0000 7f75d7600640 1 --2- 172.21.15.123:0/2170509755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75d00cd0f0 0x7f75d006eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:46.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.555+0000 7f75d7600640 1 --2- 172.21.15.123:0/2170509755 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75d0071c50 0x7f75d006e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:46.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.555+0000 7f75d7600640 1 --2- 172.21.15.123:0/2170509755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d006c510 0x7f75d01f4e70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:46.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.555+0000 7f75d7600640 1 -- 172.21.15.123:0/2170509755 >> 172.21.15.123:0/2170509755 conn(0x7f75d00b51f0 msgr2=0x7f75d006c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:46.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.556+0000 7f75d7600640 1 -- 172.21.15.123:0/2170509755 shutdown_connections 2024-04-24T05:47:46.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.556+0000 7f75d7600640 1 -- 172.21.15.123:0/2170509755 wait complete. 2024-04-24T05:47:46.579 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-24T05:47:46.579 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T05:47:46.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.770+0000 7fb8c7000640 1 Processor -- start 2024-04-24T05:47:46.772 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.771+0000 7fb8c7000640 1 -- start start 2024-04-24T05:47:46.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.772+0000 7fb8c7000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8c00ad860 0x7fb8c00af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:46.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.772+0000 7fb8c7000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb8c00af780 0x7fb8c006bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:46.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.772+0000 7fb8c7000640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8c00afbd0 con 0x7fb8c015a2a0 2024-04-24T05:47:46.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.772+0000 7fb8c7000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c015a680 con 0x7fb8c00af780 2024-04-24T05:47:46.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.772+0000 7fb8c7000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c015da00 con 0x7fb8c00ad860 2024-04-24T05:47:46.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.773+0000 7fb8c5000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb8c00af780 0x7fb8c006bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:46.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.773+0000 7fb8c5000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb8c00af780 0x7fb8c006bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38880/0 (socket says 172.21.15.123:38880) 2024-04-24T05:47:46.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.773+0000 7fb8c5000640 1 -- 172.21.15.123:0/1013238826 learned_addr learned my addr 172.21.15.123:0/1013238826 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:46.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.773+0000 7fb8bf400640 1 --2- 172.21.15.123:0/1013238826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8c00ad860 0x7fb8c00af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:46.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.773+0000 7fb8c5000640 1 -- 172.21.15.123:0/1013238826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8c00ad860 msgr2=0x7fb8c00af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:46.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.773+0000 7fb8c5000640 1 --2- 172.21.15.123:0/1013238826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8c00ad860 0x7fb8c00af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:46.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.773+0000 7fb8c5000640 1 -- 172.21.15.123:0/1013238826 >> v1:172.21.15.155:6789/0 conn(0x7fb8c015a2a0 legacy=0x7fb8c00ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:46.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.773+0000 7fb8c5000640 1 -- 172.21.15.123:0/1013238826 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c015cdf0 con 0x7fb8c00af780 2024-04-24T05:47:46.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.774+0000 7fb8c5000640 1 --2- 172.21.15.123:0/1013238826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c00af780 0x7fb8c006bb00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009870 tx=0x7fb8b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8629244b557933d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:46.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.775+0000 7fb8bea00640 1 -- 172.21.15.123:0/1013238826 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8037070 con 0x7fb8c00af780 2024-04-24T05:47:46.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.775+0000 7fb8bea00640 1 -- 172.21.15.123:0/1013238826 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b8032440 con 0x7fb8c00af780 2024-04-24T05:47:46.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.775+0000 7fb8bea00640 1 -- 172.21.15.123:0/1013238826 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8032810 con 0x7fb8c00af780 2024-04-24T05:47:46.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.776+0000 7fb8c7000640 1 -- 172.21.15.123:0/1013238826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c00af780 msgr2=0x7fb8c006bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:46.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.776+0000 7fb8c7000640 1 --2- 172.21.15.123:0/1013238826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c00af780 0x7fb8c006bb00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009870 tx=0x7fb8b802a180 comp rx=0 tx=0).stop 2024-04-24T05:47:46.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.776+0000 7fb8c7000640 1 -- 172.21.15.123:0/1013238826 shutdown_connections 2024-04-24T05:47:46.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.776+0000 7fb8c7000640 1 --2- 172.21.15.123:0/1013238826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c00af780 0x7fb8c006bb00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:46.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.776+0000 7fb8c7000640 1 --2- 172.21.15.123:0/1013238826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8c00ad860 0x7fb8c00af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:46.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.776+0000 7fb8c7000640 1 -- 172.21.15.123:0/1013238826 >> 172.21.15.123:0/1013238826 conn(0x7fb8c00b1e70 msgr2=0x7fb8c00b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:46.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.776+0000 7fb8c7000640 1 -- 172.21.15.123:0/1013238826 shutdown_connections 2024-04-24T05:47:46.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.776+0000 7fb8c7000640 1 -- 172.21.15.123:0/1013238826 wait complete. 2024-04-24T05:47:46.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.777+0000 7fb8c7000640 1 Processor -- start 2024-04-24T05:47:46.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.778+0000 7fb8c7000640 1 -- start start 2024-04-24T05:47:46.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8c7000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c00ad860 0x7fb8c00cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:46.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8c7000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8c00af780 0x7fb8c01f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:46.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8c7000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8c015a2a0 0x7fb8c00c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:46.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8c7000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c015d690 con 0x7fb8c00ad860 2024-04-24T05:47:46.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8c7000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c015d3b0 con 0x7fb8c015a2a0 2024-04-24T05:47:46.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8c7000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c006c040 con 0x7fb8c00af780 2024-04-24T05:47:46.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8bfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c00ad860 0x7fb8c00cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:46.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8bfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c00ad860 0x7fb8c00cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38892/0 (socket says 172.21.15.123:38892) 2024-04-24T05:47:46.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8bfe00640 1 -- 172.21.15.123:0/2820572128 learned_addr learned my addr 172.21.15.123:0/2820572128 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:46.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8bf400640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8c00af780 0x7fb8c01f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:46.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8bfe00640 1 -- 172.21.15.123:0/2820572128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8c00af780 msgr2=0x7fb8c01f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:46.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8bfe00640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8c00af780 0x7fb8c01f4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:46.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8c5000640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8c015a2a0 0x7fb8c00c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:46.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8bfe00640 1 -- 172.21.15.123:0/2820572128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8c015a2a0 msgr2=0x7fb8c00c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:46.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8bfe00640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8c015a2a0 0x7fb8c00c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:46.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8bfe00640 1 -- 172.21.15.123:0/2820572128 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c0071e80 con 0x7fb8c00ad860 2024-04-24T05:47:46.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.779+0000 7fb8bf400640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8c00af780 0x7fb8c01f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:46.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.780+0000 7fb8bfe00640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c00ad860 0x7fb8c00cc360 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000b420 tx=0x7fb8b000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:46.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.780+0000 7fb8b7e00640 1 -- 172.21.15.123:0/2820572128 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0011830 con 0x7fb8c00ad860 2024-04-24T05:47:46.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.780+0000 7fb8b7e00640 1 -- 172.21.15.123:0/2820572128 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b0016070 con 0x7fb8c00ad860 2024-04-24T05:47:46.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.780+0000 7fb8c7000640 1 -- 172.21.15.123:0/2820572128 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c0072160 con 0x7fb8c00ad860 2024-04-24T05:47:46.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.780+0000 7fb8b7e00640 1 -- 172.21.15.123:0/2820572128 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0011c10 con 0x7fb8c00ad860 2024-04-24T05:47:46.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.780+0000 7fb8c7000640 1 -- 172.21.15.123:0/2820572128 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c0072720 con 0x7fb8c00ad860 2024-04-24T05:47:46.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.781+0000 7fb8c7000640 1 -- 172.21.15.123:0/2820572128 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c0068a20 con 0x7fb8c00ad860 2024-04-24T05:47:46.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.782+0000 7fb8b7e00640 1 -- 172.21.15.123:0/2820572128 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8b001b460 con 0x7fb8c00ad860 2024-04-24T05:47:46.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.783+0000 7fb8b7e00640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb8a006c8f0 0x7fb8a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:46.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.783+0000 7fb8b7e00640 1 -- 172.21.15.123:0/2820572128 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fb8b008b7b0 con 0x7fb8c00ad860 2024-04-24T05:47:46.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.784+0000 7fb8bf400640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb8a006c8f0 0x7fb8a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:46.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.784+0000 7fb8bf400640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb8a006c8f0 0x7fb8a006edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0154610 tx=0x7fb8ac00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:46.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:46.788+0000 7fb8b7e00640 1 -- 172.21.15.123:0/2820572128 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8b0020020 con 0x7fb8c00ad860 2024-04-24T05:47:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.029+0000 7fb8c7000640 1 -- 172.21.15.123:0/2820572128 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8c006fdf0 con 0x7fb8c00ad860 2024-04-24T05:47:47.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.030+0000 7fb8b7e00640 1 -- 172.21.15.123:0/2820572128 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7fb8b0059a40 con 0x7fb8c00ad860 2024-04-24T05:47:47.032 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:47.032 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2481558889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2481558889},{"type":"v1","addr":"172.21.15.155:6833","nonce":2481558889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4915,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3305281906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3305281906},{"type":"v1","addr":"172.21.15.155:6835","nonce":3305281906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1261416511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1261416511},{"type":"v1","addr":"172.21.15.123:6836","nonce":1261416511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T05:47:42.953535+0000","modified":"2024-04-24T05:47:46.031567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5493":{"gid":5493,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6837/3655308480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":3655308480},{"type":"v1","addr":"172.21.15.123:6837","nonce":3655308480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5493,"qdb_cluster":[5493]},"id":2}]} 2024-04-24T05:47:47.032 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 18 2024-04-24T05:47:47.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.033+0000 7fb8c7000640 1 -- 172.21.15.123:0/2820572128 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb8a006c8f0 msgr2=0x7fb8a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:47.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.033+0000 7fb8c7000640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb8a006c8f0 0x7fb8a006edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0154610 tx=0x7fb8ac00b040 comp rx=0 tx=0).stop 2024-04-24T05:47:47.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.033+0000 7fb8c7000640 1 -- 172.21.15.123:0/2820572128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c00ad860 msgr2=0x7fb8c00cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:47.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.033+0000 7fb8c7000640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c00ad860 0x7fb8c00cc360 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000b420 tx=0x7fb8b000b8e0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.033+0000 7fb8c7000640 1 -- 172.21.15.123:0/2820572128 shutdown_connections 2024-04-24T05:47:47.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.033+0000 7fb8c7000640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fb8a006c8f0 0x7fb8a006edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.033+0000 7fb8c7000640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8c015a2a0 0x7fb8c00c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.033+0000 7fb8c7000640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8c00af780 0x7fb8c01f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.033+0000 7fb8c7000640 1 --2- 172.21.15.123:0/2820572128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c00ad860 0x7fb8c00cc360 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.033+0000 7fb8c7000640 1 -- 172.21.15.123:0/2820572128 >> 172.21.15.123:0/2820572128 conn(0x7fb8c00b1e70 msgr2=0x7fb8c006c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:47.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.034+0000 7fb8c7000640 1 -- 172.21.15.123:0/2820572128 shutdown_connections 2024-04-24T05:47:47.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.034+0000 7fb8c7000640 1 -- 172.21.15.123:0/2820572128 wait complete. 2024-04-24T05:47:47.054 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T05:47:47.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.249+0000 7fdd04800640 1 Processor -- start 2024-04-24T05:47:47.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.249+0000 7fdd04800640 1 -- start start 2024-04-24T05:47:47.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.250+0000 7fdd04800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcfc06f090 0x7fdcfc072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.251+0000 7fdd04800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdcfc0b2990 0x7fdcfc073500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.251+0000 7fdd04800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcfc071cd0 con 0x7fdcfc06f470 2024-04-24T05:47:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.251+0000 7fdd04800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdcfc071fb0 con 0x7fdcfc0b2990 2024-04-24T05:47:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.251+0000 7fdd04800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdcfc072290 con 0x7fdcfc06f090 2024-04-24T05:47:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.251+0000 7fdd02800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdcfc0b2990 0x7fdcfc073500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.251+0000 7fdd01400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fdcfc06f470 0x7fdcfc151e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:51164/0 (socket says 172.21.15.123:51164) 2024-04-24T05:47:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.251+0000 7fdd01400640 1 -- 172.21.15.123:0/4138127798 learned_addr learned my addr 172.21.15.123:0/4138127798 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.251+0000 7fdd01e00640 1 --2- 172.21.15.123:0/4138127798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcfc06f090 0x7fdcfc072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.251+0000 7fdd02800640 1 -- 172.21.15.123:0/4138127798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcfc06f090 msgr2=0x7fdcfc072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.251+0000 7fdd02800640 1 --2- 172.21.15.123:0/4138127798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcfc06f090 0x7fdcfc072fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.251+0000 7fdd02800640 1 -- 172.21.15.123:0/4138127798 >> v1:172.21.15.123:6789/0 conn(0x7fdcfc06f470 legacy=0x7fdcfc151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.251+0000 7fdd02800640 1 -- 172.21.15.123:0/4138127798 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcfc072570 con 0x7fdcfc0b2990 2024-04-24T05:47:47.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.251+0000 7fdd00a00640 1 -- 172.21.15.123:0/4138127798 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2038493881 0 0) 0x7fdcfc071cd0 con 0x7fdcfc06f470 2024-04-24T05:47:47.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.251+0000 7fdd01e00640 1 --2- 172.21.15.123:0/4138127798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcfc06f090 0x7fdcfc072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:47.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.253+0000 7fdd02800640 1 --2- 172.21.15.123:0/4138127798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc0b2990 0x7fdcfc073500 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009d90 tx=0x7fdcf802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b390b0dd72573de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:47.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.253+0000 7fdd00a00640 1 -- 172.21.15.123:0/4138127798 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8031600 con 0x7fdcfc0b2990 2024-04-24T05:47:47.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.253+0000 7fdd00a00640 1 -- 172.21.15.123:0/4138127798 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcf80317a0 con 0x7fdcfc0b2990 2024-04-24T05:47:47.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.253+0000 7fdd00a00640 1 -- 172.21.15.123:0/4138127798 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8031b70 con 0x7fdcfc0b2990 2024-04-24T05:47:47.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.253+0000 7fdd04800640 1 -- 172.21.15.123:0/4138127798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc0b2990 msgr2=0x7fdcfc073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:47.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.253+0000 7fdd04800640 1 --2- 172.21.15.123:0/4138127798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc0b2990 0x7fdcfc073500 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009d90 tx=0x7fdcf802a5d0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.253+0000 7fdd04800640 1 -- 172.21.15.123:0/4138127798 shutdown_connections 2024-04-24T05:47:47.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.253+0000 7fdd04800640 1 --2- 172.21.15.123:0/4138127798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc0b2990 0x7fdcfc073500 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.253+0000 7fdd04800640 1 --2- 172.21.15.123:0/4138127798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcfc06f090 0x7fdcfc072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.253+0000 7fdd04800640 1 -- 172.21.15.123:0/4138127798 >> 172.21.15.123:0/4138127798 conn(0x7fdcfc0b1e70 msgr2=0x7fdcfc0b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:47.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.254+0000 7fdd04800640 1 -- 172.21.15.123:0/4138127798 shutdown_connections 2024-04-24T05:47:47.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.254+0000 7fdd04800640 1 -- 172.21.15.123:0/4138127798 wait complete. 2024-04-24T05:47:47.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.255+0000 7fdd04800640 1 Processor -- start 2024-04-24T05:47:47.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.255+0000 7fdd04800640 1 -- start start 2024-04-24T05:47:47.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.256+0000 7fdd04800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcfc06f090 0x7fdcfc1f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:47.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.256+0000 7fdd04800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcfc06f470 0x7fdcfc15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:47.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.256+0000 7fdd04800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc0b2990 0x7fdcfc06bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:47.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.256+0000 7fdd04800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc15c690 con 0x7fdcfc0b2990 2024-04-24T05:47:47.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.256+0000 7fdd04800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc15c510 con 0x7fdcfc06f090 2024-04-24T05:47:47.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.256+0000 7fdd04800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdcfc072a90 con 0x7fdcfc06f470 2024-04-24T05:47:47.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.257+0000 7fdd02800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc0b2990 0x7fdcfc06bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:47.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.257+0000 7fdd02800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc0b2990 0x7fdcfc06bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38904/0 (socket says 172.21.15.123:38904) 2024-04-24T05:47:47.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.257+0000 7fdd02800640 1 -- 172.21.15.123:0/1247264376 learned_addr learned my addr 172.21.15.123:0/1247264376 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:47.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.257+0000 7fdd01400640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcfc06f470 0x7fdcfc15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:47.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.257+0000 7fdd01e00640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcfc06f090 0x7fdcfc1f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:47.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.257+0000 7fdd02800640 1 -- 172.21.15.123:0/1247264376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcfc06f470 msgr2=0x7fdcfc15cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:47.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.257+0000 7fdd02800640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcfc06f470 0x7fdcfc15cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.257+0000 7fdd02800640 1 -- 172.21.15.123:0/1247264376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcfc06f090 msgr2=0x7fdcfc1f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:47.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.257+0000 7fdd02800640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcfc06f090 0x7fdcfc1f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.257+0000 7fdd02800640 1 -- 172.21.15.123:0/1247264376 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcfc15db60 con 0x7fdcfc0b2990 2024-04-24T05:47:47.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.257+0000 7fdd01e00640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcfc06f090 0x7fdcfc1f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:47.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.258+0000 7fdd02800640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc0b2990 0x7fdcfc06bd80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009d90 tx=0x7fdcf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:47.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.258+0000 7fdcf6000640 1 -- 172.21.15.123:0/1247264376 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf803f070 con 0x7fdcfc0b2990 2024-04-24T05:47:47.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.258+0000 7fdcf6000640 1 -- 172.21.15.123:0/1247264376 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcf8030be0 con 0x7fdcfc0b2990 2024-04-24T05:47:47.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.258+0000 7fdd04800640 1 -- 172.21.15.123:0/1247264376 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcfc15e250 con 0x7fdcfc0b2990 2024-04-24T05:47:47.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.258+0000 7fdcf6000640 1 -- 172.21.15.123:0/1247264376 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8031e60 con 0x7fdcfc0b2990 2024-04-24T05:47:47.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.258+0000 7fdd04800640 1 -- 172.21.15.123:0/1247264376 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcfc15d880 con 0x7fdcfc0b2990 2024-04-24T05:47:47.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.259+0000 7fdd04800640 1 -- 172.21.15.123:0/1247264376 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcfc0b0ed0 con 0x7fdcfc0b2990 2024-04-24T05:47:47.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.260+0000 7fdcf6000640 1 -- 172.21.15.123:0/1247264376 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdcf8003990 con 0x7fdcfc0b2990 2024-04-24T05:47:47.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.261+0000 7fdcf6000640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdce006c920 0x7fdce006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:47.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.262+0000 7fdcf6000640 1 -- 172.21.15.123:0/1247264376 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fdcf80ac8c0 con 0x7fdcfc0b2990 2024-04-24T05:47:47.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.262+0000 7fdd01e00640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdce006c920 0x7fdce006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:47.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.262+0000 7fdd01e00640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdce006c920 0x7fdce006ede0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0006fd0 tx=0x7fdcf0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:47.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.266+0000 7fdcf6000640 1 -- 172.21.15.123:0/1247264376 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdcfc0b0ed0 con 0x7fdcfc0b2990 2024-04-24T05:47:47.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.466+0000 7fdd04800640 1 -- 172.21.15.123:0/1247264376 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdcfc068a20 con 0x7fdcfc0b2990 2024-04-24T05:47:47.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.469+0000 7fdcf6000640 1 -- 172.21.15.123:0/1247264376 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16978 (secure 0 0 0) 0x7fdcf807ab50 con 0x7fdcfc0b2990 2024-04-24T05:47:47.471 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:47.472 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":39,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","created":"2024-04-24T05:45:35.376972+0000","modified":"2024-04-24T05:47:44.509877+0000","last_up_change":"2024-04-24T05:45:47.829440+0000","last_in_change":"2024-04-24T05:45:38.647182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T05:45:48.542834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-24T05:47:40.759785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-24T05:47:41.914036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"52b185c5-0e58-4115-b76d-15cf4b46e446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6809","nonce":2060855224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6811","nonce":2060855224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6816","nonce":2060855224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2060855224},{"type":"v1","addr":"172.21.15.123:6813","nonce":2060855224}]},"public_addr":"172.21.15.123:6809/2060855224","cluster_addr":"172.21.15.123:6811/2060855224","heartbeat_back_addr":"172.21.15.123:6816/2060855224","heartbeat_front_addr":"172.21.15.123:6813/2060855224","state":["exists","up"]},{"osd":1,"uuid":"d11108b7-2b27-456b-8fb0-ef134af5b33a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6817","nonce":4191632066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6819","nonce":4191632066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6823","nonce":4191632066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":4191632066},{"type":"v1","addr":"172.21.15.123:6821","nonce":4191632066}]},"public_addr":"172.21.15.123:6817/4191632066","cluster_addr":"172.21.15.123:6819/4191632066","heartbeat_back_addr":"172.21.15.123:6823/4191632066","heartbeat_front_addr":"172.21.15.123:6821/4191632066","state":["exists","up"]},{"osd":2,"uuid":"5419f5d4-1cf7-49c1-a2d8-747c9ba9aebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6801","nonce":1099836695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6803","nonce":1099836695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6807","nonce":1099836695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1099836695},{"type":"v1","addr":"172.21.15.123:6805","nonce":1099836695}]},"public_addr":"172.21.15.123:6801/1099836695","cluster_addr":"172.21.15.123:6803/1099836695","heartbeat_back_addr":"172.21.15.123:6807/1099836695","heartbeat_front_addr":"172.21.15.123:6805/1099836695","state":["exists","up"]},{"osd":3,"uuid":"7ecbd207-67c6-418f-b481-96074db16fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6825","nonce":315591476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6827","nonce":315591476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6831","nonce":315591476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":315591476},{"type":"v1","addr":"172.21.15.123:6829","nonce":315591476}]},"public_addr":"172.21.15.123:6825/315591476","cluster_addr":"172.21.15.123:6827/315591476","heartbeat_back_addr":"172.21.15.123:6831/315591476","heartbeat_front_addr":"172.21.15.123:6829/315591476","state":["exists","up"]},{"osd":4,"uuid":"1fe968d4-69f8-4f2c-a1ad-455c263fd3c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6801","nonce":1404325606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6803","nonce":1404325606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6807","nonce":1404325606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1404325606},{"type":"v1","addr":"172.21.15.155:6805","nonce":1404325606}]},"public_addr":"172.21.15.155:6801/1404325606","cluster_addr":"172.21.15.155:6803/1404325606","heartbeat_back_addr":"172.21.15.155:6807/1404325606","heartbeat_front_addr":"172.21.15.155:6805/1404325606","state":["exists","up"]},{"osd":5,"uuid":"b3f5abd3-ba17-4227-a466-b75c3b10ba79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6825","nonce":2153713758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6827","nonce":2153713758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6831","nonce":2153713758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2153713758},{"type":"v1","addr":"172.21.15.155:6829","nonce":2153713758}]},"public_addr":"172.21.15.155:6825/2153713758","cluster_addr":"172.21.15.155:6827/2153713758","heartbeat_back_addr":"172.21.15.155:6831/2153713758","heartbeat_front_addr":"172.21.15.155:6829/2153713758","state":["exists","up"]},{"osd":6,"uuid":"6973cbac-812d-478e-aec9-add25fd81549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6809","nonce":570528717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6811","nonce":570528717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6815","nonce":570528717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":570528717},{"type":"v1","addr":"172.21.15.155:6813","nonce":570528717}]},"public_addr":"172.21.15.155:6809/570528717","cluster_addr":"172.21.15.155:6811/570528717","heartbeat_back_addr":"172.21.15.155:6815/570528717","heartbeat_front_addr":"172.21.15.155:6813/570528717","state":["exists","up"]},{"osd":7,"uuid":"95ff107b-7dfa-449b-a448-6d3573feb2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6817","nonce":1974539526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6819","nonce":1974539526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6823","nonce":1974539526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":1974539526},{"type":"v1","addr":"172.21.15.155:6821","nonce":1974539526}]},"public_addr":"172.21.15.155:6817/1974539526","cluster_addr":"172.21.15.155:6819/1974539526","heartbeat_back_addr":"172.21.15.155:6823/1974539526","heartbeat_front_addr":"172.21.15.155:6821/1974539526","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.715453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.989110+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.970428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.800168+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.001739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:44.037583+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.976869+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T05:45:43.827820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T05:47:47.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.472+0000 7fdd04800640 1 -- 172.21.15.123:0/1247264376 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdce006c920 msgr2=0x7fdce006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:47.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.472+0000 7fdd04800640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdce006c920 0x7fdce006ede0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0006fd0 tx=0x7fdcf0008040 comp rx=0 tx=0).stop 2024-04-24T05:47:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.472+0000 7fdd04800640 1 -- 172.21.15.123:0/1247264376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc0b2990 msgr2=0x7fdcfc06bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.472+0000 7fdd04800640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc0b2990 0x7fdcfc06bd80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009d90 tx=0x7fdcf8004a40 comp rx=0 tx=0).stop 2024-04-24T05:47:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.473+0000 7fdd04800640 1 -- 172.21.15.123:0/1247264376 shutdown_connections 2024-04-24T05:47:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.473+0000 7fdd04800640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fdce006c920 0x7fdce006ede0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.473+0000 7fdd04800640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc0b2990 0x7fdcfc06bd80 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.473+0000 7fdd04800640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcfc06f470 0x7fdcfc15cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.473+0000 7fdd04800640 1 --2- 172.21.15.123:0/1247264376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcfc06f090 0x7fdcfc1f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.473+0000 7fdd04800640 1 -- 172.21.15.123:0/1247264376 >> 172.21.15.123:0/1247264376 conn(0x7fdcfc0b1e70 msgr2=0x7fdcfc0b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.473+0000 7fdd04800640 1 -- 172.21.15.123:0/1247264376 shutdown_connections 2024-04-24T05:47:47.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.473+0000 7fdd04800640 1 -- 172.21.15.123:0/1247264376 wait complete. 2024-04-24T05:47:47.493 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T05:47:47.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T05:47:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.686+0000 7f5ebb200640 1 Processor -- start 2024-04-24T05:47:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.686+0000 7f5ebb200640 1 -- start start 2024-04-24T05:47:47.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.688+0000 7f5ebb200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5ebc15b5c0 0x7f5ebc0cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:47.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.688+0000 7f5ebb200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ebc06dbe0 0x7f5ebc06e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:47.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.688+0000 7f5ebb200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5ebc0acf40 0x7f5ebc0ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:47.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.688+0000 7f5ebb200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc06e580 con 0x7f5ebc15b5c0 2024-04-24T05:47:47.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.688+0000 7f5ebb200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc0ce4b0 con 0x7f5ebc0acf40 2024-04-24T05:47:47.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.688+0000 7f5ebb200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc0b3000 con 0x7f5ebc06dbe0 2024-04-24T05:47:47.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.688+0000 7f5eb9e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5ebc15b5c0 0x7f5ebc0cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:47.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.688+0000 7f5eb9e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5ebc15b5c0 0x7f5ebc0cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38918/0 (socket says 172.21.15.123:38918) 2024-04-24T05:47:47.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.688+0000 7f5eb9e00640 1 -- 172.21.15.123:0/3456223864 learned_addr learned my addr 172.21.15.123:0/3456223864 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.689+0000 7f5eb9e00640 1 -- 172.21.15.123:0/3456223864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ebc06dbe0 msgr2=0x7f5ebc06e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.689+0000 7f5eb9e00640 1 --2- 172.21.15.123:0/3456223864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ebc06dbe0 0x7f5ebc06e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.689+0000 7f5eba800640 1 --2- 172.21.15.123:0/3456223864 >> v2:172.21.15.155:3300/0 conn(0x7f5ebc0acf40 0x7f5ebc0ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.689+0000 7f5eb9e00640 1 -- 172.21.15.123:0/3456223864 >> v2:172.21.15.155:3300/0 conn(0x7f5ebc0acf40 msgr2=0x7f5ebc0ad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.689+0000 7f5eb9e00640 1 --2- 172.21.15.123:0/3456223864 >> v2:172.21.15.155:3300/0 conn(0x7f5ebc0acf40 0x7f5ebc0ad3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.689+0000 7f5eb9e00640 1 -- 172.21.15.123:0/3456223864 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc06d0c0 con 0x7f5ebc15b5c0 2024-04-24T05:47:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.690+0000 7f5eb9e00640 1 --2- 172.21.15.123:0/3456223864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ebc15b5c0 0x7f5ebc0cdf70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0afb80 tx=0x7f5eac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeebea1fc902cdde server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.690+0000 7f5eb9400640 1 -- 172.21.15.123:0/3456223864 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac037070 con 0x7f5ebc15b5c0 2024-04-24T05:47:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.690+0000 7f5eb9400640 1 -- 172.21.15.123:0/3456223864 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eac032440 con 0x7f5ebc15b5c0 2024-04-24T05:47:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.690+0000 7f5eb9400640 1 -- 172.21.15.123:0/3456223864 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac032810 con 0x7f5ebc15b5c0 2024-04-24T05:47:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.691+0000 7f5ebb200640 1 -- 172.21.15.123:0/3456223864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ebc15b5c0 msgr2=0x7f5ebc0cdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.691+0000 7f5ebb200640 1 --2- 172.21.15.123:0/3456223864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ebc15b5c0 0x7f5ebc0cdf70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0afb80 tx=0x7f5eac02a1e0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.691+0000 7f5ebb200640 1 -- 172.21.15.123:0/3456223864 shutdown_connections 2024-04-24T05:47:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.691+0000 7f5ebb200640 1 --2- 172.21.15.123:0/3456223864 >> v2:172.21.15.155:3300/0 conn(0x7f5ebc0acf40 0x7f5ebc0ad3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.691+0000 7f5ebb200640 1 --2- 172.21.15.123:0/3456223864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ebc06dbe0 0x7f5ebc06e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.691+0000 7f5ebb200640 1 --2- 172.21.15.123:0/3456223864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ebc15b5c0 0x7f5ebc0cdf70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.691+0000 7f5ebb200640 1 -- 172.21.15.123:0/3456223864 >> 172.21.15.123:0/3456223864 conn(0x7f5ebc0d0070 msgr2=0x7f5ebc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.691+0000 7f5ebb200640 1 -- 172.21.15.123:0/3456223864 shutdown_connections 2024-04-24T05:47:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.691+0000 7f5ebb200640 1 -- 172.21.15.123:0/3456223864 wait complete. 2024-04-24T05:47:47.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.693+0000 7f5ebb200640 1 Processor -- start 2024-04-24T05:47:47.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.693+0000 7f5ebb200640 1 -- start start 2024-04-24T05:47:47.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.694+0000 7f5ebb200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ebc06dbe0 0x7f5ebc09d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.694+0000 7f5ebb200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ebc0acf40 0x7f5ebc09bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.694+0000 7f5ebb200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ebc15b5c0 0x7f5ebc152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.694+0000 7f5ebb200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc06cd20 con 0x7f5ebc0acf40 2024-04-24T05:47:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.694+0000 7f5ebb200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0ae510 con 0x7f5ebc06dbe0 2024-04-24T05:47:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.694+0000 7f5ebb200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc06cea0 con 0x7f5ebc15b5c0 2024-04-24T05:47:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.694+0000 7f5eb1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ebc0acf40 0x7f5ebc09bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.694+0000 7f5eb1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ebc0acf40 0x7f5ebc09bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38930/0 (socket says 172.21.15.123:38930) 2024-04-24T05:47:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.694+0000 7f5eb1400640 1 -- 172.21.15.123:0/52020989 learned_addr learned my addr 172.21.15.123:0/52020989 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.695+0000 7f5eb1400640 1 -- 172.21.15.123:0/52020989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ebc15b5c0 msgr2=0x7f5ebc152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.695+0000 7f5eb1400640 1 --2- 172.21.15.123:0/52020989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ebc15b5c0 0x7f5ebc152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.695+0000 7f5eb1400640 1 -- 172.21.15.123:0/52020989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ebc06dbe0 msgr2=0x7f5ebc09d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:47.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.695+0000 7f5eb1400640 1 --2- 172.21.15.123:0/52020989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ebc06dbe0 0x7f5ebc09d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.695+0000 7f5eb1400640 1 -- 172.21.15.123:0/52020989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc09ccd0 con 0x7f5ebc0acf40 2024-04-24T05:47:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.695+0000 7f5eb1400640 1 --2- 172.21.15.123:0/52020989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ebc0acf40 0x7f5ebc09bd20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400b4d0 tx=0x7f5ea400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.695+0000 7f5eb2a00640 1 -- 172.21.15.123:0/52020989 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4014070 con 0x7f5ebc0acf40 2024-04-24T05:47:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.695+0000 7f5eb2a00640 1 -- 172.21.15.123:0/52020989 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea4010870 con 0x7f5ebc0acf40 2024-04-24T05:47:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.695+0000 7f5eb2a00640 1 -- 172.21.15.123:0/52020989 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4010c60 con 0x7f5ebc0acf40 2024-04-24T05:47:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.695+0000 7f5ebb200640 1 -- 172.21.15.123:0/52020989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc09cfb0 con 0x7f5ebc0acf40 2024-04-24T05:47:47.698 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.695+0000 7f5ebb200640 1 -- 172.21.15.123:0/52020989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc09b800 con 0x7f5ebc0acf40 2024-04-24T05:47:47.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.696+0000 7f5ebb200640 1 -- 172.21.15.123:0/52020989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc075b70 con 0x7f5ebc0acf40 2024-04-24T05:47:47.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.697+0000 7f5eb2a00640 1 -- 172.21.15.123:0/52020989 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ea4017070 con 0x7f5ebc0acf40 2024-04-24T05:47:47.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.698+0000 7f5eb2a00640 1 --2- 172.21.15.123:0/52020989 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5e9806c940 0x7f5e9806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:47.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.698+0000 7f5eb2a00640 1 -- 172.21.15.123:0/52020989 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5ea408ba70 con 0x7f5ebc0acf40 2024-04-24T05:47:47.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.699+0000 7f5eb9e00640 1 --2- 172.21.15.123:0/52020989 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5e9806c940 0x7f5e9806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:47.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.699+0000 7f5eb9e00640 1 --2- 172.21.15.123:0/52020989 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5e9806c940 0x7f5e9806ee00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5eac034000 tx=0x7f5eac02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:47.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.703+0000 7f5eb2a00640 1 -- 172.21.15.123:0/52020989 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ea4059d80 con 0x7f5ebc0acf40 2024-04-24T05:47:47.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.898+0000 7f5ebb200640 1 -- 172.21.15.123:0/52020989 --> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ebc06f890 con 0x7f5e9806c940 2024-04-24T05:47:47.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.919+0000 7f5eb2a00640 1 -- 172.21.15.123:0/52020989 <== mgr.4101 v2:172.21.15.123:6832/1503718323 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251886 (secure 0 0 0) 0x7f5ebc06f890 con 0x7f5e9806c940 2024-04-24T05:47:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-24T05:47:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.923+0000 7f5ebb200640 1 -- 172.21.15.123:0/52020989 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5e9806c940 msgr2=0x7f5e9806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.923+0000 7f5ebb200640 1 --2- 172.21.15.123:0/52020989 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5e9806c940 0x7f5e9806ee00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5eac034000 tx=0x7f5eac02c000 comp rx=0 tx=0).stop 2024-04-24T05:47:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.923+0000 7f5ebb200640 1 -- 172.21.15.123:0/52020989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ebc0acf40 msgr2=0x7f5ebc09bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.923+0000 7f5ebb200640 1 --2- 172.21.15.123:0/52020989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ebc0acf40 0x7f5ebc09bd20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400b4d0 tx=0x7f5ea400b990 comp rx=0 tx=0).stop 2024-04-24T05:47:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.924+0000 7f5ebb200640 1 -- 172.21.15.123:0/52020989 shutdown_connections 2024-04-24T05:47:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.924+0000 7f5ebb200640 1 --2- 172.21.15.123:0/52020989 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f5e9806c940 0x7f5e9806ee00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.924+0000 7f5ebb200640 1 --2- 172.21.15.123:0/52020989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ebc15b5c0 0x7f5ebc152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.924+0000 7f5ebb200640 1 --2- 172.21.15.123:0/52020989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ebc0acf40 0x7f5ebc09bd20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.924+0000 7f5ebb200640 1 --2- 172.21.15.123:0/52020989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ebc06dbe0 0x7f5ebc09d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:47.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.924+0000 7f5ebb200640 1 -- 172.21.15.123:0/52020989 >> 172.21.15.123:0/52020989 conn(0x7f5ebc0d0070 msgr2=0x7f5ebc0b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:47.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.924+0000 7f5ebb200640 1 -- 172.21.15.123:0/52020989 shutdown_connections 2024-04-24T05:47:47.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:47.924+0000 7f5ebb200640 1 -- 172.21.15.123:0/52020989 wait complete. 2024-04-24T05:47:47.945 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-04-24T05:47:46.546829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220672,"kb_used_data":4488,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754048,"statfs":{"total":773094113280,"available":772868145152,"internally_reserved":0,"allocated":4595712,"data_stored":2630800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.071176"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.535759+0000","last_change":"2024-04-24T05:47:43.532026+0000","last_active":"2024-04-24T05:47:43.535759+0000","last_peered":"2024-04-24T05:47:43.535759+0000","last_clean":"2024-04-24T05:47:43.535759+0000","last_became_active":"2024-04-24T05:47:43.531433+0000","last_became_peered":"2024-04-24T05:47:43.531433+0000","last_unstale":"2024-04-24T05:47:43.535759+0000","last_undegraded":"2024-04-24T05:47:43.535759+0000","last_fullsized":"2024-04-24T05:47:43.535759+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:45.002651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.546733+0000","last_change":"2024-04-24T05:47:43.541013+0000","last_active":"2024-04-24T05:47:43.546733+0000","last_peered":"2024-04-24T05:47:43.546733+0000","last_clean":"2024-04-24T05:47:43.546733+0000","last_became_active":"2024-04-24T05:47:43.540782+0000","last_became_peered":"2024-04-24T05:47:43.540782+0000","last_unstale":"2024-04-24T05:47:43.546733+0000","last_undegraded":"2024-04-24T05:47:43.546733+0000","last_fullsized":"2024-04-24T05:47:43.546733+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:49.952705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.555742+0000","last_change":"2024-04-24T05:47:43.550470+0000","last_active":"2024-04-24T05:47:43.555742+0000","last_peered":"2024-04-24T05:47:43.555742+0000","last_clean":"2024-04-24T05:47:43.555742+0000","last_became_active":"2024-04-24T05:47:43.550183+0000","last_became_peered":"2024-04-24T05:47:43.550183+0000","last_unstale":"2024-04-24T05:47:43.555742+0000","last_undegraded":"2024-04-24T05:47:43.555742+0000","last_fullsized":"2024-04-24T05:47:43.555742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:14:18.610223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544580+0000","last_change":"2024-04-24T05:47:43.534882+0000","last_active":"2024-04-24T05:47:43.544580+0000","last_peered":"2024-04-24T05:47:43.544580+0000","last_clean":"2024-04-24T05:47:43.544580+0000","last_became_active":"2024-04-24T05:47:43.534577+0000","last_became_peered":"2024-04-24T05:47:43.534577+0000","last_unstale":"2024-04-24T05:47:43.544580+0000","last_undegraded":"2024-04-24T05:47:43.544580+0000","last_fullsized":"2024-04-24T05:47:43.544580+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:01:42.716524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.543970+0000","last_change":"2024-04-24T05:47:43.530606+0000","last_active":"2024-04-24T05:47:43.543970+0000","last_peered":"2024-04-24T05:47:43.543970+0000","last_clean":"2024-04-24T05:47:43.543970+0000","last_became_active":"2024-04-24T05:47:43.529738+0000","last_became_peered":"2024-04-24T05:47:43.529738+0000","last_unstale":"2024-04-24T05:47:43.543970+0000","last_undegraded":"2024-04-24T05:47:43.543970+0000","last_fullsized":"2024-04-24T05:47:43.543970+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:41:45.562637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.543066+0000","last_change":"2024-04-24T05:47:43.537110+0000","last_active":"2024-04-24T05:47:43.543066+0000","last_peered":"2024-04-24T05:47:43.543066+0000","last_clean":"2024-04-24T05:47:43.543066+0000","last_became_active":"2024-04-24T05:47:43.536827+0000","last_became_peered":"2024-04-24T05:47:43.536827+0000","last_unstale":"2024-04-24T05:47:43.543066+0000","last_undegraded":"2024-04-24T05:47:43.543066+0000","last_fullsized":"2024-04-24T05:47:43.543066+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:45.002651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544096+0000","last_change":"2024-04-24T05:47:43.539466+0000","last_active":"2024-04-24T05:47:43.544096+0000","last_peered":"2024-04-24T05:47:43.544096+0000","last_clean":"2024-04-24T05:47:43.544096+0000","last_became_active":"2024-04-24T05:47:43.539108+0000","last_became_peered":"2024-04-24T05:47:43.539108+0000","last_unstale":"2024-04-24T05:47:43.544096+0000","last_undegraded":"2024-04-24T05:47:43.544096+0000","last_fullsized":"2024-04-24T05:47:43.544096+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:49.952705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.546435+0000","last_change":"2024-04-24T05:47:43.538288+0000","last_active":"2024-04-24T05:47:43.546435+0000","last_peered":"2024-04-24T05:47:43.546435+0000","last_clean":"2024-04-24T05:47:43.546435+0000","last_became_active":"2024-04-24T05:47:43.537994+0000","last_became_peered":"2024-04-24T05:47:43.537994+0000","last_unstale":"2024-04-24T05:47:43.546435+0000","last_undegraded":"2024-04-24T05:47:43.546435+0000","last_fullsized":"2024-04-24T05:47:43.546435+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:34:32.710562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.543528+0000","last_change":"2024-04-24T05:47:43.538047+0000","last_active":"2024-04-24T05:47:43.543528+0000","last_peered":"2024-04-24T05:47:43.543528+0000","last_clean":"2024-04-24T05:47:43.543528+0000","last_became_active":"2024-04-24T05:47:43.535743+0000","last_became_peered":"2024-04-24T05:47:43.535743+0000","last_unstale":"2024-04-24T05:47:43.543528+0000","last_undegraded":"2024-04-24T05:47:43.543528+0000","last_fullsized":"2024-04-24T05:47:43.543528+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:45.002651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.547329+0000","last_change":"2024-04-24T05:47:43.541239+0000","last_active":"2024-04-24T05:47:43.547329+0000","last_peered":"2024-04-24T05:47:43.547329+0000","last_clean":"2024-04-24T05:47:43.547329+0000","last_became_active":"2024-04-24T05:47:43.540936+0000","last_became_peered":"2024-04-24T05:47:43.540936+0000","last_unstale":"2024-04-24T05:47:43.547329+0000","last_undegraded":"2024-04-24T05:47:43.547329+0000","last_fullsized":"2024-04-24T05:47:43.547329+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:15.413070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.538344+0000","last_change":"2024-04-24T05:47:43.534409+0000","last_active":"2024-04-24T05:47:43.538344+0000","last_peered":"2024-04-24T05:47:43.538344+0000","last_clean":"2024-04-24T05:47:43.538344+0000","last_became_active":"2024-04-24T05:47:43.534123+0000","last_became_peered":"2024-04-24T05:47:43.534123+0000","last_unstale":"2024-04-24T05:47:43.538344+0000","last_undegraded":"2024-04-24T05:47:43.538344+0000","last_fullsized":"2024-04-24T05:47:43.538344+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:18:56.906868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.524466+0000","last_change":"2024-04-24T05:47:43.519430+0000","last_active":"2024-04-24T05:47:43.524466+0000","last_peered":"2024-04-24T05:47:43.524466+0000","last_clean":"2024-04-24T05:47:43.524466+0000","last_became_active":"2024-04-24T05:47:43.519133+0000","last_became_peered":"2024-04-24T05:47:43.519133+0000","last_unstale":"2024-04-24T05:47:43.524466+0000","last_undegraded":"2024-04-24T05:47:43.524466+0000","last_fullsized":"2024-04-24T05:47:43.524466+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:57.673598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.546336+0000","last_change":"2024-04-24T05:47:43.540156+0000","last_active":"2024-04-24T05:47:43.546336+0000","last_peered":"2024-04-24T05:47:43.546336+0000","last_clean":"2024-04-24T05:47:43.546336+0000","last_became_active":"2024-04-24T05:47:43.539866+0000","last_became_peered":"2024-04-24T05:47:43.539866+0000","last_unstale":"2024-04-24T05:47:43.546336+0000","last_undegraded":"2024-04-24T05:47:43.546336+0000","last_fullsized":"2024-04-24T05:47:43.546336+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:17:36.426617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.545197+0000","last_change":"2024-04-24T05:47:43.536776+0000","last_active":"2024-04-24T05:47:43.545197+0000","last_peered":"2024-04-24T05:47:43.545197+0000","last_clean":"2024-04-24T05:47:43.545197+0000","last_became_active":"2024-04-24T05:47:43.536496+0000","last_became_peered":"2024-04-24T05:47:43.536496+0000","last_unstale":"2024-04-24T05:47:43.545197+0000","last_undegraded":"2024-04-24T05:47:43.545197+0000","last_fullsized":"2024-04-24T05:47:43.545197+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:16.473137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.540646+0000","last_change":"2024-04-24T05:47:43.533184+0000","last_active":"2024-04-24T05:47:43.540646+0000","last_peered":"2024-04-24T05:47:43.540646+0000","last_clean":"2024-04-24T05:47:43.540646+0000","last_became_active":"2024-04-24T05:47:43.532856+0000","last_became_peered":"2024-04-24T05:47:43.532856+0000","last_unstale":"2024-04-24T05:47:43.540646+0000","last_undegraded":"2024-04-24T05:47:43.540646+0000","last_fullsized":"2024-04-24T05:47:43.540646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:54.283092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.547416+0000","last_change":"2024-04-24T05:47:43.539182+0000","last_active":"2024-04-24T05:47:43.547416+0000","last_peered":"2024-04-24T05:47:43.547416+0000","last_clean":"2024-04-24T05:47:43.547416+0000","last_became_active":"2024-04-24T05:47:43.538891+0000","last_became_peered":"2024-04-24T05:47:43.538891+0000","last_unstale":"2024-04-24T05:47:43.547416+0000","last_undegraded":"2024-04-24T05:47:43.547416+0000","last_fullsized":"2024-04-24T05:47:43.547416+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:49:26.443913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.547008+0000","last_change":"2024-04-24T05:47:43.541268+0000","last_active":"2024-04-24T05:47:43.547008+0000","last_peered":"2024-04-24T05:47:43.547008+0000","last_clean":"2024-04-24T05:47:43.547008+0000","last_became_active":"2024-04-24T05:47:43.539493+0000","last_became_peered":"2024-04-24T05:47:43.539493+0000","last_unstale":"2024-04-24T05:47:43.547008+0000","last_undegraded":"2024-04-24T05:47:43.547008+0000","last_fullsized":"2024-04-24T05:47:43.547008+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:54.283092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.549038+0000","last_change":"2024-04-24T05:47:43.544582+0000","last_active":"2024-04-24T05:47:43.549038+0000","last_peered":"2024-04-24T05:47:43.549038+0000","last_clean":"2024-04-24T05:47:43.549038+0000","last_became_active":"2024-04-24T05:47:43.540995+0000","last_became_peered":"2024-04-24T05:47:43.540995+0000","last_unstale":"2024-04-24T05:47:43.549038+0000","last_undegraded":"2024-04-24T05:47:43.549038+0000","last_fullsized":"2024-04-24T05:47:43.549038+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:57.673598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544418+0000","last_change":"2024-04-24T05:47:43.534229+0000","last_active":"2024-04-24T05:47:43.544418+0000","last_peered":"2024-04-24T05:47:43.544418+0000","last_clean":"2024-04-24T05:47:43.544418+0000","last_became_active":"2024-04-24T05:47:43.530596+0000","last_became_peered":"2024-04-24T05:47:43.530596+0000","last_unstale":"2024-04-24T05:47:43.544418+0000","last_undegraded":"2024-04-24T05:47:43.544418+0000","last_fullsized":"2024-04-24T05:47:43.544418+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:10:29.082247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.543634+0000","last_change":"2024-04-24T05:47:43.537580+0000","last_active":"2024-04-24T05:47:43.543634+0000","last_peered":"2024-04-24T05:47:43.543634+0000","last_clean":"2024-04-24T05:47:43.543634+0000","last_became_active":"2024-04-24T05:47:43.537253+0000","last_became_peered":"2024-04-24T05:47:43.537253+0000","last_unstale":"2024-04-24T05:47:43.543634+0000","last_undegraded":"2024-04-24T05:47:43.543634+0000","last_fullsized":"2024-04-24T05:47:43.543634+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:29.306299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.538263+0000","last_change":"2024-04-24T05:47:43.530351+0000","last_active":"2024-04-24T05:47:43.538263+0000","last_peered":"2024-04-24T05:47:43.538263+0000","last_clean":"2024-04-24T05:47:43.538263+0000","last_became_active":"2024-04-24T05:47:43.529857+0000","last_became_peered":"2024-04-24T05:47:43.529857+0000","last_unstale":"2024-04-24T05:47:43.538263+0000","last_undegraded":"2024-04-24T05:47:43.538263+0000","last_fullsized":"2024-04-24T05:47:43.538263+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:45.002651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544973+0000","last_change":"2024-04-24T05:47:43.533746+0000","last_active":"2024-04-24T05:47:43.544973+0000","last_peered":"2024-04-24T05:47:43.544973+0000","last_clean":"2024-04-24T05:47:43.544973+0000","last_became_active":"2024-04-24T05:47:43.532859+0000","last_became_peered":"2024-04-24T05:47:43.532859+0000","last_unstale":"2024-04-24T05:47:43.544973+0000","last_undegraded":"2024-04-24T05:47:43.544973+0000","last_fullsized":"2024-04-24T05:47:43.544973+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:22:52.410342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.546638+0000","last_change":"2024-04-24T05:47:43.541070+0000","last_active":"2024-04-24T05:47:43.546638+0000","last_peered":"2024-04-24T05:47:43.546638+0000","last_clean":"2024-04-24T05:47:43.546638+0000","last_became_active":"2024-04-24T05:47:43.540780+0000","last_became_peered":"2024-04-24T05:47:43.540780+0000","last_unstale":"2024-04-24T05:47:43.546638+0000","last_undegraded":"2024-04-24T05:47:43.546638+0000","last_fullsized":"2024-04-24T05:47:43.546638+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:14:18.610223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544065+0000","last_change":"2024-04-24T05:47:43.528015+0000","last_active":"2024-04-24T05:47:43.544065+0000","last_peered":"2024-04-24T05:47:43.544065+0000","last_clean":"2024-04-24T05:47:43.544065+0000","last_became_active":"2024-04-24T05:47:43.527684+0000","last_became_peered":"2024-04-24T05:47:43.527684+0000","last_unstale":"2024-04-24T05:47:43.544065+0000","last_undegraded":"2024-04-24T05:47:43.544065+0000","last_fullsized":"2024-04-24T05:47:43.544065+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:47.771923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.530526+0000","last_change":"2024-04-24T05:47:43.523663+0000","last_active":"2024-04-24T05:47:43.530526+0000","last_peered":"2024-04-24T05:47:43.530526+0000","last_clean":"2024-04-24T05:47:43.530526+0000","last_became_active":"2024-04-24T05:47:43.523266+0000","last_became_peered":"2024-04-24T05:47:43.523266+0000","last_unstale":"2024-04-24T05:47:43.530526+0000","last_undegraded":"2024-04-24T05:47:43.530526+0000","last_fullsized":"2024-04-24T05:47:43.530526+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:51:08.032102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.543004+0000","last_change":"2024-04-24T05:47:43.536242+0000","last_active":"2024-04-24T05:47:43.543004+0000","last_peered":"2024-04-24T05:47:43.543004+0000","last_clean":"2024-04-24T05:47:43.543004+0000","last_became_active":"2024-04-24T05:47:43.535954+0000","last_became_peered":"2024-04-24T05:47:43.535954+0000","last_unstale":"2024-04-24T05:47:43.543004+0000","last_undegraded":"2024-04-24T05:47:43.543004+0000","last_fullsized":"2024-04-24T05:47:43.543004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:59.126614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.530309+0000","last_change":"2024-04-24T05:47:43.520366+0000","last_active":"2024-04-24T05:47:43.530309+0000","last_peered":"2024-04-24T05:47:43.530309+0000","last_clean":"2024-04-24T05:47:43.530309+0000","last_became_active":"2024-04-24T05:47:43.519932+0000","last_became_peered":"2024-04-24T05:47:43.519932+0000","last_unstale":"2024-04-24T05:47:43.530309+0000","last_undegraded":"2024-04-24T05:47:43.530309+0000","last_fullsized":"2024-04-24T05:47:43.530309+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:01.335880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.548536+0000","last_change":"2024-04-24T05:47:43.538735+0000","last_active":"2024-04-24T05:47:43.548536+0000","last_peered":"2024-04-24T05:47:43.548536+0000","last_clean":"2024-04-24T05:47:43.548536+0000","last_became_active":"2024-04-24T05:47:43.538428+0000","last_became_peered":"2024-04-24T05:47:43.538428+0000","last_unstale":"2024-04-24T05:47:43.548536+0000","last_undegraded":"2024-04-24T05:47:43.548536+0000","last_fullsized":"2024-04-24T05:47:43.548536+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:21:10.691766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.538132+0000","last_change":"2024-04-24T05:47:43.531542+0000","last_active":"2024-04-24T05:47:43.538132+0000","last_peered":"2024-04-24T05:47:43.538132+0000","last_clean":"2024-04-24T05:47:43.538132+0000","last_became_active":"2024-04-24T05:47:43.531200+0000","last_became_peered":"2024-04-24T05:47:43.531200+0000","last_unstale":"2024-04-24T05:47:43.538132+0000","last_undegraded":"2024-04-24T05:47:43.538132+0000","last_fullsized":"2024-04-24T05:47:43.538132+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:22:52.410342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.538482+0000","last_change":"2024-04-24T05:47:43.531985+0000","last_active":"2024-04-24T05:47:43.538482+0000","last_peered":"2024-04-24T05:47:43.538482+0000","last_clean":"2024-04-24T05:47:43.538482+0000","last_became_active":"2024-04-24T05:47:43.530041+0000","last_became_peered":"2024-04-24T05:47:43.530041+0000","last_unstale":"2024-04-24T05:47:43.538482+0000","last_undegraded":"2024-04-24T05:47:43.538482+0000","last_fullsized":"2024-04-24T05:47:43.538482+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:09.647755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.548317+0000","last_change":"2024-04-24T05:47:43.546219+0000","last_active":"2024-04-24T05:47:43.548317+0000","last_peered":"2024-04-24T05:47:43.548317+0000","last_clean":"2024-04-24T05:47:43.548317+0000","last_became_active":"2024-04-24T05:47:43.545895+0000","last_became_peered":"2024-04-24T05:47:43.545895+0000","last_unstale":"2024-04-24T05:47:43.548317+0000","last_undegraded":"2024-04-24T05:47:43.548317+0000","last_fullsized":"2024-04-24T05:47:43.548317+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:21:10.691766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.548318+0000","last_change":"2024-04-24T05:47:43.545361+0000","last_active":"2024-04-24T05:47:43.548318+0000","last_peered":"2024-04-24T05:47:43.548318+0000","last_clean":"2024-04-24T05:47:43.548318+0000","last_became_active":"2024-04-24T05:47:43.545062+0000","last_became_peered":"2024-04-24T05:47:43.545062+0000","last_unstale":"2024-04-24T05:47:43.548318+0000","last_undegraded":"2024-04-24T05:47:43.548318+0000","last_fullsized":"2024-04-24T05:47:43.548318+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:49:26.443913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.554415+0000","last_change":"2024-04-24T05:47:43.550870+0000","last_active":"2024-04-24T05:47:43.554415+0000","last_peered":"2024-04-24T05:47:43.554415+0000","last_clean":"2024-04-24T05:47:43.554415+0000","last_became_active":"2024-04-24T05:47:43.550499+0000","last_became_peered":"2024-04-24T05:47:43.550499+0000","last_unstale":"2024-04-24T05:47:43.554415+0000","last_undegraded":"2024-04-24T05:47:43.554415+0000","last_fullsized":"2024-04-24T05:47:43.554415+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:49.952705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.553172+0000","last_change":"2024-04-24T05:47:42.530065+0000","last_active":"2024-04-24T05:47:43.553172+0000","last_peered":"2024-04-24T05:47:43.553172+0000","last_clean":"2024-04-24T05:47:43.553172+0000","last_became_active":"2024-04-24T05:47:42.529676+0000","last_became_peered":"2024-04-24T05:47:42.529676+0000","last_unstale":"2024-04-24T05:47:43.553172+0000","last_undegraded":"2024-04-24T05:47:43.553172+0000","last_fullsized":"2024-04-24T05:47:43.553172+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:36:14.055524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.553365+0000","last_change":"2024-04-24T05:47:43.549633+0000","last_active":"2024-04-24T05:47:43.553365+0000","last_peered":"2024-04-24T05:47:43.553365+0000","last_clean":"2024-04-24T05:47:43.553365+0000","last_became_active":"2024-04-24T05:47:43.549249+0000","last_became_peered":"2024-04-24T05:47:43.549249+0000","last_unstale":"2024-04-24T05:47:43.553365+0000","last_undegraded":"2024-04-24T05:47:43.553365+0000","last_fullsized":"2024-04-24T05:47:43.553365+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:14:18.610223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544392+0000","last_change":"2024-04-24T05:47:42.526660+0000","last_active":"2024-04-24T05:47:43.544392+0000","last_peered":"2024-04-24T05:47:43.544392+0000","last_clean":"2024-04-24T05:47:43.544392+0000","last_became_active":"2024-04-24T05:47:42.526345+0000","last_became_peered":"2024-04-24T05:47:42.526345+0000","last_unstale":"2024-04-24T05:47:43.544392+0000","last_undegraded":"2024-04-24T05:47:43.544392+0000","last_fullsized":"2024-04-24T05:47:43.544392+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:08.633627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.545045+0000","last_change":"2024-04-24T05:47:43.535651+0000","last_active":"2024-04-24T05:47:43.545045+0000","last_peered":"2024-04-24T05:47:43.545045+0000","last_clean":"2024-04-24T05:47:43.545045+0000","last_became_active":"2024-04-24T05:47:43.535349+0000","last_became_peered":"2024-04-24T05:47:43.535349+0000","last_unstale":"2024-04-24T05:47:43.545045+0000","last_undegraded":"2024-04-24T05:47:43.545045+0000","last_fullsized":"2024-04-24T05:47:43.545045+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:57.673598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.547922+0000","last_change":"2024-04-24T05:47:42.535682+0000","last_active":"2024-04-24T05:47:43.547922+0000","last_peered":"2024-04-24T05:47:43.547922+0000","last_clean":"2024-04-24T05:47:43.547922+0000","last_became_active":"2024-04-24T05:47:42.535382+0000","last_became_peered":"2024-04-24T05:47:42.535382+0000","last_unstale":"2024-04-24T05:47:43.547922+0000","last_undegraded":"2024-04-24T05:47:43.547922+0000","last_fullsized":"2024-04-24T05:47:43.547922+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:15.459009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.546470+0000","last_change":"2024-04-24T05:47:43.540242+0000","last_active":"2024-04-24T05:47:43.546470+0000","last_peered":"2024-04-24T05:47:43.546470+0000","last_clean":"2024-04-24T05:47:43.546470+0000","last_became_active":"2024-04-24T05:47:43.537526+0000","last_became_peered":"2024-04-24T05:47:43.537526+0000","last_unstale":"2024-04-24T05:47:43.546470+0000","last_undegraded":"2024-04-24T05:47:43.546470+0000","last_fullsized":"2024-04-24T05:47:43.546470+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:25:10.115913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.554419+0000","last_change":"2024-04-24T05:47:42.532915+0000","last_active":"2024-04-24T05:47:43.554419+0000","last_peered":"2024-04-24T05:47:43.554419+0000","last_clean":"2024-04-24T05:47:43.554419+0000","last_became_active":"2024-04-24T05:47:42.532614+0000","last_became_peered":"2024-04-24T05:47:42.532614+0000","last_unstale":"2024-04-24T05:47:43.554419+0000","last_undegraded":"2024-04-24T05:47:43.554419+0000","last_fullsized":"2024-04-24T05:47:43.554419+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:36:14.055524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.537214+0000","last_change":"2024-04-24T05:47:43.529365+0000","last_active":"2024-04-24T05:47:43.537214+0000","last_peered":"2024-04-24T05:47:43.537214+0000","last_clean":"2024-04-24T05:47:43.537214+0000","last_became_active":"2024-04-24T05:47:43.528243+0000","last_became_peered":"2024-04-24T05:47:43.528243+0000","last_unstale":"2024-04-24T05:47:43.537214+0000","last_undegraded":"2024-04-24T05:47:43.537214+0000","last_fullsized":"2024-04-24T05:47:43.537214+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:16.473137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.553435+0000","last_change":"2024-04-24T05:47:42.525848+0000","last_active":"2024-04-24T05:47:43.553435+0000","last_peered":"2024-04-24T05:47:43.553435+0000","last_clean":"2024-04-24T05:47:43.553435+0000","last_became_active":"2024-04-24T05:47:42.525150+0000","last_became_peered":"2024-04-24T05:47:42.525150+0000","last_unstale":"2024-04-24T05:47:43.553435+0000","last_undegraded":"2024-04-24T05:47:43.553435+0000","last_fullsized":"2024-04-24T05:47:43.553435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:58.112486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.540624+0000","last_change":"2024-04-24T05:47:43.532909+0000","last_active":"2024-04-24T05:47:43.540624+0000","last_peered":"2024-04-24T05:47:43.540624+0000","last_clean":"2024-04-24T05:47:43.540624+0000","last_became_active":"2024-04-24T05:47:43.532496+0000","last_became_peered":"2024-04-24T05:47:43.532496+0000","last_unstale":"2024-04-24T05:47:43.540624+0000","last_undegraded":"2024-04-24T05:47:43.540624+0000","last_fullsized":"2024-04-24T05:47:43.540624+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:14:18.610223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.554422+0000","last_change":"2024-04-24T05:47:42.536293+0000","last_active":"2024-04-24T05:47:43.554422+0000","last_peered":"2024-04-24T05:47:43.554422+0000","last_clean":"2024-04-24T05:47:43.554422+0000","last_became_active":"2024-04-24T05:47:42.535824+0000","last_became_peered":"2024-04-24T05:47:42.535824+0000","last_unstale":"2024-04-24T05:47:43.554422+0000","last_undegraded":"2024-04-24T05:47:43.554422+0000","last_fullsized":"2024-04-24T05:47:43.554422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:54:18.237837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.545348+0000","last_change":"2024-04-24T05:47:43.539477+0000","last_active":"2024-04-24T05:47:43.545348+0000","last_peered":"2024-04-24T05:47:43.545348+0000","last_clean":"2024-04-24T05:47:43.545348+0000","last_became_active":"2024-04-24T05:47:43.536101+0000","last_became_peered":"2024-04-24T05:47:43.536101+0000","last_unstale":"2024-04-24T05:47:43.545348+0000","last_undegraded":"2024-04-24T05:47:43.545348+0000","last_fullsized":"2024-04-24T05:47:43.545348+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:54:19.251964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.521139+0000","last_change":"2024-04-24T05:47:42.526187+0000","last_active":"2024-04-24T05:47:43.521139+0000","last_peered":"2024-04-24T05:47:43.521139+0000","last_clean":"2024-04-24T05:47:43.521139+0000","last_became_active":"2024-04-24T05:47:42.525891+0000","last_became_peered":"2024-04-24T05:47:42.525891+0000","last_unstale":"2024-04-24T05:47:43.521139+0000","last_undegraded":"2024-04-24T05:47:43.521139+0000","last_fullsized":"2024-04-24T05:47:43.521139+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:08.633627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.550965+0000","last_change":"2024-04-24T05:47:43.539971+0000","last_active":"2024-04-24T05:47:43.550965+0000","last_peered":"2024-04-24T05:47:43.550965+0000","last_clean":"2024-04-24T05:47:43.550965+0000","last_became_active":"2024-04-24T05:47:43.539203+0000","last_became_peered":"2024-04-24T05:47:43.539203+0000","last_unstale":"2024-04-24T05:47:43.550965+0000","last_undegraded":"2024-04-24T05:47:43.550965+0000","last_fullsized":"2024-04-24T05:47:43.550965+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:21:10.691766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.516582+0000","last_change":"2024-04-24T05:47:42.530677+0000","last_active":"2024-04-24T05:47:43.516582+0000","last_peered":"2024-04-24T05:47:43.516582+0000","last_clean":"2024-04-24T05:47:43.516582+0000","last_became_active":"2024-04-24T05:47:42.530324+0000","last_became_peered":"2024-04-24T05:47:42.530324+0000","last_unstale":"2024-04-24T05:47:43.516582+0000","last_undegraded":"2024-04-24T05:47:43.516582+0000","last_fullsized":"2024-04-24T05:47:43.516582+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:43:36.839599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544015+0000","last_change":"2024-04-24T05:47:43.531801+0000","last_active":"2024-04-24T05:47:43.544015+0000","last_peered":"2024-04-24T05:47:43.544015+0000","last_clean":"2024-04-24T05:47:43.544015+0000","last_became_active":"2024-04-24T05:47:43.531453+0000","last_became_peered":"2024-04-24T05:47:43.531453+0000","last_unstale":"2024-04-24T05:47:43.544015+0000","last_undegraded":"2024-04-24T05:47:43.544015+0000","last_fullsized":"2024-04-24T05:47:43.544015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:00:50.989959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.525208+0000","last_change":"2024-04-24T05:47:42.520876+0000","last_active":"2024-04-24T05:47:43.525208+0000","last_peered":"2024-04-24T05:47:43.525208+0000","last_clean":"2024-04-24T05:47:43.525208+0000","last_became_active":"2024-04-24T05:47:42.520589+0000","last_became_peered":"2024-04-24T05:47:42.520589+0000","last_unstale":"2024-04-24T05:47:43.525208+0000","last_undegraded":"2024-04-24T05:47:43.525208+0000","last_fullsized":"2024-04-24T05:47:43.525208+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:25.180697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.547190+0000","last_change":"2024-04-24T05:47:43.543175+0000","last_active":"2024-04-24T05:47:43.547190+0000","last_peered":"2024-04-24T05:47:43.547190+0000","last_clean":"2024-04-24T05:47:43.547190+0000","last_became_active":"2024-04-24T05:47:43.540460+0000","last_became_peered":"2024-04-24T05:47:43.540460+0000","last_unstale":"2024-04-24T05:47:43.547190+0000","last_undegraded":"2024-04-24T05:47:43.547190+0000","last_fullsized":"2024-04-24T05:47:43.547190+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:18:56.906868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.520613+0000","last_change":"2024-04-24T05:47:42.522227+0000","last_active":"2024-04-24T05:47:43.520613+0000","last_peered":"2024-04-24T05:47:43.520613+0000","last_clean":"2024-04-24T05:47:43.520613+0000","last_became_active":"2024-04-24T05:47:42.520224+0000","last_became_peered":"2024-04-24T05:47:42.520224+0000","last_unstale":"2024-04-24T05:47:43.520613+0000","last_undegraded":"2024-04-24T05:47:43.520613+0000","last_fullsized":"2024-04-24T05:47:43.520613+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:56.659470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.538983+0000","last_change":"2024-04-24T05:47:43.532331+0000","last_active":"2024-04-24T05:47:43.538983+0000","last_peered":"2024-04-24T05:47:43.538983+0000","last_clean":"2024-04-24T05:47:43.538983+0000","last_became_active":"2024-04-24T05:47:43.532005+0000","last_became_peered":"2024-04-24T05:47:43.532005+0000","last_unstale":"2024-04-24T05:47:43.538983+0000","last_undegraded":"2024-04-24T05:47:43.538983+0000","last_fullsized":"2024-04-24T05:47:43.538983+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:59.126614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.554627+0000","last_change":"2024-04-24T05:47:42.519108+0000","last_active":"2024-04-24T05:47:43.554627+0000","last_peered":"2024-04-24T05:47:43.554627+0000","last_clean":"2024-04-24T05:47:43.554627+0000","last_became_active":"2024-04-24T05:47:42.518833+0000","last_became_peered":"2024-04-24T05:47:42.518833+0000","last_unstale":"2024-04-24T05:47:43.554627+0000","last_undegraded":"2024-04-24T05:47:43.554627+0000","last_fullsized":"2024-04-24T05:47:43.554627+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:25.180697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.548991+0000","last_change":"2024-04-24T05:47:43.534485+0000","last_active":"2024-04-24T05:47:43.548991+0000","last_peered":"2024-04-24T05:47:43.548991+0000","last_clean":"2024-04-24T05:47:43.548991+0000","last_became_active":"2024-04-24T05:47:43.531899+0000","last_became_peered":"2024-04-24T05:47:43.531899+0000","last_unstale":"2024-04-24T05:47:43.548991+0000","last_undegraded":"2024-04-24T05:47:43.548991+0000","last_fullsized":"2024-04-24T05:47:43.548991+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:09.647755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.556473+0000","last_change":"2024-04-24T05:47:42.527384+0000","last_active":"2024-04-24T05:47:43.556473+0000","last_peered":"2024-04-24T05:47:43.556473+0000","last_clean":"2024-04-24T05:47:43.556473+0000","last_became_active":"2024-04-24T05:47:42.527066+0000","last_became_peered":"2024-04-24T05:47:42.527066+0000","last_unstale":"2024-04-24T05:47:43.556473+0000","last_undegraded":"2024-04-24T05:47:43.556473+0000","last_fullsized":"2024-04-24T05:47:43.556473+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:54:18.237837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.539272+0000","last_change":"2024-04-24T05:47:43.532839+0000","last_active":"2024-04-24T05:47:43.539272+0000","last_peered":"2024-04-24T05:47:43.539272+0000","last_clean":"2024-04-24T05:47:43.539272+0000","last_became_active":"2024-04-24T05:47:43.531694+0000","last_became_peered":"2024-04-24T05:47:43.531694+0000","last_unstale":"2024-04-24T05:47:43.539272+0000","last_undegraded":"2024-04-24T05:47:43.539272+0000","last_fullsized":"2024-04-24T05:47:43.539272+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:26.194824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.556042+0000","last_change":"2024-04-24T05:47:42.522788+0000","last_active":"2024-04-24T05:47:43.556042+0000","last_peered":"2024-04-24T05:47:43.556042+0000","last_clean":"2024-04-24T05:47:43.556042+0000","last_became_active":"2024-04-24T05:47:42.522202+0000","last_became_peered":"2024-04-24T05:47:42.522202+0000","last_unstale":"2024-04-24T05:47:43.556042+0000","last_undegraded":"2024-04-24T05:47:43.556042+0000","last_fullsized":"2024-04-24T05:47:43.556042+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:58.112486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.541792+0000","last_change":"2024-04-24T05:47:43.536247+0000","last_active":"2024-04-24T05:47:43.541792+0000","last_peered":"2024-04-24T05:47:43.541792+0000","last_clean":"2024-04-24T05:47:43.541792+0000","last_became_active":"2024-04-24T05:47:43.533839+0000","last_became_peered":"2024-04-24T05:47:43.533839+0000","last_unstale":"2024-04-24T05:47:43.541792+0000","last_undegraded":"2024-04-24T05:47:43.541792+0000","last_fullsized":"2024-04-24T05:47:43.541792+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:54.283092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.521418+0000","last_change":"2024-04-24T05:47:42.524094+0000","last_active":"2024-04-24T05:47:43.521418+0000","last_peered":"2024-04-24T05:47:43.521418+0000","last_clean":"2024-04-24T05:47:43.521418+0000","last_became_active":"2024-04-24T05:47:42.523855+0000","last_became_peered":"2024-04-24T05:47:42.523855+0000","last_unstale":"2024-04-24T05:47:43.521418+0000","last_undegraded":"2024-04-24T05:47:43.521418+0000","last_fullsized":"2024-04-24T05:47:43.521418+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:28.292171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.549773+0000","last_change":"2024-04-24T05:47:43.547017+0000","last_active":"2024-04-24T05:47:43.549773+0000","last_peered":"2024-04-24T05:47:43.549773+0000","last_clean":"2024-04-24T05:47:43.549773+0000","last_became_active":"2024-04-24T05:47:43.546639+0000","last_became_peered":"2024-04-24T05:47:43.546639+0000","last_unstale":"2024-04-24T05:47:43.549773+0000","last_undegraded":"2024-04-24T05:47:43.549773+0000","last_fullsized":"2024-04-24T05:47:43.549773+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:25:10.115913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.503480+0000","last_change":"2024-04-24T05:47:42.533940+0000","last_active":"2024-04-24T05:47:43.503480+0000","last_peered":"2024-04-24T05:47:43.503480+0000","last_clean":"2024-04-24T05:47:43.503480+0000","last_became_active":"2024-04-24T05:47:42.533657+0000","last_became_peered":"2024-04-24T05:47:42.533657+0000","last_unstale":"2024-04-24T05:47:43.503480+0000","last_undegraded":"2024-04-24T05:47:43.503480+0000","last_fullsized":"2024-04-24T05:47:43.503480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:43:36.839599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.540544+0000","last_change":"2024-04-24T05:47:43.534454+0000","last_active":"2024-04-24T05:47:43.540544+0000","last_peered":"2024-04-24T05:47:43.540544+0000","last_clean":"2024-04-24T05:47:43.540544+0000","last_became_active":"2024-04-24T05:47:43.532646+0000","last_became_peered":"2024-04-24T05:47:43.532646+0000","last_unstale":"2024-04-24T05:47:43.540544+0000","last_undegraded":"2024-04-24T05:47:43.540544+0000","last_fullsized":"2024-04-24T05:47:43.540544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:14:18.610223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.555396+0000","last_change":"2024-04-24T05:47:42.536450+0000","last_active":"2024-04-24T05:47:43.555396+0000","last_peered":"2024-04-24T05:47:43.555396+0000","last_clean":"2024-04-24T05:47:43.555396+0000","last_became_active":"2024-04-24T05:47:42.535464+0000","last_became_peered":"2024-04-24T05:47:42.535464+0000","last_unstale":"2024-04-24T05:47:43.555396+0000","last_undegraded":"2024-04-24T05:47:43.555396+0000","last_fullsized":"2024-04-24T05:47:43.555396+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:56.659470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.550459+0000","last_change":"2024-04-24T05:47:43.542694+0000","last_active":"2024-04-24T05:47:43.550459+0000","last_peered":"2024-04-24T05:47:43.550459+0000","last_clean":"2024-04-24T05:47:43.550459+0000","last_became_active":"2024-04-24T05:47:43.542427+0000","last_became_peered":"2024-04-24T05:47:43.542427+0000","last_unstale":"2024-04-24T05:47:43.550459+0000","last_undegraded":"2024-04-24T05:47:43.550459+0000","last_fullsized":"2024-04-24T05:47:43.550459+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:17:36.426617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.554592+0000","last_change":"2024-04-24T05:47:42.536077+0000","last_active":"2024-04-24T05:47:43.554592+0000","last_peered":"2024-04-24T05:47:43.554592+0000","last_clean":"2024-04-24T05:47:43.554592+0000","last_became_active":"2024-04-24T05:47:42.535513+0000","last_became_peered":"2024-04-24T05:47:42.535513+0000","last_unstale":"2024-04-24T05:47:43.554592+0000","last_undegraded":"2024-04-24T05:47:43.554592+0000","last_fullsized":"2024-04-24T05:47:43.554592+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:15.459009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.538321+0000","last_change":"2024-04-24T05:47:43.534212+0000","last_active":"2024-04-24T05:47:43.538321+0000","last_peered":"2024-04-24T05:47:43.538321+0000","last_clean":"2024-04-24T05:47:43.538321+0000","last_became_active":"2024-04-24T05:47:43.533887+0000","last_became_peered":"2024-04-24T05:47:43.533887+0000","last_unstale":"2024-04-24T05:47:43.538321+0000","last_undegraded":"2024-04-24T05:47:43.538321+0000","last_fullsized":"2024-04-24T05:47:43.538321+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:34:32.710562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.553388+0000","last_change":"2024-04-24T05:47:42.530588+0000","last_active":"2024-04-24T05:47:43.553388+0000","last_peered":"2024-04-24T05:47:43.553388+0000","last_clean":"2024-04-24T05:47:43.553388+0000","last_became_active":"2024-04-24T05:47:42.530057+0000","last_became_peered":"2024-04-24T05:47:42.530057+0000","last_unstale":"2024-04-24T05:47:43.553388+0000","last_undegraded":"2024-04-24T05:47:43.553388+0000","last_fullsized":"2024-04-24T05:47:43.553388+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:00.321752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.545019+0000","last_change":"2024-04-24T05:47:43.535568+0000","last_active":"2024-04-24T05:47:43.545019+0000","last_peered":"2024-04-24T05:47:43.545019+0000","last_clean":"2024-04-24T05:47:43.545019+0000","last_became_active":"2024-04-24T05:47:43.532446+0000","last_became_peered":"2024-04-24T05:47:43.532446+0000","last_unstale":"2024-04-24T05:47:43.545019+0000","last_undegraded":"2024-04-24T05:47:43.545019+0000","last_fullsized":"2024-04-24T05:47:43.545019+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:29.306299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.548629+0000","last_change":"2024-04-24T05:47:42.525010+0000","last_active":"2024-04-24T05:47:43.548629+0000","last_peered":"2024-04-24T05:47:43.548629+0000","last_clean":"2024-04-24T05:47:43.548629+0000","last_became_active":"2024-04-24T05:47:42.523442+0000","last_became_peered":"2024-04-24T05:47:42.523442+0000","last_unstale":"2024-04-24T05:47:43.548629+0000","last_undegraded":"2024-04-24T05:47:43.548629+0000","last_fullsized":"2024-04-24T05:47:43.548629+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:34:57.869333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.548498+0000","last_change":"2024-04-24T05:47:43.537238+0000","last_active":"2024-04-24T05:47:43.548498+0000","last_peered":"2024-04-24T05:47:43.548498+0000","last_clean":"2024-04-24T05:47:43.548498+0000","last_became_active":"2024-04-24T05:47:43.536876+0000","last_became_peered":"2024-04-24T05:47:43.536876+0000","last_unstale":"2024-04-24T05:47:43.548498+0000","last_undegraded":"2024-04-24T05:47:43.548498+0000","last_fullsized":"2024-04-24T05:47:43.548498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:30:56.456827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.522241+0000","last_change":"2024-04-24T05:47:42.529983+0000","last_active":"2024-04-24T05:47:43.522241+0000","last_peered":"2024-04-24T05:47:43.522241+0000","last_clean":"2024-04-24T05:47:43.522241+0000","last_became_active":"2024-04-24T05:47:42.529524+0000","last_became_peered":"2024-04-24T05:47:42.529524+0000","last_unstale":"2024-04-24T05:47:43.522241+0000","last_undegraded":"2024-04-24T05:47:43.522241+0000","last_fullsized":"2024-04-24T05:47:43.522241+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:56.659470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.537009+0000","last_change":"2024-04-24T05:47:43.530928+0000","last_active":"2024-04-24T05:47:43.537009+0000","last_peered":"2024-04-24T05:47:43.537009+0000","last_clean":"2024-04-24T05:47:43.537009+0000","last_became_active":"2024-04-24T05:47:43.530555+0000","last_became_peered":"2024-04-24T05:47:43.530555+0000","last_unstale":"2024-04-24T05:47:43.537009+0000","last_undegraded":"2024-04-24T05:47:43.537009+0000","last_fullsized":"2024-04-24T05:47:43.537009+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:26.194824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.548680+0000","last_change":"2024-04-24T05:47:42.524423+0000","last_active":"2024-04-24T05:47:43.548680+0000","last_peered":"2024-04-24T05:47:43.548680+0000","last_clean":"2024-04-24T05:47:43.548680+0000","last_became_active":"2024-04-24T05:47:42.524143+0000","last_became_peered":"2024-04-24T05:47:42.524143+0000","last_unstale":"2024-04-24T05:47:43.548680+0000","last_undegraded":"2024-04-24T05:47:43.548680+0000","last_fullsized":"2024-04-24T05:47:43.548680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:44:07.195469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.543590+0000","last_change":"2024-04-24T05:47:43.538671+0000","last_active":"2024-04-24T05:47:43.543590+0000","last_peered":"2024-04-24T05:47:43.543590+0000","last_clean":"2024-04-24T05:47:43.543590+0000","last_became_active":"2024-04-24T05:47:43.538051+0000","last_became_peered":"2024-04-24T05:47:43.538051+0000","last_unstale":"2024-04-24T05:47:43.543590+0000","last_undegraded":"2024-04-24T05:47:43.543590+0000","last_fullsized":"2024-04-24T05:47:43.543590+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:49:26.443913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.500297+0000","last_change":"2024-04-24T05:47:42.535958+0000","last_active":"2024-04-24T05:47:43.500297+0000","last_peered":"2024-04-24T05:47:43.500297+0000","last_clean":"2024-04-24T05:47:43.500297+0000","last_became_active":"2024-04-24T05:47:42.535518+0000","last_became_peered":"2024-04-24T05:47:42.535518+0000","last_unstale":"2024-04-24T05:47:43.500297+0000","last_undegraded":"2024-04-24T05:47:43.500297+0000","last_fullsized":"2024-04-24T05:47:43.500297+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:52:19.757263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.538399+0000","last_change":"2024-04-24T05:47:43.531515+0000","last_active":"2024-04-24T05:47:43.538399+0000","last_peered":"2024-04-24T05:47:43.538399+0000","last_clean":"2024-04-24T05:47:43.538399+0000","last_became_active":"2024-04-24T05:47:43.531054+0000","last_became_peered":"2024-04-24T05:47:43.531054+0000","last_unstale":"2024-04-24T05:47:43.538399+0000","last_undegraded":"2024-04-24T05:47:43.538399+0000","last_fullsized":"2024-04-24T05:47:43.538399+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:49:26.443913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.522641+0000","last_change":"2024-04-24T05:47:42.526989+0000","last_active":"2024-04-24T05:47:43.522641+0000","last_peered":"2024-04-24T05:47:43.522641+0000","last_clean":"2024-04-24T05:47:43.522641+0000","last_became_active":"2024-04-24T05:47:42.526694+0000","last_became_peered":"2024-04-24T05:47:42.526694+0000","last_unstale":"2024-04-24T05:47:43.522641+0000","last_undegraded":"2024-04-24T05:47:43.522641+0000","last_fullsized":"2024-04-24T05:47:43.522641+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:15.459009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.550944+0000","last_change":"2024-04-24T05:47:43.539677+0000","last_active":"2024-04-24T05:47:43.550944+0000","last_peered":"2024-04-24T05:47:43.550944+0000","last_clean":"2024-04-24T05:47:43.550944+0000","last_became_active":"2024-04-24T05:47:43.538868+0000","last_became_peered":"2024-04-24T05:47:43.538868+0000","last_unstale":"2024-04-24T05:47:43.550944+0000","last_undegraded":"2024-04-24T05:47:43.550944+0000","last_fullsized":"2024-04-24T05:47:43.550944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:18:56.906868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.501455+0000","last_change":"2024-04-24T05:47:42.517297+0000","last_active":"2024-04-24T05:47:43.501455+0000","last_peered":"2024-04-24T05:47:43.501455+0000","last_clean":"2024-04-24T05:47:43.501455+0000","last_became_active":"2024-04-24T05:47:42.516932+0000","last_became_peered":"2024-04-24T05:47:42.516932+0000","last_unstale":"2024-04-24T05:47:43.501455+0000","last_undegraded":"2024-04-24T05:47:43.501455+0000","last_fullsized":"2024-04-24T05:47:43.501455+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:34.506495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.543113+0000","last_change":"2024-04-24T05:47:43.537034+0000","last_active":"2024-04-24T05:47:43.543113+0000","last_peered":"2024-04-24T05:47:43.543113+0000","last_clean":"2024-04-24T05:47:43.543113+0000","last_became_active":"2024-04-24T05:47:43.536689+0000","last_became_peered":"2024-04-24T05:47:43.536689+0000","last_unstale":"2024-04-24T05:47:43.543113+0000","last_undegraded":"2024-04-24T05:47:43.543113+0000","last_fullsized":"2024-04-24T05:47:43.543113+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:26.194824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.554166+0000","last_change":"2024-04-24T05:47:42.519722+0000","last_active":"2024-04-24T05:47:43.554166+0000","last_peered":"2024-04-24T05:47:43.554166+0000","last_clean":"2024-04-24T05:47:43.554166+0000","last_became_active":"2024-04-24T05:47:42.517547+0000","last_became_peered":"2024-04-24T05:47:42.517547+0000","last_unstale":"2024-04-24T05:47:43.554166+0000","last_undegraded":"2024-04-24T05:47:43.554166+0000","last_fullsized":"2024-04-24T05:47:43.554166+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:54:18.237837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.547167+0000","last_change":"2024-04-24T05:47:43.542515+0000","last_active":"2024-04-24T05:47:43.547167+0000","last_peered":"2024-04-24T05:47:43.547167+0000","last_clean":"2024-04-24T05:47:43.547167+0000","last_became_active":"2024-04-24T05:47:43.540837+0000","last_became_peered":"2024-04-24T05:47:43.540837+0000","last_unstale":"2024-04-24T05:47:43.547167+0000","last_undegraded":"2024-04-24T05:47:43.547167+0000","last_fullsized":"2024-04-24T05:47:43.547167+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:34:32.710562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.501269+0000","last_change":"2024-04-24T05:47:42.520876+0000","last_active":"2024-04-24T05:47:43.501269+0000","last_peered":"2024-04-24T05:47:43.501269+0000","last_clean":"2024-04-24T05:47:43.501269+0000","last_became_active":"2024-04-24T05:47:42.520234+0000","last_became_peered":"2024-04-24T05:47:42.520234+0000","last_unstale":"2024-04-24T05:47:43.501269+0000","last_undegraded":"2024-04-24T05:47:43.501269+0000","last_fullsized":"2024-04-24T05:47:43.501269+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:18.866010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.537620+0000","last_change":"2024-04-24T05:47:43.534027+0000","last_active":"2024-04-24T05:47:43.537620+0000","last_peered":"2024-04-24T05:47:43.537620+0000","last_clean":"2024-04-24T05:47:43.537620+0000","last_became_active":"2024-04-24T05:47:43.532634+0000","last_became_peered":"2024-04-24T05:47:43.532634+0000","last_unstale":"2024-04-24T05:47:43.537620+0000","last_undegraded":"2024-04-24T05:47:43.537620+0000","last_fullsized":"2024-04-24T05:47:43.537620+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:49:26.443913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.514524+0000","last_change":"2024-04-24T05:47:42.519666+0000","last_active":"2024-04-24T05:47:43.514524+0000","last_peered":"2024-04-24T05:47:43.514524+0000","last_clean":"2024-04-24T05:47:43.514524+0000","last_became_active":"2024-04-24T05:47:42.519374+0000","last_became_peered":"2024-04-24T05:47:42.519374+0000","last_unstale":"2024-04-24T05:47:43.514524+0000","last_undegraded":"2024-04-24T05:47:43.514524+0000","last_fullsized":"2024-04-24T05:47:43.514524+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:28.292171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.537237+0000","last_change":"2024-04-24T05:47:43.529861+0000","last_active":"2024-04-24T05:47:43.537237+0000","last_peered":"2024-04-24T05:47:43.537237+0000","last_clean":"2024-04-24T05:47:43.537237+0000","last_became_active":"2024-04-24T05:47:43.528889+0000","last_became_peered":"2024-04-24T05:47:43.528889+0000","last_unstale":"2024-04-24T05:47:43.537237+0000","last_undegraded":"2024-04-24T05:47:43.537237+0000","last_fullsized":"2024-04-24T05:47:43.537237+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:54:19.251964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.512625+0000","last_change":"2024-04-24T05:47:42.516031+0000","last_active":"2024-04-24T05:47:43.512625+0000","last_peered":"2024-04-24T05:47:43.512625+0000","last_clean":"2024-04-24T05:47:43.512625+0000","last_became_active":"2024-04-24T05:47:42.515701+0000","last_became_peered":"2024-04-24T05:47:42.515701+0000","last_unstale":"2024-04-24T05:47:43.512625+0000","last_undegraded":"2024-04-24T05:47:43.512625+0000","last_fullsized":"2024-04-24T05:47:43.512625+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:22:51.396214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.541776+0000","last_change":"2024-04-24T05:47:43.532188+0000","last_active":"2024-04-24T05:47:43.541776+0000","last_peered":"2024-04-24T05:47:43.541776+0000","last_clean":"2024-04-24T05:47:43.541776+0000","last_became_active":"2024-04-24T05:47:43.530759+0000","last_became_peered":"2024-04-24T05:47:43.530759+0000","last_unstale":"2024-04-24T05:47:43.541776+0000","last_undegraded":"2024-04-24T05:47:43.541776+0000","last_fullsized":"2024-04-24T05:47:43.541776+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:54:19.251964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.504907+0000","last_change":"2024-04-24T05:47:42.524200+0000","last_active":"2024-04-24T05:47:43.504907+0000","last_peered":"2024-04-24T05:47:43.504907+0000","last_clean":"2024-04-24T05:47:43.504907+0000","last_became_active":"2024-04-24T05:47:42.523356+0000","last_became_peered":"2024-04-24T05:47:42.523356+0000","last_unstale":"2024-04-24T05:47:43.504907+0000","last_undegraded":"2024-04-24T05:47:43.504907+0000","last_fullsized":"2024-04-24T05:47:43.504907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:25.030481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544945+0000","last_change":"2024-04-24T05:47:43.532567+0000","last_active":"2024-04-24T05:47:43.544945+0000","last_peered":"2024-04-24T05:47:43.544945+0000","last_clean":"2024-04-24T05:47:43.544945+0000","last_became_active":"2024-04-24T05:47:43.529947+0000","last_became_peered":"2024-04-24T05:47:43.529947+0000","last_unstale":"2024-04-24T05:47:43.544945+0000","last_undegraded":"2024-04-24T05:47:43.544945+0000","last_fullsized":"2024-04-24T05:47:43.544945+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:51:08.032102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.517525+0000","last_change":"2024-04-24T05:47:42.526473+0000","last_active":"2024-04-24T05:47:43.517525+0000","last_peered":"2024-04-24T05:47:43.517525+0000","last_clean":"2024-04-24T05:47:43.517525+0000","last_became_active":"2024-04-24T05:47:42.526206+0000","last_became_peered":"2024-04-24T05:47:42.526206+0000","last_unstale":"2024-04-24T05:47:43.517525+0000","last_undegraded":"2024-04-24T05:47:43.517525+0000","last_fullsized":"2024-04-24T05:47:43.517525+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:25.030481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.540324+0000","last_change":"2024-04-24T05:47:43.537238+0000","last_active":"2024-04-24T05:47:43.540324+0000","last_peered":"2024-04-24T05:47:43.540324+0000","last_clean":"2024-04-24T05:47:43.540324+0000","last_became_active":"2024-04-24T05:47:43.536913+0000","last_became_peered":"2024-04-24T05:47:43.536913+0000","last_unstale":"2024-04-24T05:47:43.540324+0000","last_undegraded":"2024-04-24T05:47:43.540324+0000","last_fullsized":"2024-04-24T05:47:43.540324+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:45.002651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.523708+0000","last_change":"2024-04-24T05:47:42.522073+0000","last_active":"2024-04-24T05:47:43.523708+0000","last_peered":"2024-04-24T05:47:43.523708+0000","last_clean":"2024-04-24T05:47:43.523708+0000","last_became_active":"2024-04-24T05:47:42.521791+0000","last_became_peered":"2024-04-24T05:47:42.521791+0000","last_unstale":"2024-04-24T05:47:43.523708+0000","last_undegraded":"2024-04-24T05:47:43.523708+0000","last_fullsized":"2024-04-24T05:47:43.523708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:15.459009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.545892+0000","last_change":"2024-04-24T05:47:43.539201+0000","last_active":"2024-04-24T05:47:43.545892+0000","last_peered":"2024-04-24T05:47:43.545892+0000","last_clean":"2024-04-24T05:47:43.545892+0000","last_became_active":"2024-04-24T05:47:43.538881+0000","last_became_peered":"2024-04-24T05:47:43.538881+0000","last_unstale":"2024-04-24T05:47:43.545892+0000","last_undegraded":"2024-04-24T05:47:43.545892+0000","last_fullsized":"2024-04-24T05:47:43.545892+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:25:10.115913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.525919+0000","last_change":"2024-04-24T05:47:42.532510+0000","last_active":"2024-04-24T05:47:43.525919+0000","last_peered":"2024-04-24T05:47:43.525919+0000","last_clean":"2024-04-24T05:47:43.525919+0000","last_became_active":"2024-04-24T05:47:42.530821+0000","last_became_peered":"2024-04-24T05:47:42.530821+0000","last_unstale":"2024-04-24T05:47:43.525919+0000","last_undegraded":"2024-04-24T05:47:43.525919+0000","last_fullsized":"2024-04-24T05:47:43.525919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:58.112486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.517450+0000","last_change":"2024-04-24T05:45:51.019903+0000","last_active":"2024-04-24T05:47:43.517450+0000","last_peered":"2024-04-24T05:47:43.517450+0000","last_clean":"2024-04-24T05:47:43.517450+0000","last_became_active":"2024-04-24T05:45:51.017323+0000","last_became_peered":"2024-04-24T05:45:51.017323+0000","last_unstale":"2024-04-24T05:47:43.517450+0000","last_undegraded":"2024-04-24T05:47:43.517450+0000","last_fullsized":"2024-04-24T05:47:43.517450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:48.855871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:48.855871+0000","last_clean_scrub_stamp":"2024-04-24T05:45:48.855871+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:10:41.045203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":771626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":181258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":181258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":181258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":181258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":181258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":181258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":771626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T05:47:47.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T05:47:48.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.183+0000 7fe973200640 1 Processor -- start 2024-04-24T05:47:48.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.183+0000 7fe973200640 1 -- start start 2024-04-24T05:47:48.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.185+0000 7fe973200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe9740b4f70 0x7fe97406bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:48.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.185+0000 7fe973200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9740cc1e0 0x7fe97406c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:48.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.185+0000 7fe973200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9740cf1c0 con 0x7fe9740ad7d0 2024-04-24T05:47:48.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.185+0000 7fe973200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe9740cfd90 con 0x7fe9740b4f70 2024-04-24T05:47:48.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.185+0000 7fe973200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe9740cf460 con 0x7fe9740cc1e0 2024-04-24T05:47:48.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.185+0000 7fe971400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe9740b4f70 0x7fe97406bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:48.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.185+0000 7fe971400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe9740b4f70 0x7fe97406bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38944/0 (socket says 172.21.15.123:38944) 2024-04-24T05:47:48.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.185+0000 7fe971400640 1 -- 172.21.15.123:0/54736337 learned_addr learned my addr 172.21.15.123:0/54736337 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:48.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.185+0000 7fe972800640 1 --2- 172.21.15.123:0/54736337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9740cc1e0 0x7fe97406c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:48.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.185+0000 7fe971400640 1 -- 172.21.15.123:0/54736337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9740cc1e0 msgr2=0x7fe97406c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:48.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.185+0000 7fe971400640 1 --2- 172.21.15.123:0/54736337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9740cc1e0 0x7fe97406c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.185+0000 7fe971400640 1 -- 172.21.15.123:0/54736337 >> v1:172.21.15.155:6789/0 conn(0x7fe9740ad7d0 legacy=0x7fe9740adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:48.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.185+0000 7fe971400640 1 -- 172.21.15.123:0/54736337 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9740cf740 con 0x7fe9740b4f70 2024-04-24T05:47:48.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.186+0000 7fe971400640 1 --2- 172.21.15.123:0/54736337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9740b4f70 0x7fe97406bbb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe964009870 tx=0x7fe96402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7ce2342fddc1b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:48.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.187+0000 7fe970a00640 1 -- 172.21.15.123:0/54736337 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964037070 con 0x7fe9740b4f70 2024-04-24T05:47:48.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.187+0000 7fe970a00640 1 -- 172.21.15.123:0/54736337 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe964032440 con 0x7fe9740b4f70 2024-04-24T05:47:48.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.187+0000 7fe970a00640 1 -- 172.21.15.123:0/54736337 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964032810 con 0x7fe9740b4f70 2024-04-24T05:47:48.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.187+0000 7fe973200640 1 -- 172.21.15.123:0/54736337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9740b4f70 msgr2=0x7fe97406bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:48.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.187+0000 7fe973200640 1 --2- 172.21.15.123:0/54736337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9740b4f70 0x7fe97406bbb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe964009870 tx=0x7fe96402a210 comp rx=0 tx=0).stop 2024-04-24T05:47:48.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.188+0000 7fe973200640 1 -- 172.21.15.123:0/54736337 shutdown_connections 2024-04-24T05:47:48.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.188+0000 7fe973200640 1 --2- 172.21.15.123:0/54736337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9740cc1e0 0x7fe97406c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.188+0000 7fe973200640 1 --2- 172.21.15.123:0/54736337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9740b4f70 0x7fe97406bbb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.188+0000 7fe973200640 1 -- 172.21.15.123:0/54736337 >> 172.21.15.123:0/54736337 conn(0x7fe9740d0070 msgr2=0x7fe9740d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:48.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.188+0000 7fe973200640 1 -- 172.21.15.123:0/54736337 shutdown_connections 2024-04-24T05:47:48.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.188+0000 7fe973200640 1 -- 172.21.15.123:0/54736337 wait complete. 2024-04-24T05:47:48.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.189+0000 7fe973200640 1 Processor -- start 2024-04-24T05:47:48.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.189+0000 7fe973200640 1 -- start start 2024-04-24T05:47:48.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.190+0000 7fe973200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9740ad7d0 0x7fe974168cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:48.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.190+0000 7fe973200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9740b4f70 0x7fe97415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:48.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.190+0000 7fe973200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9740cc1e0 0x7fe974158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:48.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.190+0000 7fe973200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9740d0e00 con 0x7fe9740b4f70 2024-04-24T05:47:48.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.190+0000 7fe973200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9740b2f60 con 0x7fe9740ad7d0 2024-04-24T05:47:48.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.190+0000 7fe973200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe9740d0f80 con 0x7fe9740cc1e0 2024-04-24T05:47:48.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.191+0000 7fe971400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9740b4f70 0x7fe97415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:48.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.191+0000 7fe971400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9740b4f70 0x7fe97415adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38960/0 (socket says 172.21.15.123:38960) 2024-04-24T05:47:48.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.191+0000 7fe971400640 1 -- 172.21.15.123:0/3416238856 learned_addr learned my addr 172.21.15.123:0/3416238856 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:48.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.191+0000 7fe971400640 1 -- 172.21.15.123:0/3416238856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9740cc1e0 msgr2=0x7fe974158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:48.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.191+0000 7fe971400640 1 --2- 172.21.15.123:0/3416238856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9740cc1e0 0x7fe974158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.191+0000 7fe971400640 1 -- 172.21.15.123:0/3416238856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9740ad7d0 msgr2=0x7fe974168cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:48.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.191+0000 7fe971400640 1 --2- 172.21.15.123:0/3416238856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9740ad7d0 0x7fe974168cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.191+0000 7fe971400640 1 -- 172.21.15.123:0/3416238856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe974167c40 con 0x7fe9740b4f70 2024-04-24T05:47:48.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.191+0000 7fe971400640 1 --2- 172.21.15.123:0/3416238856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9740b4f70 0x7fe97415adc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe964009870 tx=0x7fe9640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:48.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.192+0000 7fe96a000640 1 -- 172.21.15.123:0/3416238856 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964041020 con 0x7fe9740b4f70 2024-04-24T05:47:48.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.192+0000 7fe96a000640 1 -- 172.21.15.123:0/3416238856 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe964030970 con 0x7fe9740b4f70 2024-04-24T05:47:48.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.192+0000 7fe96a000640 1 -- 172.21.15.123:0/3416238856 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964037040 con 0x7fe9740b4f70 2024-04-24T05:47:48.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.192+0000 7fe973200640 1 -- 172.21.15.123:0/3416238856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe974167f00 con 0x7fe9740b4f70 2024-04-24T05:47:48.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.192+0000 7fe973200640 1 -- 172.21.15.123:0/3416238856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9741684c0 con 0x7fe9740b4f70 2024-04-24T05:47:48.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.193+0000 7fe973200640 1 -- 172.21.15.123:0/3416238856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe974075b70 con 0x7fe9740b4f70 2024-04-24T05:47:48.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.194+0000 7fe96a000640 1 -- 172.21.15.123:0/3416238856 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe964044020 con 0x7fe9740b4f70 2024-04-24T05:47:48.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.195+0000 7fe96a000640 1 --2- 172.21.15.123:0/3416238856 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe95006c8f0 0x7fe95006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:48.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.195+0000 7fe96a000640 1 -- 172.21.15.123:0/3416238856 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe9640ac730 con 0x7fe9740b4f70 2024-04-24T05:47:48.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.195+0000 7fe971e00640 1 --2- 172.21.15.123:0/3416238856 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe95006c8f0 0x7fe95006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:48.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.196+0000 7fe971e00640 1 --2- 172.21.15.123:0/3416238856 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe95006c8f0 0x7fe95006edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe95c003cc0 tx=0x7fe95c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:48.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.199+0000 7fe96a000640 1 -- 172.21.15.123:0/3416238856 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe96404b030 con 0x7fe9740b4f70 2024-04-24T05:47:48.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.396+0000 7fe973200640 1 -- 172.21.15.123:0/3416238856 --> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe97406d2d0 con 0x7fe95006c8f0 2024-04-24T05:47:48.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.403+0000 7fe96a000640 1 -- 172.21.15.123:0/3416238856 <== mgr.4101 v2:172.21.15.123:6832/1503718323 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251886 (secure 0 0 0) 0x7fe97406d2d0 con 0x7fe95006c8f0 2024-04-24T05:47:48.409 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:48.410 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-04-24T05:47:48.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.407+0000 7fe973200640 1 -- 172.21.15.123:0/3416238856 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe95006c8f0 msgr2=0x7fe95006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:48.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.407+0000 7fe973200640 1 --2- 172.21.15.123:0/3416238856 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe95006c8f0 0x7fe95006edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe95c003cc0 tx=0x7fe95c0079b0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.407+0000 7fe973200640 1 -- 172.21.15.123:0/3416238856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9740b4f70 msgr2=0x7fe97415adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:48.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.407+0000 7fe973200640 1 --2- 172.21.15.123:0/3416238856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9740b4f70 0x7fe97415adc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe964009870 tx=0x7fe9640046c0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.407+0000 7fe973200640 1 -- 172.21.15.123:0/3416238856 shutdown_connections 2024-04-24T05:47:48.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.407+0000 7fe973200640 1 --2- 172.21.15.123:0/3416238856 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fe95006c8f0 0x7fe95006edb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.407+0000 7fe973200640 1 --2- 172.21.15.123:0/3416238856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9740cc1e0 0x7fe974158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.407+0000 7fe973200640 1 --2- 172.21.15.123:0/3416238856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9740b4f70 0x7fe97415adc0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.407+0000 7fe973200640 1 --2- 172.21.15.123:0/3416238856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9740ad7d0 0x7fe974168cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.407+0000 7fe973200640 1 -- 172.21.15.123:0/3416238856 >> 172.21.15.123:0/3416238856 conn(0x7fe9740d0070 msgr2=0x7fe97406bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:48.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.407+0000 7fe973200640 1 -- 172.21.15.123:0/3416238856 shutdown_connections 2024-04-24T05:47:48.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.407+0000 7fe973200640 1 -- 172.21.15.123:0/3416238856 wait complete. 2024-04-24T05:47:48.427 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-04-24T05:47:46.546829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220672,"kb_used_data":4488,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754048,"statfs":{"total":773094113280,"available":772868145152,"internally_reserved":0,"allocated":4595712,"data_stored":2630800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.071176"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.535759+0000","last_change":"2024-04-24T05:47:43.532026+0000","last_active":"2024-04-24T05:47:43.535759+0000","last_peered":"2024-04-24T05:47:43.535759+0000","last_clean":"2024-04-24T05:47:43.535759+0000","last_became_active":"2024-04-24T05:47:43.531433+0000","last_became_peered":"2024-04-24T05:47:43.531433+0000","last_unstale":"2024-04-24T05:47:43.535759+0000","last_undegraded":"2024-04-24T05:47:43.535759+0000","last_fullsized":"2024-04-24T05:47:43.535759+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:45.002651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.546733+0000","last_change":"2024-04-24T05:47:43.541013+0000","last_active":"2024-04-24T05:47:43.546733+0000","last_peered":"2024-04-24T05:47:43.546733+0000","last_clean":"2024-04-24T05:47:43.546733+0000","last_became_active":"2024-04-24T05:47:43.540782+0000","last_became_peered":"2024-04-24T05:47:43.540782+0000","last_unstale":"2024-04-24T05:47:43.546733+0000","last_undegraded":"2024-04-24T05:47:43.546733+0000","last_fullsized":"2024-04-24T05:47:43.546733+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:49.952705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.555742+0000","last_change":"2024-04-24T05:47:43.550470+0000","last_active":"2024-04-24T05:47:43.555742+0000","last_peered":"2024-04-24T05:47:43.555742+0000","last_clean":"2024-04-24T05:47:43.555742+0000","last_became_active":"2024-04-24T05:47:43.550183+0000","last_became_peered":"2024-04-24T05:47:43.550183+0000","last_unstale":"2024-04-24T05:47:43.555742+0000","last_undegraded":"2024-04-24T05:47:43.555742+0000","last_fullsized":"2024-04-24T05:47:43.555742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:14:18.610223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544580+0000","last_change":"2024-04-24T05:47:43.534882+0000","last_active":"2024-04-24T05:47:43.544580+0000","last_peered":"2024-04-24T05:47:43.544580+0000","last_clean":"2024-04-24T05:47:43.544580+0000","last_became_active":"2024-04-24T05:47:43.534577+0000","last_became_peered":"2024-04-24T05:47:43.534577+0000","last_unstale":"2024-04-24T05:47:43.544580+0000","last_undegraded":"2024-04-24T05:47:43.544580+0000","last_fullsized":"2024-04-24T05:47:43.544580+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:01:42.716524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.543970+0000","last_change":"2024-04-24T05:47:43.530606+0000","last_active":"2024-04-24T05:47:43.543970+0000","last_peered":"2024-04-24T05:47:43.543970+0000","last_clean":"2024-04-24T05:47:43.543970+0000","last_became_active":"2024-04-24T05:47:43.529738+0000","last_became_peered":"2024-04-24T05:47:43.529738+0000","last_unstale":"2024-04-24T05:47:43.543970+0000","last_undegraded":"2024-04-24T05:47:43.543970+0000","last_fullsized":"2024-04-24T05:47:43.543970+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:41:45.562637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.543066+0000","last_change":"2024-04-24T05:47:43.537110+0000","last_active":"2024-04-24T05:47:43.543066+0000","last_peered":"2024-04-24T05:47:43.543066+0000","last_clean":"2024-04-24T05:47:43.543066+0000","last_became_active":"2024-04-24T05:47:43.536827+0000","last_became_peered":"2024-04-24T05:47:43.536827+0000","last_unstale":"2024-04-24T05:47:43.543066+0000","last_undegraded":"2024-04-24T05:47:43.543066+0000","last_fullsized":"2024-04-24T05:47:43.543066+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:45.002651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544096+0000","last_change":"2024-04-24T05:47:43.539466+0000","last_active":"2024-04-24T05:47:43.544096+0000","last_peered":"2024-04-24T05:47:43.544096+0000","last_clean":"2024-04-24T05:47:43.544096+0000","last_became_active":"2024-04-24T05:47:43.539108+0000","last_became_peered":"2024-04-24T05:47:43.539108+0000","last_unstale":"2024-04-24T05:47:43.544096+0000","last_undegraded":"2024-04-24T05:47:43.544096+0000","last_fullsized":"2024-04-24T05:47:43.544096+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:49.952705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.546435+0000","last_change":"2024-04-24T05:47:43.538288+0000","last_active":"2024-04-24T05:47:43.546435+0000","last_peered":"2024-04-24T05:47:43.546435+0000","last_clean":"2024-04-24T05:47:43.546435+0000","last_became_active":"2024-04-24T05:47:43.537994+0000","last_became_peered":"2024-04-24T05:47:43.537994+0000","last_unstale":"2024-04-24T05:47:43.546435+0000","last_undegraded":"2024-04-24T05:47:43.546435+0000","last_fullsized":"2024-04-24T05:47:43.546435+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:34:32.710562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.543528+0000","last_change":"2024-04-24T05:47:43.538047+0000","last_active":"2024-04-24T05:47:43.543528+0000","last_peered":"2024-04-24T05:47:43.543528+0000","last_clean":"2024-04-24T05:47:43.543528+0000","last_became_active":"2024-04-24T05:47:43.535743+0000","last_became_peered":"2024-04-24T05:47:43.535743+0000","last_unstale":"2024-04-24T05:47:43.543528+0000","last_undegraded":"2024-04-24T05:47:43.543528+0000","last_fullsized":"2024-04-24T05:47:43.543528+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:45.002651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.547329+0000","last_change":"2024-04-24T05:47:43.541239+0000","last_active":"2024-04-24T05:47:43.547329+0000","last_peered":"2024-04-24T05:47:43.547329+0000","last_clean":"2024-04-24T05:47:43.547329+0000","last_became_active":"2024-04-24T05:47:43.540936+0000","last_became_peered":"2024-04-24T05:47:43.540936+0000","last_unstale":"2024-04-24T05:47:43.547329+0000","last_undegraded":"2024-04-24T05:47:43.547329+0000","last_fullsized":"2024-04-24T05:47:43.547329+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:15.413070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.538344+0000","last_change":"2024-04-24T05:47:43.534409+0000","last_active":"2024-04-24T05:47:43.538344+0000","last_peered":"2024-04-24T05:47:43.538344+0000","last_clean":"2024-04-24T05:47:43.538344+0000","last_became_active":"2024-04-24T05:47:43.534123+0000","last_became_peered":"2024-04-24T05:47:43.534123+0000","last_unstale":"2024-04-24T05:47:43.538344+0000","last_undegraded":"2024-04-24T05:47:43.538344+0000","last_fullsized":"2024-04-24T05:47:43.538344+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:18:56.906868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.524466+0000","last_change":"2024-04-24T05:47:43.519430+0000","last_active":"2024-04-24T05:47:43.524466+0000","last_peered":"2024-04-24T05:47:43.524466+0000","last_clean":"2024-04-24T05:47:43.524466+0000","last_became_active":"2024-04-24T05:47:43.519133+0000","last_became_peered":"2024-04-24T05:47:43.519133+0000","last_unstale":"2024-04-24T05:47:43.524466+0000","last_undegraded":"2024-04-24T05:47:43.524466+0000","last_fullsized":"2024-04-24T05:47:43.524466+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:57.673598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.546336+0000","last_change":"2024-04-24T05:47:43.540156+0000","last_active":"2024-04-24T05:47:43.546336+0000","last_peered":"2024-04-24T05:47:43.546336+0000","last_clean":"2024-04-24T05:47:43.546336+0000","last_became_active":"2024-04-24T05:47:43.539866+0000","last_became_peered":"2024-04-24T05:47:43.539866+0000","last_unstale":"2024-04-24T05:47:43.546336+0000","last_undegraded":"2024-04-24T05:47:43.546336+0000","last_fullsized":"2024-04-24T05:47:43.546336+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:17:36.426617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.545197+0000","last_change":"2024-04-24T05:47:43.536776+0000","last_active":"2024-04-24T05:47:43.545197+0000","last_peered":"2024-04-24T05:47:43.545197+0000","last_clean":"2024-04-24T05:47:43.545197+0000","last_became_active":"2024-04-24T05:47:43.536496+0000","last_became_peered":"2024-04-24T05:47:43.536496+0000","last_unstale":"2024-04-24T05:47:43.545197+0000","last_undegraded":"2024-04-24T05:47:43.545197+0000","last_fullsized":"2024-04-24T05:47:43.545197+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:16.473137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.540646+0000","last_change":"2024-04-24T05:47:43.533184+0000","last_active":"2024-04-24T05:47:43.540646+0000","last_peered":"2024-04-24T05:47:43.540646+0000","last_clean":"2024-04-24T05:47:43.540646+0000","last_became_active":"2024-04-24T05:47:43.532856+0000","last_became_peered":"2024-04-24T05:47:43.532856+0000","last_unstale":"2024-04-24T05:47:43.540646+0000","last_undegraded":"2024-04-24T05:47:43.540646+0000","last_fullsized":"2024-04-24T05:47:43.540646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:54.283092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.547416+0000","last_change":"2024-04-24T05:47:43.539182+0000","last_active":"2024-04-24T05:47:43.547416+0000","last_peered":"2024-04-24T05:47:43.547416+0000","last_clean":"2024-04-24T05:47:43.547416+0000","last_became_active":"2024-04-24T05:47:43.538891+0000","last_became_peered":"2024-04-24T05:47:43.538891+0000","last_unstale":"2024-04-24T05:47:43.547416+0000","last_undegraded":"2024-04-24T05:47:43.547416+0000","last_fullsized":"2024-04-24T05:47:43.547416+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:49:26.443913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.547008+0000","last_change":"2024-04-24T05:47:43.541268+0000","last_active":"2024-04-24T05:47:43.547008+0000","last_peered":"2024-04-24T05:47:43.547008+0000","last_clean":"2024-04-24T05:47:43.547008+0000","last_became_active":"2024-04-24T05:47:43.539493+0000","last_became_peered":"2024-04-24T05:47:43.539493+0000","last_unstale":"2024-04-24T05:47:43.547008+0000","last_undegraded":"2024-04-24T05:47:43.547008+0000","last_fullsized":"2024-04-24T05:47:43.547008+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:54.283092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.549038+0000","last_change":"2024-04-24T05:47:43.544582+0000","last_active":"2024-04-24T05:47:43.549038+0000","last_peered":"2024-04-24T05:47:43.549038+0000","last_clean":"2024-04-24T05:47:43.549038+0000","last_became_active":"2024-04-24T05:47:43.540995+0000","last_became_peered":"2024-04-24T05:47:43.540995+0000","last_unstale":"2024-04-24T05:47:43.549038+0000","last_undegraded":"2024-04-24T05:47:43.549038+0000","last_fullsized":"2024-04-24T05:47:43.549038+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:57.673598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544418+0000","last_change":"2024-04-24T05:47:43.534229+0000","last_active":"2024-04-24T05:47:43.544418+0000","last_peered":"2024-04-24T05:47:43.544418+0000","last_clean":"2024-04-24T05:47:43.544418+0000","last_became_active":"2024-04-24T05:47:43.530596+0000","last_became_peered":"2024-04-24T05:47:43.530596+0000","last_unstale":"2024-04-24T05:47:43.544418+0000","last_undegraded":"2024-04-24T05:47:43.544418+0000","last_fullsized":"2024-04-24T05:47:43.544418+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:10:29.082247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.543634+0000","last_change":"2024-04-24T05:47:43.537580+0000","last_active":"2024-04-24T05:47:43.543634+0000","last_peered":"2024-04-24T05:47:43.543634+0000","last_clean":"2024-04-24T05:47:43.543634+0000","last_became_active":"2024-04-24T05:47:43.537253+0000","last_became_peered":"2024-04-24T05:47:43.537253+0000","last_unstale":"2024-04-24T05:47:43.543634+0000","last_undegraded":"2024-04-24T05:47:43.543634+0000","last_fullsized":"2024-04-24T05:47:43.543634+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:29.306299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.538263+0000","last_change":"2024-04-24T05:47:43.530351+0000","last_active":"2024-04-24T05:47:43.538263+0000","last_peered":"2024-04-24T05:47:43.538263+0000","last_clean":"2024-04-24T05:47:43.538263+0000","last_became_active":"2024-04-24T05:47:43.529857+0000","last_became_peered":"2024-04-24T05:47:43.529857+0000","last_unstale":"2024-04-24T05:47:43.538263+0000","last_undegraded":"2024-04-24T05:47:43.538263+0000","last_fullsized":"2024-04-24T05:47:43.538263+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:45.002651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544973+0000","last_change":"2024-04-24T05:47:43.533746+0000","last_active":"2024-04-24T05:47:43.544973+0000","last_peered":"2024-04-24T05:47:43.544973+0000","last_clean":"2024-04-24T05:47:43.544973+0000","last_became_active":"2024-04-24T05:47:43.532859+0000","last_became_peered":"2024-04-24T05:47:43.532859+0000","last_unstale":"2024-04-24T05:47:43.544973+0000","last_undegraded":"2024-04-24T05:47:43.544973+0000","last_fullsized":"2024-04-24T05:47:43.544973+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:22:52.410342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.546638+0000","last_change":"2024-04-24T05:47:43.541070+0000","last_active":"2024-04-24T05:47:43.546638+0000","last_peered":"2024-04-24T05:47:43.546638+0000","last_clean":"2024-04-24T05:47:43.546638+0000","last_became_active":"2024-04-24T05:47:43.540780+0000","last_became_peered":"2024-04-24T05:47:43.540780+0000","last_unstale":"2024-04-24T05:47:43.546638+0000","last_undegraded":"2024-04-24T05:47:43.546638+0000","last_fullsized":"2024-04-24T05:47:43.546638+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:14:18.610223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544065+0000","last_change":"2024-04-24T05:47:43.528015+0000","last_active":"2024-04-24T05:47:43.544065+0000","last_peered":"2024-04-24T05:47:43.544065+0000","last_clean":"2024-04-24T05:47:43.544065+0000","last_became_active":"2024-04-24T05:47:43.527684+0000","last_became_peered":"2024-04-24T05:47:43.527684+0000","last_unstale":"2024-04-24T05:47:43.544065+0000","last_undegraded":"2024-04-24T05:47:43.544065+0000","last_fullsized":"2024-04-24T05:47:43.544065+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:54:47.771923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.530526+0000","last_change":"2024-04-24T05:47:43.523663+0000","last_active":"2024-04-24T05:47:43.530526+0000","last_peered":"2024-04-24T05:47:43.530526+0000","last_clean":"2024-04-24T05:47:43.530526+0000","last_became_active":"2024-04-24T05:47:43.523266+0000","last_became_peered":"2024-04-24T05:47:43.523266+0000","last_unstale":"2024-04-24T05:47:43.530526+0000","last_undegraded":"2024-04-24T05:47:43.530526+0000","last_fullsized":"2024-04-24T05:47:43.530526+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:51:08.032102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.543004+0000","last_change":"2024-04-24T05:47:43.536242+0000","last_active":"2024-04-24T05:47:43.543004+0000","last_peered":"2024-04-24T05:47:43.543004+0000","last_clean":"2024-04-24T05:47:43.543004+0000","last_became_active":"2024-04-24T05:47:43.535954+0000","last_became_peered":"2024-04-24T05:47:43.535954+0000","last_unstale":"2024-04-24T05:47:43.543004+0000","last_undegraded":"2024-04-24T05:47:43.543004+0000","last_fullsized":"2024-04-24T05:47:43.543004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:59.126614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.530309+0000","last_change":"2024-04-24T05:47:43.520366+0000","last_active":"2024-04-24T05:47:43.530309+0000","last_peered":"2024-04-24T05:47:43.530309+0000","last_clean":"2024-04-24T05:47:43.530309+0000","last_became_active":"2024-04-24T05:47:43.519932+0000","last_became_peered":"2024-04-24T05:47:43.519932+0000","last_unstale":"2024-04-24T05:47:43.530309+0000","last_undegraded":"2024-04-24T05:47:43.530309+0000","last_fullsized":"2024-04-24T05:47:43.530309+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:01.335880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.548536+0000","last_change":"2024-04-24T05:47:43.538735+0000","last_active":"2024-04-24T05:47:43.548536+0000","last_peered":"2024-04-24T05:47:43.548536+0000","last_clean":"2024-04-24T05:47:43.548536+0000","last_became_active":"2024-04-24T05:47:43.538428+0000","last_became_peered":"2024-04-24T05:47:43.538428+0000","last_unstale":"2024-04-24T05:47:43.548536+0000","last_undegraded":"2024-04-24T05:47:43.548536+0000","last_fullsized":"2024-04-24T05:47:43.548536+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:21:10.691766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.538132+0000","last_change":"2024-04-24T05:47:43.531542+0000","last_active":"2024-04-24T05:47:43.538132+0000","last_peered":"2024-04-24T05:47:43.538132+0000","last_clean":"2024-04-24T05:47:43.538132+0000","last_became_active":"2024-04-24T05:47:43.531200+0000","last_became_peered":"2024-04-24T05:47:43.531200+0000","last_unstale":"2024-04-24T05:47:43.538132+0000","last_undegraded":"2024-04-24T05:47:43.538132+0000","last_fullsized":"2024-04-24T05:47:43.538132+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:22:52.410342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.538482+0000","last_change":"2024-04-24T05:47:43.531985+0000","last_active":"2024-04-24T05:47:43.538482+0000","last_peered":"2024-04-24T05:47:43.538482+0000","last_clean":"2024-04-24T05:47:43.538482+0000","last_became_active":"2024-04-24T05:47:43.530041+0000","last_became_peered":"2024-04-24T05:47:43.530041+0000","last_unstale":"2024-04-24T05:47:43.538482+0000","last_undegraded":"2024-04-24T05:47:43.538482+0000","last_fullsized":"2024-04-24T05:47:43.538482+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:09.647755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.548317+0000","last_change":"2024-04-24T05:47:43.546219+0000","last_active":"2024-04-24T05:47:43.548317+0000","last_peered":"2024-04-24T05:47:43.548317+0000","last_clean":"2024-04-24T05:47:43.548317+0000","last_became_active":"2024-04-24T05:47:43.545895+0000","last_became_peered":"2024-04-24T05:47:43.545895+0000","last_unstale":"2024-04-24T05:47:43.548317+0000","last_undegraded":"2024-04-24T05:47:43.548317+0000","last_fullsized":"2024-04-24T05:47:43.548317+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:21:10.691766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.548318+0000","last_change":"2024-04-24T05:47:43.545361+0000","last_active":"2024-04-24T05:47:43.548318+0000","last_peered":"2024-04-24T05:47:43.548318+0000","last_clean":"2024-04-24T05:47:43.548318+0000","last_became_active":"2024-04-24T05:47:43.545062+0000","last_became_peered":"2024-04-24T05:47:43.545062+0000","last_unstale":"2024-04-24T05:47:43.548318+0000","last_undegraded":"2024-04-24T05:47:43.548318+0000","last_fullsized":"2024-04-24T05:47:43.548318+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:49:26.443913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.554415+0000","last_change":"2024-04-24T05:47:43.550870+0000","last_active":"2024-04-24T05:47:43.554415+0000","last_peered":"2024-04-24T05:47:43.554415+0000","last_clean":"2024-04-24T05:47:43.554415+0000","last_became_active":"2024-04-24T05:47:43.550499+0000","last_became_peered":"2024-04-24T05:47:43.550499+0000","last_unstale":"2024-04-24T05:47:43.554415+0000","last_undegraded":"2024-04-24T05:47:43.554415+0000","last_fullsized":"2024-04-24T05:47:43.554415+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:49.952705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.553172+0000","last_change":"2024-04-24T05:47:42.530065+0000","last_active":"2024-04-24T05:47:43.553172+0000","last_peered":"2024-04-24T05:47:43.553172+0000","last_clean":"2024-04-24T05:47:43.553172+0000","last_became_active":"2024-04-24T05:47:42.529676+0000","last_became_peered":"2024-04-24T05:47:42.529676+0000","last_unstale":"2024-04-24T05:47:43.553172+0000","last_undegraded":"2024-04-24T05:47:43.553172+0000","last_fullsized":"2024-04-24T05:47:43.553172+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:36:14.055524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.553365+0000","last_change":"2024-04-24T05:47:43.549633+0000","last_active":"2024-04-24T05:47:43.553365+0000","last_peered":"2024-04-24T05:47:43.553365+0000","last_clean":"2024-04-24T05:47:43.553365+0000","last_became_active":"2024-04-24T05:47:43.549249+0000","last_became_peered":"2024-04-24T05:47:43.549249+0000","last_unstale":"2024-04-24T05:47:43.553365+0000","last_undegraded":"2024-04-24T05:47:43.553365+0000","last_fullsized":"2024-04-24T05:47:43.553365+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:14:18.610223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544392+0000","last_change":"2024-04-24T05:47:42.526660+0000","last_active":"2024-04-24T05:47:43.544392+0000","last_peered":"2024-04-24T05:47:43.544392+0000","last_clean":"2024-04-24T05:47:43.544392+0000","last_became_active":"2024-04-24T05:47:42.526345+0000","last_became_peered":"2024-04-24T05:47:42.526345+0000","last_unstale":"2024-04-24T05:47:43.544392+0000","last_undegraded":"2024-04-24T05:47:43.544392+0000","last_fullsized":"2024-04-24T05:47:43.544392+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:08.633627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.545045+0000","last_change":"2024-04-24T05:47:43.535651+0000","last_active":"2024-04-24T05:47:43.545045+0000","last_peered":"2024-04-24T05:47:43.545045+0000","last_clean":"2024-04-24T05:47:43.545045+0000","last_became_active":"2024-04-24T05:47:43.535349+0000","last_became_peered":"2024-04-24T05:47:43.535349+0000","last_unstale":"2024-04-24T05:47:43.545045+0000","last_undegraded":"2024-04-24T05:47:43.545045+0000","last_fullsized":"2024-04-24T05:47:43.545045+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:57.673598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.547922+0000","last_change":"2024-04-24T05:47:42.535682+0000","last_active":"2024-04-24T05:47:43.547922+0000","last_peered":"2024-04-24T05:47:43.547922+0000","last_clean":"2024-04-24T05:47:43.547922+0000","last_became_active":"2024-04-24T05:47:42.535382+0000","last_became_peered":"2024-04-24T05:47:42.535382+0000","last_unstale":"2024-04-24T05:47:43.547922+0000","last_undegraded":"2024-04-24T05:47:43.547922+0000","last_fullsized":"2024-04-24T05:47:43.547922+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:15.459009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.546470+0000","last_change":"2024-04-24T05:47:43.540242+0000","last_active":"2024-04-24T05:47:43.546470+0000","last_peered":"2024-04-24T05:47:43.546470+0000","last_clean":"2024-04-24T05:47:43.546470+0000","last_became_active":"2024-04-24T05:47:43.537526+0000","last_became_peered":"2024-04-24T05:47:43.537526+0000","last_unstale":"2024-04-24T05:47:43.546470+0000","last_undegraded":"2024-04-24T05:47:43.546470+0000","last_fullsized":"2024-04-24T05:47:43.546470+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:25:10.115913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.554419+0000","last_change":"2024-04-24T05:47:42.532915+0000","last_active":"2024-04-24T05:47:43.554419+0000","last_peered":"2024-04-24T05:47:43.554419+0000","last_clean":"2024-04-24T05:47:43.554419+0000","last_became_active":"2024-04-24T05:47:42.532614+0000","last_became_peered":"2024-04-24T05:47:42.532614+0000","last_unstale":"2024-04-24T05:47:43.554419+0000","last_undegraded":"2024-04-24T05:47:43.554419+0000","last_fullsized":"2024-04-24T05:47:43.554419+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:36:14.055524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.537214+0000","last_change":"2024-04-24T05:47:43.529365+0000","last_active":"2024-04-24T05:47:43.537214+0000","last_peered":"2024-04-24T05:47:43.537214+0000","last_clean":"2024-04-24T05:47:43.537214+0000","last_became_active":"2024-04-24T05:47:43.528243+0000","last_became_peered":"2024-04-24T05:47:43.528243+0000","last_unstale":"2024-04-24T05:47:43.537214+0000","last_undegraded":"2024-04-24T05:47:43.537214+0000","last_fullsized":"2024-04-24T05:47:43.537214+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:16.473137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.553435+0000","last_change":"2024-04-24T05:47:42.525848+0000","last_active":"2024-04-24T05:47:43.553435+0000","last_peered":"2024-04-24T05:47:43.553435+0000","last_clean":"2024-04-24T05:47:43.553435+0000","last_became_active":"2024-04-24T05:47:42.525150+0000","last_became_peered":"2024-04-24T05:47:42.525150+0000","last_unstale":"2024-04-24T05:47:43.553435+0000","last_undegraded":"2024-04-24T05:47:43.553435+0000","last_fullsized":"2024-04-24T05:47:43.553435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:58.112486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.540624+0000","last_change":"2024-04-24T05:47:43.532909+0000","last_active":"2024-04-24T05:47:43.540624+0000","last_peered":"2024-04-24T05:47:43.540624+0000","last_clean":"2024-04-24T05:47:43.540624+0000","last_became_active":"2024-04-24T05:47:43.532496+0000","last_became_peered":"2024-04-24T05:47:43.532496+0000","last_unstale":"2024-04-24T05:47:43.540624+0000","last_undegraded":"2024-04-24T05:47:43.540624+0000","last_fullsized":"2024-04-24T05:47:43.540624+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:14:18.610223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.554422+0000","last_change":"2024-04-24T05:47:42.536293+0000","last_active":"2024-04-24T05:47:43.554422+0000","last_peered":"2024-04-24T05:47:43.554422+0000","last_clean":"2024-04-24T05:47:43.554422+0000","last_became_active":"2024-04-24T05:47:42.535824+0000","last_became_peered":"2024-04-24T05:47:42.535824+0000","last_unstale":"2024-04-24T05:47:43.554422+0000","last_undegraded":"2024-04-24T05:47:43.554422+0000","last_fullsized":"2024-04-24T05:47:43.554422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:54:18.237837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.545348+0000","last_change":"2024-04-24T05:47:43.539477+0000","last_active":"2024-04-24T05:47:43.545348+0000","last_peered":"2024-04-24T05:47:43.545348+0000","last_clean":"2024-04-24T05:47:43.545348+0000","last_became_active":"2024-04-24T05:47:43.536101+0000","last_became_peered":"2024-04-24T05:47:43.536101+0000","last_unstale":"2024-04-24T05:47:43.545348+0000","last_undegraded":"2024-04-24T05:47:43.545348+0000","last_fullsized":"2024-04-24T05:47:43.545348+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:54:19.251964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.521139+0000","last_change":"2024-04-24T05:47:42.526187+0000","last_active":"2024-04-24T05:47:43.521139+0000","last_peered":"2024-04-24T05:47:43.521139+0000","last_clean":"2024-04-24T05:47:43.521139+0000","last_became_active":"2024-04-24T05:47:42.525891+0000","last_became_peered":"2024-04-24T05:47:42.525891+0000","last_unstale":"2024-04-24T05:47:43.521139+0000","last_undegraded":"2024-04-24T05:47:43.521139+0000","last_fullsized":"2024-04-24T05:47:43.521139+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:08.633627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.550965+0000","last_change":"2024-04-24T05:47:43.539971+0000","last_active":"2024-04-24T05:47:43.550965+0000","last_peered":"2024-04-24T05:47:43.550965+0000","last_clean":"2024-04-24T05:47:43.550965+0000","last_became_active":"2024-04-24T05:47:43.539203+0000","last_became_peered":"2024-04-24T05:47:43.539203+0000","last_unstale":"2024-04-24T05:47:43.550965+0000","last_undegraded":"2024-04-24T05:47:43.550965+0000","last_fullsized":"2024-04-24T05:47:43.550965+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:21:10.691766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.516582+0000","last_change":"2024-04-24T05:47:42.530677+0000","last_active":"2024-04-24T05:47:43.516582+0000","last_peered":"2024-04-24T05:47:43.516582+0000","last_clean":"2024-04-24T05:47:43.516582+0000","last_became_active":"2024-04-24T05:47:42.530324+0000","last_became_peered":"2024-04-24T05:47:42.530324+0000","last_unstale":"2024-04-24T05:47:43.516582+0000","last_undegraded":"2024-04-24T05:47:43.516582+0000","last_fullsized":"2024-04-24T05:47:43.516582+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:43:36.839599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544015+0000","last_change":"2024-04-24T05:47:43.531801+0000","last_active":"2024-04-24T05:47:43.544015+0000","last_peered":"2024-04-24T05:47:43.544015+0000","last_clean":"2024-04-24T05:47:43.544015+0000","last_became_active":"2024-04-24T05:47:43.531453+0000","last_became_peered":"2024-04-24T05:47:43.531453+0000","last_unstale":"2024-04-24T05:47:43.544015+0000","last_undegraded":"2024-04-24T05:47:43.544015+0000","last_fullsized":"2024-04-24T05:47:43.544015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:00:50.989959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.525208+0000","last_change":"2024-04-24T05:47:42.520876+0000","last_active":"2024-04-24T05:47:43.525208+0000","last_peered":"2024-04-24T05:47:43.525208+0000","last_clean":"2024-04-24T05:47:43.525208+0000","last_became_active":"2024-04-24T05:47:42.520589+0000","last_became_peered":"2024-04-24T05:47:42.520589+0000","last_unstale":"2024-04-24T05:47:43.525208+0000","last_undegraded":"2024-04-24T05:47:43.525208+0000","last_fullsized":"2024-04-24T05:47:43.525208+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:25.180697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.547190+0000","last_change":"2024-04-24T05:47:43.543175+0000","last_active":"2024-04-24T05:47:43.547190+0000","last_peered":"2024-04-24T05:47:43.547190+0000","last_clean":"2024-04-24T05:47:43.547190+0000","last_became_active":"2024-04-24T05:47:43.540460+0000","last_became_peered":"2024-04-24T05:47:43.540460+0000","last_unstale":"2024-04-24T05:47:43.547190+0000","last_undegraded":"2024-04-24T05:47:43.547190+0000","last_fullsized":"2024-04-24T05:47:43.547190+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:18:56.906868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.520613+0000","last_change":"2024-04-24T05:47:42.522227+0000","last_active":"2024-04-24T05:47:43.520613+0000","last_peered":"2024-04-24T05:47:43.520613+0000","last_clean":"2024-04-24T05:47:43.520613+0000","last_became_active":"2024-04-24T05:47:42.520224+0000","last_became_peered":"2024-04-24T05:47:42.520224+0000","last_unstale":"2024-04-24T05:47:43.520613+0000","last_undegraded":"2024-04-24T05:47:43.520613+0000","last_fullsized":"2024-04-24T05:47:43.520613+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:56.659470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.538983+0000","last_change":"2024-04-24T05:47:43.532331+0000","last_active":"2024-04-24T05:47:43.538983+0000","last_peered":"2024-04-24T05:47:43.538983+0000","last_clean":"2024-04-24T05:47:43.538983+0000","last_became_active":"2024-04-24T05:47:43.532005+0000","last_became_peered":"2024-04-24T05:47:43.532005+0000","last_unstale":"2024-04-24T05:47:43.538983+0000","last_undegraded":"2024-04-24T05:47:43.538983+0000","last_fullsized":"2024-04-24T05:47:43.538983+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:59.126614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.554627+0000","last_change":"2024-04-24T05:47:42.519108+0000","last_active":"2024-04-24T05:47:43.554627+0000","last_peered":"2024-04-24T05:47:43.554627+0000","last_clean":"2024-04-24T05:47:43.554627+0000","last_became_active":"2024-04-24T05:47:42.518833+0000","last_became_peered":"2024-04-24T05:47:42.518833+0000","last_unstale":"2024-04-24T05:47:43.554627+0000","last_undegraded":"2024-04-24T05:47:43.554627+0000","last_fullsized":"2024-04-24T05:47:43.554627+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:25.180697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.548991+0000","last_change":"2024-04-24T05:47:43.534485+0000","last_active":"2024-04-24T05:47:43.548991+0000","last_peered":"2024-04-24T05:47:43.548991+0000","last_clean":"2024-04-24T05:47:43.548991+0000","last_became_active":"2024-04-24T05:47:43.531899+0000","last_became_peered":"2024-04-24T05:47:43.531899+0000","last_unstale":"2024-04-24T05:47:43.548991+0000","last_undegraded":"2024-04-24T05:47:43.548991+0000","last_fullsized":"2024-04-24T05:47:43.548991+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:30:09.647755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.556473+0000","last_change":"2024-04-24T05:47:42.527384+0000","last_active":"2024-04-24T05:47:43.556473+0000","last_peered":"2024-04-24T05:47:43.556473+0000","last_clean":"2024-04-24T05:47:43.556473+0000","last_became_active":"2024-04-24T05:47:42.527066+0000","last_became_peered":"2024-04-24T05:47:42.527066+0000","last_unstale":"2024-04-24T05:47:43.556473+0000","last_undegraded":"2024-04-24T05:47:43.556473+0000","last_fullsized":"2024-04-24T05:47:43.556473+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:54:18.237837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.539272+0000","last_change":"2024-04-24T05:47:43.532839+0000","last_active":"2024-04-24T05:47:43.539272+0000","last_peered":"2024-04-24T05:47:43.539272+0000","last_clean":"2024-04-24T05:47:43.539272+0000","last_became_active":"2024-04-24T05:47:43.531694+0000","last_became_peered":"2024-04-24T05:47:43.531694+0000","last_unstale":"2024-04-24T05:47:43.539272+0000","last_undegraded":"2024-04-24T05:47:43.539272+0000","last_fullsized":"2024-04-24T05:47:43.539272+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:26.194824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.556042+0000","last_change":"2024-04-24T05:47:42.522788+0000","last_active":"2024-04-24T05:47:43.556042+0000","last_peered":"2024-04-24T05:47:43.556042+0000","last_clean":"2024-04-24T05:47:43.556042+0000","last_became_active":"2024-04-24T05:47:42.522202+0000","last_became_peered":"2024-04-24T05:47:42.522202+0000","last_unstale":"2024-04-24T05:47:43.556042+0000","last_undegraded":"2024-04-24T05:47:43.556042+0000","last_fullsized":"2024-04-24T05:47:43.556042+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:58.112486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.541792+0000","last_change":"2024-04-24T05:47:43.536247+0000","last_active":"2024-04-24T05:47:43.541792+0000","last_peered":"2024-04-24T05:47:43.541792+0000","last_clean":"2024-04-24T05:47:43.541792+0000","last_became_active":"2024-04-24T05:47:43.533839+0000","last_became_peered":"2024-04-24T05:47:43.533839+0000","last_unstale":"2024-04-24T05:47:43.541792+0000","last_undegraded":"2024-04-24T05:47:43.541792+0000","last_fullsized":"2024-04-24T05:47:43.541792+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:33:54.283092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.521418+0000","last_change":"2024-04-24T05:47:42.524094+0000","last_active":"2024-04-24T05:47:43.521418+0000","last_peered":"2024-04-24T05:47:43.521418+0000","last_clean":"2024-04-24T05:47:43.521418+0000","last_became_active":"2024-04-24T05:47:42.523855+0000","last_became_peered":"2024-04-24T05:47:42.523855+0000","last_unstale":"2024-04-24T05:47:43.521418+0000","last_undegraded":"2024-04-24T05:47:43.521418+0000","last_fullsized":"2024-04-24T05:47:43.521418+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:28.292171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.549773+0000","last_change":"2024-04-24T05:47:43.547017+0000","last_active":"2024-04-24T05:47:43.549773+0000","last_peered":"2024-04-24T05:47:43.549773+0000","last_clean":"2024-04-24T05:47:43.549773+0000","last_became_active":"2024-04-24T05:47:43.546639+0000","last_became_peered":"2024-04-24T05:47:43.546639+0000","last_unstale":"2024-04-24T05:47:43.549773+0000","last_undegraded":"2024-04-24T05:47:43.549773+0000","last_fullsized":"2024-04-24T05:47:43.549773+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:25:10.115913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.503480+0000","last_change":"2024-04-24T05:47:42.533940+0000","last_active":"2024-04-24T05:47:43.503480+0000","last_peered":"2024-04-24T05:47:43.503480+0000","last_clean":"2024-04-24T05:47:43.503480+0000","last_became_active":"2024-04-24T05:47:42.533657+0000","last_became_peered":"2024-04-24T05:47:42.533657+0000","last_unstale":"2024-04-24T05:47:43.503480+0000","last_undegraded":"2024-04-24T05:47:43.503480+0000","last_fullsized":"2024-04-24T05:47:43.503480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:43:36.839599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.540544+0000","last_change":"2024-04-24T05:47:43.534454+0000","last_active":"2024-04-24T05:47:43.540544+0000","last_peered":"2024-04-24T05:47:43.540544+0000","last_clean":"2024-04-24T05:47:43.540544+0000","last_became_active":"2024-04-24T05:47:43.532646+0000","last_became_peered":"2024-04-24T05:47:43.532646+0000","last_unstale":"2024-04-24T05:47:43.540544+0000","last_undegraded":"2024-04-24T05:47:43.540544+0000","last_fullsized":"2024-04-24T05:47:43.540544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:14:18.610223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.555396+0000","last_change":"2024-04-24T05:47:42.536450+0000","last_active":"2024-04-24T05:47:43.555396+0000","last_peered":"2024-04-24T05:47:43.555396+0000","last_clean":"2024-04-24T05:47:43.555396+0000","last_became_active":"2024-04-24T05:47:42.535464+0000","last_became_peered":"2024-04-24T05:47:42.535464+0000","last_unstale":"2024-04-24T05:47:43.555396+0000","last_undegraded":"2024-04-24T05:47:43.555396+0000","last_fullsized":"2024-04-24T05:47:43.555396+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:56.659470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.550459+0000","last_change":"2024-04-24T05:47:43.542694+0000","last_active":"2024-04-24T05:47:43.550459+0000","last_peered":"2024-04-24T05:47:43.550459+0000","last_clean":"2024-04-24T05:47:43.550459+0000","last_became_active":"2024-04-24T05:47:43.542427+0000","last_became_peered":"2024-04-24T05:47:43.542427+0000","last_unstale":"2024-04-24T05:47:43.550459+0000","last_undegraded":"2024-04-24T05:47:43.550459+0000","last_fullsized":"2024-04-24T05:47:43.550459+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:17:36.426617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.554592+0000","last_change":"2024-04-24T05:47:42.536077+0000","last_active":"2024-04-24T05:47:43.554592+0000","last_peered":"2024-04-24T05:47:43.554592+0000","last_clean":"2024-04-24T05:47:43.554592+0000","last_became_active":"2024-04-24T05:47:42.535513+0000","last_became_peered":"2024-04-24T05:47:42.535513+0000","last_unstale":"2024-04-24T05:47:43.554592+0000","last_undegraded":"2024-04-24T05:47:43.554592+0000","last_fullsized":"2024-04-24T05:47:43.554592+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:15.459009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.538321+0000","last_change":"2024-04-24T05:47:43.534212+0000","last_active":"2024-04-24T05:47:43.538321+0000","last_peered":"2024-04-24T05:47:43.538321+0000","last_clean":"2024-04-24T05:47:43.538321+0000","last_became_active":"2024-04-24T05:47:43.533887+0000","last_became_peered":"2024-04-24T05:47:43.533887+0000","last_unstale":"2024-04-24T05:47:43.538321+0000","last_undegraded":"2024-04-24T05:47:43.538321+0000","last_fullsized":"2024-04-24T05:47:43.538321+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:34:32.710562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.553388+0000","last_change":"2024-04-24T05:47:42.530588+0000","last_active":"2024-04-24T05:47:43.553388+0000","last_peered":"2024-04-24T05:47:43.553388+0000","last_clean":"2024-04-24T05:47:43.553388+0000","last_became_active":"2024-04-24T05:47:42.530057+0000","last_became_peered":"2024-04-24T05:47:42.530057+0000","last_unstale":"2024-04-24T05:47:43.553388+0000","last_undegraded":"2024-04-24T05:47:43.553388+0000","last_fullsized":"2024-04-24T05:47:43.553388+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:00.321752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.545019+0000","last_change":"2024-04-24T05:47:43.535568+0000","last_active":"2024-04-24T05:47:43.545019+0000","last_peered":"2024-04-24T05:47:43.545019+0000","last_clean":"2024-04-24T05:47:43.545019+0000","last_became_active":"2024-04-24T05:47:43.532446+0000","last_became_peered":"2024-04-24T05:47:43.532446+0000","last_unstale":"2024-04-24T05:47:43.545019+0000","last_undegraded":"2024-04-24T05:47:43.545019+0000","last_fullsized":"2024-04-24T05:47:43.545019+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:29.306299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.548629+0000","last_change":"2024-04-24T05:47:42.525010+0000","last_active":"2024-04-24T05:47:43.548629+0000","last_peered":"2024-04-24T05:47:43.548629+0000","last_clean":"2024-04-24T05:47:43.548629+0000","last_became_active":"2024-04-24T05:47:42.523442+0000","last_became_peered":"2024-04-24T05:47:42.523442+0000","last_unstale":"2024-04-24T05:47:43.548629+0000","last_undegraded":"2024-04-24T05:47:43.548629+0000","last_fullsized":"2024-04-24T05:47:43.548629+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:34:57.869333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.548498+0000","last_change":"2024-04-24T05:47:43.537238+0000","last_active":"2024-04-24T05:47:43.548498+0000","last_peered":"2024-04-24T05:47:43.548498+0000","last_clean":"2024-04-24T05:47:43.548498+0000","last_became_active":"2024-04-24T05:47:43.536876+0000","last_became_peered":"2024-04-24T05:47:43.536876+0000","last_unstale":"2024-04-24T05:47:43.548498+0000","last_undegraded":"2024-04-24T05:47:43.548498+0000","last_fullsized":"2024-04-24T05:47:43.548498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:30:56.456827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.522241+0000","last_change":"2024-04-24T05:47:42.529983+0000","last_active":"2024-04-24T05:47:43.522241+0000","last_peered":"2024-04-24T05:47:43.522241+0000","last_clean":"2024-04-24T05:47:43.522241+0000","last_became_active":"2024-04-24T05:47:42.529524+0000","last_became_peered":"2024-04-24T05:47:42.529524+0000","last_unstale":"2024-04-24T05:47:43.522241+0000","last_undegraded":"2024-04-24T05:47:43.522241+0000","last_fullsized":"2024-04-24T05:47:43.522241+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:52:56.659470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.537009+0000","last_change":"2024-04-24T05:47:43.530928+0000","last_active":"2024-04-24T05:47:43.537009+0000","last_peered":"2024-04-24T05:47:43.537009+0000","last_clean":"2024-04-24T05:47:43.537009+0000","last_became_active":"2024-04-24T05:47:43.530555+0000","last_became_peered":"2024-04-24T05:47:43.530555+0000","last_unstale":"2024-04-24T05:47:43.537009+0000","last_undegraded":"2024-04-24T05:47:43.537009+0000","last_fullsized":"2024-04-24T05:47:43.537009+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:26.194824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.548680+0000","last_change":"2024-04-24T05:47:42.524423+0000","last_active":"2024-04-24T05:47:43.548680+0000","last_peered":"2024-04-24T05:47:43.548680+0000","last_clean":"2024-04-24T05:47:43.548680+0000","last_became_active":"2024-04-24T05:47:42.524143+0000","last_became_peered":"2024-04-24T05:47:42.524143+0000","last_unstale":"2024-04-24T05:47:43.548680+0000","last_undegraded":"2024-04-24T05:47:43.548680+0000","last_fullsized":"2024-04-24T05:47:43.548680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:44:07.195469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.543590+0000","last_change":"2024-04-24T05:47:43.538671+0000","last_active":"2024-04-24T05:47:43.543590+0000","last_peered":"2024-04-24T05:47:43.543590+0000","last_clean":"2024-04-24T05:47:43.543590+0000","last_became_active":"2024-04-24T05:47:43.538051+0000","last_became_peered":"2024-04-24T05:47:43.538051+0000","last_unstale":"2024-04-24T05:47:43.543590+0000","last_undegraded":"2024-04-24T05:47:43.543590+0000","last_fullsized":"2024-04-24T05:47:43.543590+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:49:26.443913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.500297+0000","last_change":"2024-04-24T05:47:42.535958+0000","last_active":"2024-04-24T05:47:43.500297+0000","last_peered":"2024-04-24T05:47:43.500297+0000","last_clean":"2024-04-24T05:47:43.500297+0000","last_became_active":"2024-04-24T05:47:42.535518+0000","last_became_peered":"2024-04-24T05:47:42.535518+0000","last_unstale":"2024-04-24T05:47:43.500297+0000","last_undegraded":"2024-04-24T05:47:43.500297+0000","last_fullsized":"2024-04-24T05:47:43.500297+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:52:19.757263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.538399+0000","last_change":"2024-04-24T05:47:43.531515+0000","last_active":"2024-04-24T05:47:43.538399+0000","last_peered":"2024-04-24T05:47:43.538399+0000","last_clean":"2024-04-24T05:47:43.538399+0000","last_became_active":"2024-04-24T05:47:43.531054+0000","last_became_peered":"2024-04-24T05:47:43.531054+0000","last_unstale":"2024-04-24T05:47:43.538399+0000","last_undegraded":"2024-04-24T05:47:43.538399+0000","last_fullsized":"2024-04-24T05:47:43.538399+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:49:26.443913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.522641+0000","last_change":"2024-04-24T05:47:42.526989+0000","last_active":"2024-04-24T05:47:43.522641+0000","last_peered":"2024-04-24T05:47:43.522641+0000","last_clean":"2024-04-24T05:47:43.522641+0000","last_became_active":"2024-04-24T05:47:42.526694+0000","last_became_peered":"2024-04-24T05:47:42.526694+0000","last_unstale":"2024-04-24T05:47:43.522641+0000","last_undegraded":"2024-04-24T05:47:43.522641+0000","last_fullsized":"2024-04-24T05:47:43.522641+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:15.459009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.550944+0000","last_change":"2024-04-24T05:47:43.539677+0000","last_active":"2024-04-24T05:47:43.550944+0000","last_peered":"2024-04-24T05:47:43.550944+0000","last_clean":"2024-04-24T05:47:43.550944+0000","last_became_active":"2024-04-24T05:47:43.538868+0000","last_became_peered":"2024-04-24T05:47:43.538868+0000","last_unstale":"2024-04-24T05:47:43.550944+0000","last_undegraded":"2024-04-24T05:47:43.550944+0000","last_fullsized":"2024-04-24T05:47:43.550944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:18:56.906868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.501455+0000","last_change":"2024-04-24T05:47:42.517297+0000","last_active":"2024-04-24T05:47:43.501455+0000","last_peered":"2024-04-24T05:47:43.501455+0000","last_clean":"2024-04-24T05:47:43.501455+0000","last_became_active":"2024-04-24T05:47:42.516932+0000","last_became_peered":"2024-04-24T05:47:42.516932+0000","last_unstale":"2024-04-24T05:47:43.501455+0000","last_undegraded":"2024-04-24T05:47:43.501455+0000","last_fullsized":"2024-04-24T05:47:43.501455+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:58:34.506495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.543113+0000","last_change":"2024-04-24T05:47:43.537034+0000","last_active":"2024-04-24T05:47:43.543113+0000","last_peered":"2024-04-24T05:47:43.543113+0000","last_clean":"2024-04-24T05:47:43.543113+0000","last_became_active":"2024-04-24T05:47:43.536689+0000","last_became_peered":"2024-04-24T05:47:43.536689+0000","last_unstale":"2024-04-24T05:47:43.543113+0000","last_undegraded":"2024-04-24T05:47:43.543113+0000","last_fullsized":"2024-04-24T05:47:43.543113+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:26:26.194824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.554166+0000","last_change":"2024-04-24T05:47:42.519722+0000","last_active":"2024-04-24T05:47:43.554166+0000","last_peered":"2024-04-24T05:47:43.554166+0000","last_clean":"2024-04-24T05:47:43.554166+0000","last_became_active":"2024-04-24T05:47:42.517547+0000","last_became_peered":"2024-04-24T05:47:42.517547+0000","last_unstale":"2024-04-24T05:47:43.554166+0000","last_undegraded":"2024-04-24T05:47:43.554166+0000","last_fullsized":"2024-04-24T05:47:43.554166+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:54:18.237837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.547167+0000","last_change":"2024-04-24T05:47:43.542515+0000","last_active":"2024-04-24T05:47:43.547167+0000","last_peered":"2024-04-24T05:47:43.547167+0000","last_clean":"2024-04-24T05:47:43.547167+0000","last_became_active":"2024-04-24T05:47:43.540837+0000","last_became_peered":"2024-04-24T05:47:43.540837+0000","last_unstale":"2024-04-24T05:47:43.547167+0000","last_undegraded":"2024-04-24T05:47:43.547167+0000","last_fullsized":"2024-04-24T05:47:43.547167+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:34:32.710562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.501269+0000","last_change":"2024-04-24T05:47:42.520876+0000","last_active":"2024-04-24T05:47:43.501269+0000","last_peered":"2024-04-24T05:47:43.501269+0000","last_clean":"2024-04-24T05:47:43.501269+0000","last_became_active":"2024-04-24T05:47:42.520234+0000","last_became_peered":"2024-04-24T05:47:42.520234+0000","last_unstale":"2024-04-24T05:47:43.501269+0000","last_undegraded":"2024-04-24T05:47:43.501269+0000","last_fullsized":"2024-04-24T05:47:43.501269+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:18.866010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.537620+0000","last_change":"2024-04-24T05:47:43.534027+0000","last_active":"2024-04-24T05:47:43.537620+0000","last_peered":"2024-04-24T05:47:43.537620+0000","last_clean":"2024-04-24T05:47:43.537620+0000","last_became_active":"2024-04-24T05:47:43.532634+0000","last_became_peered":"2024-04-24T05:47:43.532634+0000","last_unstale":"2024-04-24T05:47:43.537620+0000","last_undegraded":"2024-04-24T05:47:43.537620+0000","last_fullsized":"2024-04-24T05:47:43.537620+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:49:26.443913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.514524+0000","last_change":"2024-04-24T05:47:42.519666+0000","last_active":"2024-04-24T05:47:43.514524+0000","last_peered":"2024-04-24T05:47:43.514524+0000","last_clean":"2024-04-24T05:47:43.514524+0000","last_became_active":"2024-04-24T05:47:42.519374+0000","last_became_peered":"2024-04-24T05:47:42.519374+0000","last_unstale":"2024-04-24T05:47:43.514524+0000","last_undegraded":"2024-04-24T05:47:43.514524+0000","last_fullsized":"2024-04-24T05:47:43.514524+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:36:28.292171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.537237+0000","last_change":"2024-04-24T05:47:43.529861+0000","last_active":"2024-04-24T05:47:43.537237+0000","last_peered":"2024-04-24T05:47:43.537237+0000","last_clean":"2024-04-24T05:47:43.537237+0000","last_became_active":"2024-04-24T05:47:43.528889+0000","last_became_peered":"2024-04-24T05:47:43.528889+0000","last_unstale":"2024-04-24T05:47:43.537237+0000","last_undegraded":"2024-04-24T05:47:43.537237+0000","last_fullsized":"2024-04-24T05:47:43.537237+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:54:19.251964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.512625+0000","last_change":"2024-04-24T05:47:42.516031+0000","last_active":"2024-04-24T05:47:43.512625+0000","last_peered":"2024-04-24T05:47:43.512625+0000","last_clean":"2024-04-24T05:47:43.512625+0000","last_became_active":"2024-04-24T05:47:42.515701+0000","last_became_peered":"2024-04-24T05:47:42.515701+0000","last_unstale":"2024-04-24T05:47:43.512625+0000","last_undegraded":"2024-04-24T05:47:43.512625+0000","last_fullsized":"2024-04-24T05:47:43.512625+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:22:51.396214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.541776+0000","last_change":"2024-04-24T05:47:43.532188+0000","last_active":"2024-04-24T05:47:43.541776+0000","last_peered":"2024-04-24T05:47:43.541776+0000","last_clean":"2024-04-24T05:47:43.541776+0000","last_became_active":"2024-04-24T05:47:43.530759+0000","last_became_peered":"2024-04-24T05:47:43.530759+0000","last_unstale":"2024-04-24T05:47:43.541776+0000","last_undegraded":"2024-04-24T05:47:43.541776+0000","last_fullsized":"2024-04-24T05:47:43.541776+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:54:19.251964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.504907+0000","last_change":"2024-04-24T05:47:42.524200+0000","last_active":"2024-04-24T05:47:43.504907+0000","last_peered":"2024-04-24T05:47:43.504907+0000","last_clean":"2024-04-24T05:47:43.504907+0000","last_became_active":"2024-04-24T05:47:42.523356+0000","last_became_peered":"2024-04-24T05:47:42.523356+0000","last_unstale":"2024-04-24T05:47:43.504907+0000","last_undegraded":"2024-04-24T05:47:43.504907+0000","last_fullsized":"2024-04-24T05:47:43.504907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:25.030481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.544945+0000","last_change":"2024-04-24T05:47:43.532567+0000","last_active":"2024-04-24T05:47:43.544945+0000","last_peered":"2024-04-24T05:47:43.544945+0000","last_clean":"2024-04-24T05:47:43.544945+0000","last_became_active":"2024-04-24T05:47:43.529947+0000","last_became_peered":"2024-04-24T05:47:43.529947+0000","last_unstale":"2024-04-24T05:47:43.544945+0000","last_undegraded":"2024-04-24T05:47:43.544945+0000","last_fullsized":"2024-04-24T05:47:43.544945+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:51:08.032102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.517525+0000","last_change":"2024-04-24T05:47:42.526473+0000","last_active":"2024-04-24T05:47:43.517525+0000","last_peered":"2024-04-24T05:47:43.517525+0000","last_clean":"2024-04-24T05:47:43.517525+0000","last_became_active":"2024-04-24T05:47:42.526206+0000","last_became_peered":"2024-04-24T05:47:42.526206+0000","last_unstale":"2024-04-24T05:47:43.517525+0000","last_undegraded":"2024-04-24T05:47:43.517525+0000","last_fullsized":"2024-04-24T05:47:43.517525+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:52:25.030481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.540324+0000","last_change":"2024-04-24T05:47:43.537238+0000","last_active":"2024-04-24T05:47:43.540324+0000","last_peered":"2024-04-24T05:47:43.540324+0000","last_clean":"2024-04-24T05:47:43.540324+0000","last_became_active":"2024-04-24T05:47:43.536913+0000","last_became_peered":"2024-04-24T05:47:43.536913+0000","last_unstale":"2024-04-24T05:47:43.540324+0000","last_undegraded":"2024-04-24T05:47:43.540324+0000","last_fullsized":"2024-04-24T05:47:43.540324+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:45.002651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.523708+0000","last_change":"2024-04-24T05:47:42.522073+0000","last_active":"2024-04-24T05:47:43.523708+0000","last_peered":"2024-04-24T05:47:43.523708+0000","last_clean":"2024-04-24T05:47:43.523708+0000","last_became_active":"2024-04-24T05:47:42.521791+0000","last_became_peered":"2024-04-24T05:47:42.521791+0000","last_unstale":"2024-04-24T05:47:43.523708+0000","last_undegraded":"2024-04-24T05:47:43.523708+0000","last_fullsized":"2024-04-24T05:47:43.523708+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:15.459009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.545892+0000","last_change":"2024-04-24T05:47:43.539201+0000","last_active":"2024-04-24T05:47:43.545892+0000","last_peered":"2024-04-24T05:47:43.545892+0000","last_clean":"2024-04-24T05:47:43.545892+0000","last_became_active":"2024-04-24T05:47:43.538881+0000","last_became_peered":"2024-04-24T05:47:43.538881+0000","last_unstale":"2024-04-24T05:47:43.545892+0000","last_undegraded":"2024-04-24T05:47:43.545892+0000","last_fullsized":"2024-04-24T05:47:43.545892+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:42.474766+0000","last_clean_scrub_stamp":"2024-04-24T05:47:42.474766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:25:10.115913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.525919+0000","last_change":"2024-04-24T05:47:42.532510+0000","last_active":"2024-04-24T05:47:43.525919+0000","last_peered":"2024-04-24T05:47:43.525919+0000","last_clean":"2024-04-24T05:47:43.525919+0000","last_became_active":"2024-04-24T05:47:42.530821+0000","last_became_peered":"2024-04-24T05:47:42.530821+0000","last_unstale":"2024-04-24T05:47:43.525919+0000","last_undegraded":"2024-04-24T05:47:43.525919+0000","last_fullsized":"2024-04-24T05:47:43.525919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:47:41.460638+0000","last_clean_scrub_stamp":"2024-04-24T05:47:41.460638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:58.112486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-24T05:47:43.517450+0000","last_change":"2024-04-24T05:45:51.019903+0000","last_active":"2024-04-24T05:47:43.517450+0000","last_peered":"2024-04-24T05:47:43.517450+0000","last_clean":"2024-04-24T05:47:43.517450+0000","last_became_active":"2024-04-24T05:45:51.017323+0000","last_became_peered":"2024-04-24T05:45:51.017323+0000","last_unstale":"2024-04-24T05:47:43.517450+0000","last_undegraded":"2024-04-24T05:47:43.517450+0000","last_fullsized":"2024-04-24T05:47:43.517450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T05:45:48.855871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T05:45:48.855871+0000","last_clean_scrub_stamp":"2024-04-24T05:45:48.855871+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:10:41.045203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":771626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":181258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":181258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":181258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":181258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":181258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":181258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":771626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T05:47:48.431 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T05:47:48.431 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-24T05:47:48.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.695+0000 7fbe11600640 1 Processor -- start 2024-04-24T05:47:48.697 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.696+0000 7fbe11600640 1 -- start start 2024-04-24T05:47:48.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.697+0000 7fbe11600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe0c147e70 0x7fbe0c0acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:48.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.697+0000 7fbe11600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbe0c162ce0 0x7fbe0c0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:48.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.697+0000 7fbe11600640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbe0c15ea60 0x7fbe0c15eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:48.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.697+0000 7fbe11600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbe0c162850 con 0x7fbe0c162ce0 2024-04-24T05:47:48.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.697+0000 7fbe11600640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbe0c15f7e0 con 0x7fbe0c15ea60 2024-04-24T05:47:48.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.697+0000 7fbe11600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbe0c162200 con 0x7fbe0c147e70 2024-04-24T05:47:48.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.697+0000 7fbe03e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbe0c162ce0 0x7fbe0c0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:48.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.698+0000 7fbe03e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbe0c162ce0 0x7fbe0c0ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38972/0 (socket says 172.21.15.123:38972) 2024-04-24T05:47:48.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.698+0000 7fbe03e00640 1 -- 172.21.15.123:0/3454966489 learned_addr learned my addr 172.21.15.123:0/3454966489 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:48.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.698+0000 7fbe0be00640 1 --2- 172.21.15.123:0/3454966489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe0c147e70 0x7fbe0c0acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:48.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.698+0000 7fbe10c00640 1 --2- 172.21.15.123:0/3454966489 >> v2:172.21.15.155:3300/0 conn(0x7fbe0c15ea60 0x7fbe0c15eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:48.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.698+0000 7fbe03e00640 1 -- 172.21.15.123:0/3454966489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe0c147e70 msgr2=0x7fbe0c0acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:48.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.698+0000 7fbe03e00640 1 --2- 172.21.15.123:0/3454966489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe0c147e70 0x7fbe0c0acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.698+0000 7fbe03e00640 1 -- 172.21.15.123:0/3454966489 >> v2:172.21.15.155:3300/0 conn(0x7fbe0c15ea60 msgr2=0x7fbe0c15eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:48.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.698+0000 7fbe03e00640 1 --2- 172.21.15.123:0/3454966489 >> v2:172.21.15.155:3300/0 conn(0x7fbe0c15ea60 0x7fbe0c15eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.698+0000 7fbe03e00640 1 -- 172.21.15.123:0/3454966489 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c1624e0 con 0x7fbe0c162ce0 2024-04-24T05:47:48.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.698+0000 7fbe10c00640 1 --2- 172.21.15.123:0/3454966489 >> v2:172.21.15.155:3300/0 conn(0x7fbe0c15ea60 0x7fbe0c15eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:48.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.699+0000 7fbe03e00640 1 --2- 172.21.15.123:0/3454966489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe0c162ce0 0x7fbe0c0ad480 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c071940 tx=0x7fbdf402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e69b0e30011fd86 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:48.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.700+0000 7fbe0b400640 1 -- 172.21.15.123:0/3454966489 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4037070 con 0x7fbe0c162ce0 2024-04-24T05:47:48.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.700+0000 7fbe0b400640 1 -- 172.21.15.123:0/3454966489 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf4032440 con 0x7fbe0c162ce0 2024-04-24T05:47:48.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.700+0000 7fbe0b400640 1 -- 172.21.15.123:0/3454966489 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4032810 con 0x7fbe0c162ce0 2024-04-24T05:47:48.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.700+0000 7fbe11600640 1 -- 172.21.15.123:0/3454966489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe0c162ce0 msgr2=0x7fbe0c0ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:48.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.700+0000 7fbe11600640 1 --2- 172.21.15.123:0/3454966489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe0c162ce0 0x7fbe0c0ad480 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c071940 tx=0x7fbdf402a140 comp rx=0 tx=0).stop 2024-04-24T05:47:48.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.700+0000 7fbe11600640 1 -- 172.21.15.123:0/3454966489 shutdown_connections 2024-04-24T05:47:48.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.700+0000 7fbe11600640 1 --2- 172.21.15.123:0/3454966489 >> v2:172.21.15.155:3300/0 conn(0x7fbe0c15ea60 0x7fbe0c15eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.700+0000 7fbe11600640 1 --2- 172.21.15.123:0/3454966489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe0c162ce0 0x7fbe0c0ad480 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.700+0000 7fbe11600640 1 --2- 172.21.15.123:0/3454966489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe0c147e70 0x7fbe0c0acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.700+0000 7fbe11600640 1 -- 172.21.15.123:0/3454966489 >> 172.21.15.123:0/3454966489 conn(0x7fbe0c0b1e40 msgr2=0x7fbe0c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:48.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.701+0000 7fbe11600640 1 -- 172.21.15.123:0/3454966489 shutdown_connections 2024-04-24T05:47:48.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.701+0000 7fbe11600640 1 -- 172.21.15.123:0/3454966489 wait complete. 2024-04-24T05:47:48.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.702+0000 7fbe11600640 1 Processor -- start 2024-04-24T05:47:48.704 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.703+0000 7fbe11600640 1 -- start start 2024-04-24T05:47:48.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.703+0000 7fbe11600640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe0c147e70 0x7fbe0c09d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:48.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe11600640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe0c15ea60 0x7fbe0c15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:48.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe11600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe0c162ce0 0x7fbe0c140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:48.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe11600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c15fdb0 con 0x7fbe0c162ce0 2024-04-24T05:47:48.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe11600640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c15fc30 con 0x7fbe0c147e70 2024-04-24T05:47:48.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe11600640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbe0c15ff30 con 0x7fbe0c15ea60 2024-04-24T05:47:48.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe10c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe0c162ce0 0x7fbe0c140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:48.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe10c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe0c162ce0 0x7fbe0c140fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38974/0 (socket says 172.21.15.123:38974) 2024-04-24T05:47:48.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe10c00640 1 -- 172.21.15.123:0/3350623800 learned_addr learned my addr 172.21.15.123:0/3350623800 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:48.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe03e00640 1 --2- 172.21.15.123:0/3350623800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe0c15ea60 0x7fbe0c15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:48.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe0be00640 1 --2- 172.21.15.123:0/3350623800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe0c147e70 0x7fbe0c09d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:48.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe10c00640 1 -- 172.21.15.123:0/3350623800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe0c15ea60 msgr2=0x7fbe0c15dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:48.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe10c00640 1 --2- 172.21.15.123:0/3350623800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe0c15ea60 0x7fbe0c15dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe10c00640 1 -- 172.21.15.123:0/3350623800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe0c147e70 msgr2=0x7fbe0c09d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:48.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe10c00640 1 --2- 172.21.15.123:0/3350623800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe0c147e70 0x7fbe0c09d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.704+0000 7fbe10c00640 1 -- 172.21.15.123:0/3350623800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c09bf00 con 0x7fbe0c162ce0 2024-04-24T05:47:48.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.705+0000 7fbe10c00640 1 --2- 172.21.15.123:0/3350623800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe0c162ce0 0x7fbe0c140fd0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400cef0 tx=0x7fbe0400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:48.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.705+0000 7fbe08c00640 1 -- 172.21.15.123:0/3350623800 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0401a070 con 0x7fbe0c162ce0 2024-04-24T05:47:48.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.705+0000 7fbe08c00640 1 -- 172.21.15.123:0/3350623800 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0400d860 con 0x7fbe0c162ce0 2024-04-24T05:47:48.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.705+0000 7fbe08c00640 1 -- 172.21.15.123:0/3350623800 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0400dc50 con 0x7fbe0c162ce0 2024-04-24T05:47:48.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.705+0000 7fbe11600640 1 -- 172.21.15.123:0/3350623800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0c09c1e0 con 0x7fbe0c162ce0 2024-04-24T05:47:48.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.705+0000 7fbe11600640 1 -- 172.21.15.123:0/3350623800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe0c09c7a0 con 0x7fbe0c162ce0 2024-04-24T05:47:48.708 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.706+0000 7fbe11600640 1 -- 172.21.15.123:0/3350623800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe0c068a20 con 0x7fbe0c162ce0 2024-04-24T05:47:48.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.707+0000 7fbe08c00640 1 -- 172.21.15.123:0/3350623800 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe0401d070 con 0x7fbe0c162ce0 2024-04-24T05:47:48.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.709+0000 7fbe08c00640 1 --2- 172.21.15.123:0/3350623800 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbde806c8f0 0x7fbde806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:48.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.709+0000 7fbe08c00640 1 -- 172.21.15.123:0/3350623800 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fbe0408c5c0 con 0x7fbe0c162ce0 2024-04-24T05:47:48.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.709+0000 7fbe0be00640 1 --2- 172.21.15.123:0/3350623800 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbde806c8f0 0x7fbde806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:48.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.710+0000 7fbe0be00640 1 --2- 172.21.15.123:0/3350623800 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbde806c8f0 0x7fbde806edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc006fd0 tx=0x7fbdfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:48.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.713+0000 7fbe08c00640 1 -- 172.21.15.123:0/3350623800 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe0405a910 con 0x7fbe0c162ce0 2024-04-24T05:47:48.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.986+0000 7fbe11600640 1 -- 172.21.15.123:0/3350623800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fbe0c15efb0 con 0x7fbe0c162ce0 2024-04-24T05:47:48.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.991+0000 7fbe08c00640 1 -- 172.21.15.123:0/3350623800 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fbe0405a210 con 0x7fbe0c162ce0 2024-04-24T05:47:48.992 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2024-04-24T05:47:48.992 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBZnChmQ4k9BhAAqlK7y/LktbsFTIUhRQCMYg== 2024-04-24T05:47:48.992 INFO:teuthology.orchestra.run.smithi123.stdout: caps mds = "allow" 2024-04-24T05:47:48.993 INFO:teuthology.orchestra.run.smithi123.stdout: caps mon = "allow r" 2024-04-24T05:47:48.993 INFO:teuthology.orchestra.run.smithi123.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-24T05:47:48.993 INFO:teuthology.orchestra.run.smithi123.stderr:updated caps for client.0 2024-04-24T05:47:48.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.993+0000 7fbe11600640 1 -- 172.21.15.123:0/3350623800 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbde806c8f0 msgr2=0x7fbde806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:48.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.993+0000 7fbe11600640 1 --2- 172.21.15.123:0/3350623800 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbde806c8f0 0x7fbde806edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc006fd0 tx=0x7fbdfc008040 comp rx=0 tx=0).stop 2024-04-24T05:47:48.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.993+0000 7fbe11600640 1 -- 172.21.15.123:0/3350623800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe0c162ce0 msgr2=0x7fbe0c140fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:48.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.993+0000 7fbe11600640 1 --2- 172.21.15.123:0/3350623800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe0c162ce0 0x7fbe0c140fd0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400cef0 tx=0x7fbe0400a710 comp rx=0 tx=0).stop 2024-04-24T05:47:48.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.993+0000 7fbe11600640 1 -- 172.21.15.123:0/3350623800 shutdown_connections 2024-04-24T05:47:48.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.993+0000 7fbe11600640 1 --2- 172.21.15.123:0/3350623800 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fbde806c8f0 0x7fbde806edb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.993+0000 7fbe11600640 1 --2- 172.21.15.123:0/3350623800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe0c162ce0 0x7fbe0c140fd0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.993+0000 7fbe11600640 1 --2- 172.21.15.123:0/3350623800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe0c15ea60 0x7fbe0c15dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.993+0000 7fbe11600640 1 --2- 172.21.15.123:0/3350623800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe0c147e70 0x7fbe0c09d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:48.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.993+0000 7fbe11600640 1 -- 172.21.15.123:0/3350623800 >> 172.21.15.123:0/3350623800 conn(0x7fbe0c0b1e40 msgr2=0x7fbe0c0d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:48.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.994+0000 7fbe11600640 1 -- 172.21.15.123:0/3350623800 shutdown_connections 2024-04-24T05:47:48.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:48.994+0000 7fbe11600640 1 -- 172.21.15.123:0/3350623800 wait complete. 2024-04-24T05:47:49.012 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T05:47:49.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.183+0000 7f79fb200640 1 Processor -- start 2024-04-24T05:47:49.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.183+0000 7f79fb200640 1 -- start start 2024-04-24T05:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.184+0000 7f79fb200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f79fc0adee0 0x7f79fc0acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.184+0000 7f79fb200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79fc0ae2e0 0x7f79fc0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.184+0000 7f79fb200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79fc160120 con 0x7f79fc15c5d0 2024-04-24T05:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.184+0000 7f79fb200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f79fc160470 con 0x7f79fc0adee0 2024-04-24T05:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.184+0000 7f79fb200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc161010 con 0x7f79fc0ae2e0 2024-04-24T05:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.185+0000 7f79f3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f79fc0adee0 0x7f79fc0acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.185+0000 7f79f3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f79fc0adee0 0x7f79fc0acf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46364/0 (socket says 172.21.15.123:46364) 2024-04-24T05:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.185+0000 7f79f3e00640 1 -- 172.21.15.123:0/7635553 learned_addr learned my addr 172.21.15.123:0/7635553 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.185+0000 7f79fa800640 1 --2- 172.21.15.123:0/7635553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79fc0ae2e0 0x7f79fc0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.185+0000 7f79f3e00640 1 -- 172.21.15.123:0/7635553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79fc0ae2e0 msgr2=0x7f79fc0ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.185+0000 7f79f3e00640 1 --2- 172.21.15.123:0/7635553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79fc0ae2e0 0x7f79fc0ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:49.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.185+0000 7f79f3e00640 1 -- 172.21.15.123:0/7635553 >> v1:172.21.15.123:6789/0 conn(0x7f79fc15c5d0 legacy=0x7f79fc160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:49.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.185+0000 7f79f3e00640 1 -- 172.21.15.123:0/7635553 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc06dd80 con 0x7f79fc0adee0 2024-04-24T05:47:49.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.185+0000 7f79f9400640 1 -- 172.21.15.123:0/7635553 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 128847221 0 0) 0x7f79fc160120 con 0x7f79fc15c5d0 2024-04-24T05:47:49.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.185+0000 7f79fa800640 1 --2- 172.21.15.123:0/7635553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79fc0ae2e0 0x7f79fc0ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.186+0000 7f79f3e00640 1 --2- 172.21.15.123:0/7635553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc0adee0 0x7f79fc0acf40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009870 tx=0x7f79e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39329b8aee39d526 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.186+0000 7f79f9400640 1 -- 172.21.15.123:0/7635553 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4037070 con 0x7f79fc0adee0 2024-04-24T05:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.187+0000 7f79f9400640 1 -- 172.21.15.123:0/7635553 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e4032440 con 0x7f79fc0adee0 2024-04-24T05:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.187+0000 7f79f9400640 1 -- 172.21.15.123:0/7635553 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4032810 con 0x7f79fc0adee0 2024-04-24T05:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.187+0000 7f79fb200640 1 -- 172.21.15.123:0/7635553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc0adee0 msgr2=0x7f79fc0acf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.187+0000 7f79fb200640 1 --2- 172.21.15.123:0/7635553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc0adee0 0x7f79fc0acf40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009870 tx=0x7f79e402a270 comp rx=0 tx=0).stop 2024-04-24T05:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.187+0000 7f79fb200640 1 -- 172.21.15.123:0/7635553 shutdown_connections 2024-04-24T05:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.187+0000 7f79fb200640 1 --2- 172.21.15.123:0/7635553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79fc0ae2e0 0x7f79fc0ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.187+0000 7f79fb200640 1 --2- 172.21.15.123:0/7635553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc0adee0 0x7f79fc0acf40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.187+0000 7f79fb200640 1 -- 172.21.15.123:0/7635553 >> 172.21.15.123:0/7635553 conn(0x7f79fc0d0070 msgr2=0x7f79fc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.187+0000 7f79fb200640 1 -- 172.21.15.123:0/7635553 shutdown_connections 2024-04-24T05:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.187+0000 7f79fb200640 1 -- 172.21.15.123:0/7635553 wait complete. 2024-04-24T05:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.189+0000 7f79fb200640 1 Processor -- start 2024-04-24T05:47:49.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.189+0000 7f79fb200640 1 -- start start 2024-04-24T05:47:49.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79fb200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79fc0adee0 0x7f79fc15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79fb200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc0ae2e0 0x7f79fc15f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79fb200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79fc15c5d0 0x7f79fc09c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79fb200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc0d0b60 con 0x7f79fc0ae2e0 2024-04-24T05:47:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79fb200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc15c9b0 con 0x7f79fc15c5d0 2024-04-24T05:47:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79fb200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc0d0ce0 con 0x7f79fc0adee0 2024-04-24T05:47:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79f3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc0ae2e0 0x7f79fc15f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79f3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc0ae2e0 0x7f79fc15f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46374/0 (socket says 172.21.15.123:46374) 2024-04-24T05:47:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79f3e00640 1 -- 172.21.15.123:0/1185303859 learned_addr learned my addr 172.21.15.123:0/1185303859 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79f9e00640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79fc0adee0 0x7f79fc15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79f3e00640 1 -- 172.21.15.123:0/1185303859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79fc0adee0 msgr2=0x7f79fc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79fa800640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79fc15c5d0 0x7f79fc09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79f3e00640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79fc0adee0 0x7f79fc15a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79f3e00640 1 -- 172.21.15.123:0/1185303859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79fc15c5d0 msgr2=0x7f79fc09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79f3e00640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79fc15c5d0 0x7f79fc09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79f3e00640 1 -- 172.21.15.123:0/1185303859 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc15d820 con 0x7f79fc0ae2e0 2024-04-24T05:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.190+0000 7f79f9e00640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79fc0adee0 0x7f79fc15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.191+0000 7f79f3e00640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc0ae2e0 0x7f79fc15f470 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f79e40097c0 tx=0x7f79e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.191+0000 7f79f2000640 1 -- 172.21.15.123:0/1185303859 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4037040 con 0x7f79fc0ae2e0 2024-04-24T05:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.191+0000 7f79f2000640 1 -- 172.21.15.123:0/1185303859 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e4004220 con 0x7f79fc0ae2e0 2024-04-24T05:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.191+0000 7f79f2000640 1 -- 172.21.15.123:0/1185303859 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e403c670 con 0x7f79fc0ae2e0 2024-04-24T05:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.191+0000 7f79fb200640 1 -- 172.21.15.123:0/1185303859 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79fc15db00 con 0x7f79fc0ae2e0 2024-04-24T05:47:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.191+0000 7f79fb200640 1 -- 172.21.15.123:0/1185303859 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79fc15e0c0 con 0x7f79fc0ae2e0 2024-04-24T05:47:49.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.192+0000 7f79fb200640 1 -- 172.21.15.123:0/1185303859 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79fc0cf730 con 0x7f79fc0ae2e0 2024-04-24T05:47:49.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.193+0000 7f79f2000640 1 -- 172.21.15.123:0/1185303859 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f79e403a070 con 0x7f79fc0ae2e0 2024-04-24T05:47:49.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.194+0000 7f79f2000640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f79d806c8f0 0x7f79d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:49.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.194+0000 7f79f2000640 1 -- 172.21.15.123:0/1185303859 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f79e40ac590 con 0x7f79fc0ae2e0 2024-04-24T05:47:49.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.194+0000 7f79f9e00640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f79d806c8f0 0x7f79d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:49.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.195+0000 7f79f9e00640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f79d806c8f0 0x7f79d806edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f79ec000f60 tx=0x7f79ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:49.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.198+0000 7f79f2000640 1 -- 172.21.15.123:0/1185303859 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79fc0cf730 con 0x7f79fc0ae2e0 2024-04-24T05:47:49.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.419+0000 7f79fb200640 1 -- 172.21.15.123:0/1185303859 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79fc075b70 con 0x7f79fc0ae2e0 2024-04-24T05:47:49.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.420+0000 7f79f2000640 1 -- 172.21.15.123:0/1185303859 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f79e404a030 con 0x7f79fc0ae2e0 2024-04-24T05:47:49.422 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:49.422 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2481558889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2481558889},{"type":"v1","addr":"172.21.15.155:6833","nonce":2481558889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4915,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3305281906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3305281906},{"type":"v1","addr":"172.21.15.155:6835","nonce":3305281906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/1261416511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1261416511},{"type":"v1","addr":"172.21.15.123:6836","nonce":1261416511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T05:47:42.953535+0000","modified":"2024-04-24T05:47:46.031567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5493":{"gid":5493,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6837/3655308480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":3655308480},{"type":"v1","addr":"172.21.15.123:6837","nonce":3655308480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5493,"qdb_cluster":[5493]},"id":2}]} 2024-04-24T05:47:49.422 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 18 2024-04-24T05:47:49.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.422+0000 7f79fb200640 1 -- 172.21.15.123:0/1185303859 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f79d806c8f0 msgr2=0x7f79d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:49.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.422+0000 7f79fb200640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f79d806c8f0 0x7f79d806edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f79ec000f60 tx=0x7f79ec009040 comp rx=0 tx=0).stop 2024-04-24T05:47:49.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.422+0000 7f79fb200640 1 -- 172.21.15.123:0/1185303859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc0ae2e0 msgr2=0x7f79fc15f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:49.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.422+0000 7f79fb200640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc0ae2e0 0x7f79fc15f470 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f79e40097c0 tx=0x7f79e40046c0 comp rx=0 tx=0).stop 2024-04-24T05:47:49.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.422+0000 7f79fb200640 1 -- 172.21.15.123:0/1185303859 shutdown_connections 2024-04-24T05:47:49.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.422+0000 7f79fb200640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f79d806c8f0 0x7f79d806edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:49.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.422+0000 7f79fb200640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79fc15c5d0 0x7f79fc09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:49.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.422+0000 7f79fb200640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc0ae2e0 0x7f79fc15f470 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:49.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.422+0000 7f79fb200640 1 --2- 172.21.15.123:0/1185303859 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79fc0adee0 0x7f79fc15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:49.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.422+0000 7f79fb200640 1 -- 172.21.15.123:0/1185303859 >> 172.21.15.123:0/1185303859 conn(0x7f79fc0d0070 msgr2=0x7f79fc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:49.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.423+0000 7f79fb200640 1 -- 172.21.15.123:0/1185303859 shutdown_connections 2024-04-24T05:47:49.425 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:49.423+0000 7f79fb200640 1 -- 172.21.15.123:0/1185303859 wait complete. 2024-04-24T05:47:49.844 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T05:47:42.953535+0000', 'modified': '2024-04-24T05:47:46.031567+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5493': {'gid': 5493, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.123:6837/3655308480', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6835', 'nonce': 3655308480}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 3655308480}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5493, 'qdb_cluster': [5493]} 2024-04-24T05:47:49.844 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T05:47:49.844 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T05:47:50.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.053+0000 7fa839200640 1 Processor -- start 2024-04-24T05:47:50.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.054+0000 7fa839200640 1 -- start start 2024-04-24T05:47:50.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.055+0000 7fa839200640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa834071e40 0x7fa834072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.055+0000 7fa839200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa83406ba00 0x7fa83415ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.055+0000 7fa839200640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa834072760 con 0x7fa83406be60 2024-04-24T05:47:50.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.055+0000 7fa839200640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa83406edd0 con 0x7fa834071e40 2024-04-24T05:47:50.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.055+0000 7fa839200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa83406c3e0 con 0x7fa83406ba00 2024-04-24T05:47:50.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.056+0000 7fa831400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa83406ba00 0x7fa83415ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.056+0000 7fa831e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa834071e40 0x7fa834072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.056+0000 7fa831400640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa83406ba00 0x7fa83415ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.123:54324/0 (socket says 172.21.15.123:54324) 2024-04-24T05:47:50.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.056+0000 7fa831e00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa834071e40 0x7fa834072220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.123:43472/0 (socket says 172.21.15.123:43472) 2024-04-24T05:47:50.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.056+0000 7fa831400640 1 -- 172.21.15.123:0/3289016581 learned_addr learned my addr 172.21.15.123:0/3289016581 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:50.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.057+0000 7fa831400640 1 -- 172.21.15.123:0/3289016581 >> v2:172.21.15.155:3300/0 conn(0x7fa834071e40 msgr2=0x7fa834072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:50.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.057+0000 7fa831400640 1 --2- 172.21.15.123:0/3289016581 >> v2:172.21.15.155:3300/0 conn(0x7fa834071e40 0x7fa834072220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.057+0000 7fa831400640 1 -- 172.21.15.123:0/3289016581 >> v1:172.21.15.155:6789/0 conn(0x7fa83406be60 legacy=0x7fa83415f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.057+0000 7fa831400640 1 -- 172.21.15.123:0/3289016581 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83406c6c0 con 0x7fa83406ba00 2024-04-24T05:47:50.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.057+0000 7fa831e00640 1 --2- 172.21.15.123:0/3289016581 >> v2:172.21.15.155:3300/0 conn(0x7fa834071e40 0x7fa834072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T05:47:50.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.058+0000 7fa831400640 1 --2- 172.21.15.123:0/3289016581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa83406ba00 0x7fa83415ed20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009e50 tx=0x7fa81c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7318a383e0cd8e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:50.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.059+0000 7fa830a00640 1 -- 172.21.15.123:0/3289016581 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c036070 con 0x7fa83406ba00 2024-04-24T05:47:50.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.059+0000 7fa830a00640 1 -- 172.21.15.123:0/3289016581 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c031600 con 0x7fa83406ba00 2024-04-24T05:47:50.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.059+0000 7fa830a00640 1 -- 172.21.15.123:0/3289016581 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c0319d0 con 0x7fa83406ba00 2024-04-24T05:47:50.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.059+0000 7fa839200640 1 -- 172.21.15.123:0/3289016581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa83406ba00 msgr2=0x7fa83415ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.059+0000 7fa839200640 1 --2- 172.21.15.123:0/3289016581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa83406ba00 0x7fa83415ed20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009e50 tx=0x7fa81c02a680 comp rx=0 tx=0).stop 2024-04-24T05:47:50.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.060+0000 7fa839200640 1 -- 172.21.15.123:0/3289016581 shutdown_connections 2024-04-24T05:47:50.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.060+0000 7fa839200640 1 --2- 172.21.15.123:0/3289016581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa83406ba00 0x7fa83415ed20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.060+0000 7fa839200640 1 --2- 172.21.15.123:0/3289016581 >> v2:172.21.15.155:3300/0 conn(0x7fa834071e40 0x7fa834072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.060+0000 7fa839200640 1 -- 172.21.15.123:0/3289016581 >> 172.21.15.123:0/3289016581 conn(0x7fa8340b1e70 msgr2=0x7fa8340b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:50.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.060+0000 7fa839200640 1 -- 172.21.15.123:0/3289016581 shutdown_connections 2024-04-24T05:47:50.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.060+0000 7fa839200640 1 -- 172.21.15.123:0/3289016581 wait complete. 2024-04-24T05:47:50.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.061+0000 7fa839200640 1 Processor -- start 2024-04-24T05:47:50.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.062+0000 7fa839200640 1 -- start start 2024-04-24T05:47:50.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.062+0000 7fa839200640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83406ba00 0x7fa834149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.062+0000 7fa839200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406be60 0x7fa8340cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.062+0000 7fa839200640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa834071e40 0x7fa8340cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.062+0000 7fa839200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa83406cf60 con 0x7fa83406be60 2024-04-24T05:47:50.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.062+0000 7fa839200640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa83406cc80 con 0x7fa83406ba00 2024-04-24T05:47:50.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.062+0000 7fa839200640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa8341623f0 con 0x7fa834071e40 2024-04-24T05:47:50.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.063+0000 7fa831400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406be60 0x7fa8340cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.063+0000 7fa831400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406be60 0x7fa8340cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46376/0 (socket says 172.21.15.123:46376) 2024-04-24T05:47:50.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.063+0000 7fa831400640 1 -- 172.21.15.123:0/171053636 learned_addr learned my addr 172.21.15.123:0/171053636 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:50.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.063+0000 7fa831e00640 1 --2- 172.21.15.123:0/171053636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83406ba00 0x7fa834149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.063+0000 7fa832800640 1 --2- 172.21.15.123:0/171053636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa834071e40 0x7fa8340cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.063+0000 7fa831400640 1 -- 172.21.15.123:0/171053636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa834071e40 msgr2=0x7fa8340cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.063+0000 7fa831400640 1 --2- 172.21.15.123:0/171053636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa834071e40 0x7fa8340cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.063+0000 7fa831400640 1 -- 172.21.15.123:0/171053636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83406ba00 msgr2=0x7fa834149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.063+0000 7fa831400640 1 --2- 172.21.15.123:0/171053636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83406ba00 0x7fa834149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.063+0000 7fa831400640 1 -- 172.21.15.123:0/171053636 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8340cb480 con 0x7fa83406be60 2024-04-24T05:47:50.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.064+0000 7fa831400640 1 --2- 172.21.15.123:0/171053636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406be60 0x7fa8340cef90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa81c034040 tx=0x7fa81c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:50.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.064+0000 7fa82a000640 1 -- 172.21.15.123:0/171053636 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c036050 con 0x7fa83406be60 2024-04-24T05:47:50.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.064+0000 7fa82a000640 1 -- 172.21.15.123:0/171053636 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c042070 con 0x7fa83406be60 2024-04-24T05:47:50.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.064+0000 7fa82a000640 1 -- 172.21.15.123:0/171053636 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c009510 con 0x7fa83406be60 2024-04-24T05:47:50.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.064+0000 7fa839200640 1 -- 172.21.15.123:0/171053636 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8340cb760 con 0x7fa83406be60 2024-04-24T05:47:50.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.064+0000 7fa839200640 1 -- 172.21.15.123:0/171053636 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8340cbd20 con 0x7fa83406be60 2024-04-24T05:47:50.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.067+0000 7fa82a000640 1 -- 172.21.15.123:0/171053636 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa81c007c70 con 0x7fa83406be60 2024-04-24T05:47:50.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.068+0000 7fa82a000640 1 --2- 172.21.15.123:0/171053636 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa81006c8f0 0x7fa81006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.068+0000 7fa82a000640 1 -- 172.21.15.123:0/171053636 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa81c0acad0 con 0x7fa83406be60 2024-04-24T05:47:50.069 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.068+0000 7fa831e00640 1 --2- 172.21.15.123:0/171053636 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa81006c8f0 0x7fa81006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.068+0000 7fa83bf80740 1 Processor -- start 2024-04-24T05:47:50.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.069+0000 7fa83bf80740 1 -- start start 2024-04-24T05:47:50.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.069+0000 7fa831e00640 1 --2- 172.21.15.123:0/171053636 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa81006c8f0 0x7fa81006edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa824009560 tx=0x7fa824008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:50.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.069+0000 7fa83bf80740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce097b0 0x55b80cdbc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.069+0000 7fa83bf80740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 0x55b80cdebc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.069+0000 7fa83bf80740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80cdec190 0x55b80cdc0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.069+0000 7fa83bf80740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b80cd26b10 con 0x55b80ce097b0 2024-04-24T05:47:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.069+0000 7fa83bf80740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b80cccd2b0 con 0x55b80cdec8d0 2024-04-24T05:47:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.069+0000 7fa83bf80740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55b80cd22ad0 con 0x55b80cdec190 2024-04-24T05:47:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.070+0000 7fa831e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce097b0 0x55b80cdbc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.070+0000 7fa831e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce097b0 0x55b80cdbc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46380/0 (socket says 172.21.15.123:46380) 2024-04-24T05:47:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.070+0000 7fa831e00640 1 -- 172.21.15.123:0/601246887 learned_addr learned my addr 172.21.15.123:0/601246887 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.070+0000 7fa831e00640 1 -- 172.21.15.123:0/601246887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80cdec190 msgr2=0x55b80cdc0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.070+0000 7fa832800640 1 --2- 172.21.15.123:0/601246887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80cdec190 0x55b80cdc0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.070+0000 7fa831400640 1 --2- 172.21.15.123:0/601246887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 0x55b80cdebc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.073 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.070+0000 7fa831e00640 1 --2- 172.21.15.123:0/601246887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80cdec190 0x55b80cdc0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.070+0000 7fa831e00640 1 -- 172.21.15.123:0/601246887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 msgr2=0x55b80cdebc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:50.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.070+0000 7fa831e00640 1 --2- 172.21.15.123:0/601246887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 0x55b80cdebc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.070+0000 7fa831e00640 1 -- 172.21.15.123:0/601246887 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b80ca392d0 con 0x55b80ce097b0 2024-04-24T05:47:50.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.071+0000 7fa831e00640 1 --2- 172.21.15.123:0/601246887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce097b0 0x55b80cdbc140 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa824011df0 tx=0x7fa824014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cbfcc56f52e942 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:50.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.071+0000 7fa822a00640 1 -- 172.21.15.123:0/601246887 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824011e20 con 0x55b80ce097b0 2024-04-24T05:47:50.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.071+0000 7fa822a00640 1 -- 172.21.15.123:0/601246887 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa824026070 con 0x55b80ce097b0 2024-04-24T05:47:50.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.071+0000 7fa822a00640 1 -- 172.21.15.123:0/601246887 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824021640 con 0x55b80ce097b0 2024-04-24T05:47:50.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.072+0000 7fa83bf80740 1 -- 172.21.15.123:0/601246887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce097b0 msgr2=0x55b80cdbc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.072+0000 7fa83bf80740 1 --2- 172.21.15.123:0/601246887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce097b0 0x55b80cdbc140 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa824011df0 tx=0x7fa824014000 comp rx=0 tx=0).stop 2024-04-24T05:47:50.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.072+0000 7fa83bf80740 1 -- 172.21.15.123:0/601246887 shutdown_connections 2024-04-24T05:47:50.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.072+0000 7fa83bf80740 1 --2- 172.21.15.123:0/601246887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80cdec190 0x55b80cdc0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.072+0000 7fa83bf80740 1 --2- 172.21.15.123:0/601246887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 0x55b80cdebc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.072+0000 7fa83bf80740 1 --2- 172.21.15.123:0/601246887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce097b0 0x55b80cdbc140 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.072+0000 7fa83bf80740 1 -- 172.21.15.123:0/601246887 >> 172.21.15.123:0/601246887 conn(0x55b80cdbe3b0 msgr2=0x55b80ce06c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:50.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.072+0000 7fa83bf80740 1 -- 172.21.15.123:0/601246887 shutdown_connections 2024-04-24T05:47:50.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.072+0000 7fa83bf80740 1 -- 172.21.15.123:0/601246887 wait complete. 2024-04-24T05:47:50.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.072+0000 7fa83bf80740 1 -- start start 2024-04-24T05:47:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.073+0000 7fa83bf80740 1 Processor -- start 2024-04-24T05:47:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa83bf80740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 0x55b80cdff520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa83bf80740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80ce097b0 0x55b80ce179a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa83bf80740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce084e0 0x55b80ce08960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa83bf80740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b80cdc0560 con 0x55b80ce084e0 2024-04-24T05:47:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa83bf80740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b80cce0590 con 0x55b80cdec8d0 2024-04-24T05:47:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa83bf80740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55b80cdbf950 con 0x55b80ce097b0 2024-04-24T05:47:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa83bf80740 10 client.0 _ll_register_callbacks cb 0x55b80ccc33a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T05:47:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa832800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce084e0 0x55b80ce08960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa832800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce084e0 0x55b80ce08960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46382/0 (socket says 172.21.15.123:46382) 2024-04-24T05:47:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa832800640 1 -- 172.21.15.123:0/460297055 learned_addr learned my addr 172.21.15.123:0/460297055 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa831e00640 1 --2- 172.21.15.123:0/460297055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 0x55b80cdff520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa832800640 1 -- 172.21.15.123:0/460297055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80ce097b0 msgr2=0x55b80ce179a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:50.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa832800640 1 --2- 172.21.15.123:0/460297055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80ce097b0 0x55b80ce179a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa832800640 1 -- 172.21.15.123:0/460297055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 msgr2=0x55b80cdff520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa832800640 1 --2- 172.21.15.123:0/460297055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 0x55b80cdff520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.074+0000 7fa832800640 1 -- 172.21.15.123:0/460297055 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b80cb036b0 con 0x55b80ce084e0 2024-04-24T05:47:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.075+0000 7fa832800640 1 --2- 172.21.15.123:0/460297055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce084e0 0x55b80ce08960 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa82c001d90 tx=0x7fa82c00f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38424a372f24978d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.075+0000 7fa81b400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-24T05:47:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.075+0000 7fa81b400640 1 -- 172.21.15.123:0/460297055 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0050d0 con 0x55b80ce084e0 2024-04-24T05:47:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.075+0000 7fa81b400640 1 -- 172.21.15.123:0/460297055 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa82c005270 con 0x55b80ce084e0 2024-04-24T05:47:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.075+0000 7fa83bf80740 1 -- 172.21.15.123:0/460297055 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa824026070 con 0x55b80ce084e0 2024-04-24T05:47:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.075+0000 7fa81b400640 1 -- 172.21.15.123:0/460297055 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fa82c020050 con 0x55b80ce084e0 2024-04-24T05:47:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.075+0000 7fa83bf80740 10 client.5673 fetch_fsmap learned FSMap version 18 2024-04-24T05:47:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.075+0000 7fa83bf80740 1 -- 172.21.15.123:0/460297055 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55b80ce17ee0 con 0x55b80ce084e0 2024-04-24T05:47:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.075+0000 7fa81b400640 1 -- 172.21.15.123:0/460297055 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa824026070 con 0x55b80ce084e0 2024-04-24T05:47:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.075+0000 7fa81b400640 1 -- 172.21.15.123:0/460297055 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fa82c023020 con 0x55b80ce084e0 2024-04-24T05:47:50.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.076+0000 7fa83bf80740 10 client.5673 fetch_fsmap finished waiting for FSMap version 18 2024-04-24T05:47:50.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.076+0000 7fa83bf80740 10 client.5673 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T05:47:50.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.076+0000 7fa83bf80740 20 client.5673 populate_metadata read hostname 'smithi123' 2024-04-24T05:47:50.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.076+0000 7fa83bf80740 1 --2- 172.21.15.123:0/460297055 >> [v2:172.21.15.123:6835/3655308480,v1:172.21.15.123:6837/3655308480] conn(0x55b80cded5f0 0x55b80ce25860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.076+0000 7fa83bf80740 4 client.5673 mds_command: new command op to 5493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T05:47:50.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.076+0000 7fa83bf80740 1 -- 172.21.15.123:0/460297055 --> [v2:172.21.15.123:6835/3655308480,v1:172.21.15.123:6837/3655308480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b80cd34530 con 0x55b80cded5f0 2024-04-24T05:47:50.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.076+0000 7fa831e00640 1 --2- 172.21.15.123:0/460297055 >> [v2:172.21.15.123:6835/3655308480,v1:172.21.15.123:6837/3655308480] conn(0x55b80cded5f0 0x55b80ce25860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.077+0000 7fa831e00640 1 --2- 172.21.15.123:0/460297055 >> [v2:172.21.15.123:6835/3655308480,v1:172.21.15.123:6837/3655308480] conn(0x55b80cded5f0 0x55b80ce25860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5290581dbafe498 server_cookie=9cf58774034bf013 in_seq=0 out_seq=0 2024-04-24T05:47:50.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.077+0000 7fa81b400640 10 client.5673 ms_handle_connect on v2:172.21.15.123:6835/3655308480 2024-04-24T05:47:50.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.077+0000 7fa81b400640 1 -- 172.21.15.123:0/460297055 <== mds.0 v2:172.21.15.123:6835/3655308480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b80cd34530 con 0x55b80cded5f0 2024-04-24T05:47:50.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.077+0000 7fa81b400640 10 client.5673 handle_command_reply: tid=0 2024-04-24T05:47:50.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.077+0000 7fa83bf80740 1 client.5673 shutdown 2024-04-24T05:47:50.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.078+0000 7fa83bf80740 1 -- 172.21.15.123:0/460297055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce084e0 msgr2=0x55b80ce08960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.078+0000 7fa83bf80740 1 --2- 172.21.15.123:0/460297055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce084e0 0x55b80ce08960 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa82c001d90 tx=0x7fa82c00f5a0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.078+0000 7fa83bf80740 1 -- 172.21.15.123:0/460297055 shutdown_connections 2024-04-24T05:47:50.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.078+0000 7fa83bf80740 1 --2- 172.21.15.123:0/460297055 >> [v2:172.21.15.123:6835/3655308480,v1:172.21.15.123:6837/3655308480] conn(0x55b80cded5f0 0x55b80ce25860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.078+0000 7fa83bf80740 1 --2- 172.21.15.123:0/460297055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce084e0 0x55b80ce08960 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.078+0000 7fa83bf80740 1 --2- 172.21.15.123:0/460297055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80ce097b0 0x55b80ce179a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.078+0000 7fa83bf80740 1 --2- 172.21.15.123:0/460297055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 0x55b80cdff520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.078+0000 7fa81b400640 0 client.5673 ms_handle_reset on v2:172.21.15.123:6835/3655308480 2024-04-24T05:47:50.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.078+0000 7fa83bf80740 1 -- 172.21.15.123:0/460297055 >> 172.21.15.123:0/460297055 conn(0x55b80cdec190 msgr2=0x55b80ce06c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:50.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.078+0000 7fa83bf80740 1 -- 172.21.15.123:0/460297055 shutdown_connections 2024-04-24T05:47:50.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.078+0000 7fa83bf80740 1 -- 172.21.15.123:0/460297055 wait complete. 2024-04-24T05:47:50.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.078+0000 7fa83bf80740 20 client.5673 trim_cache size 0 max 16384 2024-04-24T05:47:50.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.098+0000 7fa83bf80740 1 Processor -- start 2024-04-24T05:47:50.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.098+0000 7fa83bf80740 1 -- start start 2024-04-24T05:47:50.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.098+0000 7fa83bf80740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 0x55b80cdd90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.098+0000 7fa83bf80740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce084e0 0x55b80ce28890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.098+0000 7fa83bf80740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80ce097b0 0x55b80ce28dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.098+0000 7fa83bf80740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa8342014d0 con 0x55b80ce084e0 2024-04-24T05:47:50.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.098+0000 7fa83bf80740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa8342005a0 con 0x55b80cdec8d0 2024-04-24T05:47:50.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.098+0000 7fa83bf80740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa8341559b0 con 0x55b80ce097b0 2024-04-24T05:47:50.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.099+0000 7fa831400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce084e0 0x55b80ce28890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.099+0000 7fa831400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce084e0 0x55b80ce28890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46386/0 (socket says 172.21.15.123:46386) 2024-04-24T05:47:50.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.099+0000 7fa831400640 1 -- 172.21.15.123:0/3518670395 learned_addr learned my addr 172.21.15.123:0/3518670395 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:50.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.099+0000 7fa832800640 1 --2- 172.21.15.123:0/3518670395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80ce097b0 0x55b80ce28dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.099+0000 7fa831e00640 1 --2- 172.21.15.123:0/3518670395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 0x55b80cdd90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.099+0000 7fa831400640 1 -- 172.21.15.123:0/3518670395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80ce097b0 msgr2=0x55b80ce28dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:50.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.099+0000 7fa831400640 1 --2- 172.21.15.123:0/3518670395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80ce097b0 0x55b80ce28dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.099+0000 7fa831400640 1 -- 172.21.15.123:0/3518670395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 msgr2=0x55b80cdd90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:50.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.099+0000 7fa831400640 1 --2- 172.21.15.123:0/3518670395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 0x55b80cdd90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.099+0000 7fa831400640 1 -- 172.21.15.123:0/3518670395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b80ce005f0 con 0x55b80ce084e0 2024-04-24T05:47:50.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.100+0000 7fa831400640 1 --2- 172.21.15.123:0/3518670395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce084e0 0x55b80ce28890 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa81c07a5f0 tx=0x7fa81c07a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5336d57b1e7e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:50.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.100+0000 7fa822000640 1 -- 172.21.15.123:0/3518670395 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c092020 con 0x55b80ce084e0 2024-04-24T05:47:50.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.100+0000 7fa822000640 1 -- 172.21.15.123:0/3518670395 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c037040 con 0x55b80ce084e0 2024-04-24T05:47:50.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.100+0000 7fa822000640 1 -- 172.21.15.123:0/3518670395 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c031600 con 0x55b80ce084e0 2024-04-24T05:47:50.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.101+0000 7fa83bf80740 1 -- 172.21.15.123:0/3518670395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce084e0 msgr2=0x55b80ce28890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.101+0000 7fa83bf80740 1 --2- 172.21.15.123:0/3518670395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce084e0 0x55b80ce28890 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa81c07a5f0 tx=0x7fa81c07a430 comp rx=0 tx=0).stop 2024-04-24T05:47:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.101+0000 7fa83bf80740 1 -- 172.21.15.123:0/3518670395 shutdown_connections 2024-04-24T05:47:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.101+0000 7fa83bf80740 1 --2- 172.21.15.123:0/3518670395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80ce097b0 0x55b80ce28dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.101+0000 7fa83bf80740 1 --2- 172.21.15.123:0/3518670395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80ce084e0 0x55b80ce28890 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.101+0000 7fa83bf80740 1 --2- 172.21.15.123:0/3518670395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80cdec8d0 0x55b80cdd90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.101+0000 7fa83bf80740 1 -- 172.21.15.123:0/3518670395 >> 172.21.15.123:0/3518670395 conn(0x55b80cdec190 msgr2=0x55b80ce07d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.101+0000 7fa83bf80740 1 -- 172.21.15.123:0/3518670395 shutdown_connections 2024-04-24T05:47:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.101+0000 7fa83bf80740 1 -- 172.21.15.123:0/3518670395 wait complete. 2024-04-24T05:47:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.102+0000 7fa83bf80740 1 -- start start 2024-04-24T05:47:50.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.102+0000 7fa83bf80740 1 Processor -- start 2024-04-24T05:47:50.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa83bf80740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80cdec8d0 0x55b80ce7fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa83bf80740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80ce084e0 0x55b80cdb77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa83bf80740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80ce097b0 0x55b80ce8c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa83bf80740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b80cdef420 con 0x55b80cdec8d0 2024-04-24T05:47:50.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa83bf80740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa834156000 con 0x55b80ce097b0 2024-04-24T05:47:50.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa83bf80740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55b80cdfe6e0 con 0x55b80ce084e0 2024-04-24T05:47:50.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa83bf80740 10 client.0 _ll_register_callbacks cb 0x7fa82c001970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T05:47:50.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa831e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80cdec8d0 0x55b80ce7fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa831e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80cdec8d0 0x55b80ce7fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46396/0 (socket says 172.21.15.123:46396) 2024-04-24T05:47:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa831e00640 1 -- 172.21.15.123:0/526743093 learned_addr learned my addr 172.21.15.123:0/526743093 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa831e00640 1 -- 172.21.15.123:0/526743093 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80ce084e0 msgr2=0x55b80cdb77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa831e00640 1 --2- 172.21.15.123:0/526743093 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80ce084e0 0x55b80cdb77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa831e00640 1 -- 172.21.15.123:0/526743093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80ce097b0 msgr2=0x55b80ce8c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa831e00640 1 --2- 172.21.15.123:0/526743093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80ce097b0 0x55b80ce8c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.103+0000 7fa831e00640 1 -- 172.21.15.123:0/526743093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b80cdb7d30 con 0x55b80cdec8d0 2024-04-24T05:47:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.104+0000 7fa831e00640 1 --2- 172.21.15.123:0/526743093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80cdec8d0 0x55b80ce7fa60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa82400a010 tx=0x7fa824004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678f1973a965285b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.104+0000 7fa81b400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-04-24T05:47:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.104+0000 7fa81b400640 1 -- 172.21.15.123:0/526743093 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824011700 con 0x55b80cdec8d0 2024-04-24T05:47:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.104+0000 7fa81b400640 1 -- 172.21.15.123:0/526743093 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa824034bc0 con 0x55b80cdec8d0 2024-04-24T05:47:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.104+0000 7fa83bf80740 1 -- 172.21.15.123:0/526743093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa81c037040 con 0x55b80cdec8d0 2024-04-24T05:47:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.104+0000 7fa81b400640 1 -- 172.21.15.123:0/526743093 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fa824042050 con 0x55b80cdec8d0 2024-04-24T05:47:50.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.104+0000 7fa83bf80740 10 client.5679 fetch_fsmap learned FSMap version 18 2024-04-24T05:47:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.104+0000 7fa83bf80740 1 -- 172.21.15.123:0/526743093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55b80cde7b40 con 0x55b80cdec8d0 2024-04-24T05:47:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.105+0000 7fa81b400640 1 -- 172.21.15.123:0/526743093 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c037040 con 0x55b80cdec8d0 2024-04-24T05:47:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.105+0000 7fa81b400640 1 -- 172.21.15.123:0/526743093 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7fa824045020 con 0x55b80cdec8d0 2024-04-24T05:47:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.105+0000 7fa83bf80740 10 client.5679 fetch_fsmap finished waiting for FSMap version 18 2024-04-24T05:47:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.105+0000 7fa83bf80740 10 client.5679 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T05:47:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.105+0000 7fa83bf80740 20 client.5679 populate_metadata read hostname 'smithi123' 2024-04-24T05:47:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.105+0000 7fa83bf80740 1 --2- 172.21.15.123:0/526743093 >> [v2:172.21.15.123:6835/3655308480,v1:172.21.15.123:6837/3655308480] conn(0x55b80cdfc5e0 0x55b80cdfc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.105+0000 7fa83bf80740 4 client.5679 mds_command: new command op to 5493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T05:47:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.105+0000 7fa83bf80740 1 -- 172.21.15.123:0/526743093 --> [v2:172.21.15.123:6835/3655308480,v1:172.21.15.123:6837/3655308480] -- command(tid 0: {"prefix": "status"}) -- 0x55b80ce007d0 con 0x55b80cdfc5e0 2024-04-24T05:47:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.105+0000 7fa831400640 1 --2- 172.21.15.123:0/526743093 >> [v2:172.21.15.123:6835/3655308480,v1:172.21.15.123:6837/3655308480] conn(0x55b80cdfc5e0 0x55b80cdfc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.105+0000 7fa831400640 1 --2- 172.21.15.123:0/526743093 >> [v2:172.21.15.123:6835/3655308480,v1:172.21.15.123:6837/3655308480] conn(0x55b80cdfc5e0 0x55b80cdfc9c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db703c11dd3204c8 server_cookie=e029fedac99132bb in_seq=0 out_seq=0 2024-04-24T05:47:50.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.105+0000 7fa81b400640 10 client.5679 ms_handle_connect on v2:172.21.15.123:6835/3655308480 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa81b400640 1 -- 172.21.15.123:0/526743093 <== mds.0 v2:172.21.15.123:6835/3655308480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b80ce007d0 con 0x55b80cdfc5e0 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa81b400640 10 client.5679 handle_command_reply: tid=0 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa83bf80740 1 client.5679 shutdown 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa83bf80740 1 -- 172.21.15.123:0/526743093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80cdec8d0 msgr2=0x55b80ce7fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa83bf80740 1 --2- 172.21.15.123:0/526743093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80cdec8d0 0x55b80ce7fa60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa82400a010 tx=0x7fa824004730 comp rx=0 tx=0).stop 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa83bf80740 1 -- 172.21.15.123:0/526743093 shutdown_connections 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa83bf80740 1 --2- 172.21.15.123:0/526743093 >> [v2:172.21.15.123:6835/3655308480,v1:172.21.15.123:6837/3655308480] conn(0x55b80cdfc5e0 0x55b80cdfc9c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa83bf80740 1 --2- 172.21.15.123:0/526743093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b80ce097b0 0x55b80ce8c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa83bf80740 1 --2- 172.21.15.123:0/526743093 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b80ce084e0 0x55b80cdb77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa81b400640 0 client.5679 ms_handle_reset on v2:172.21.15.123:6835/3655308480 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa83bf80740 1 --2- 172.21.15.123:0/526743093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b80cdec8d0 0x55b80ce7fa60 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa83bf80740 1 -- 172.21.15.123:0/526743093 >> 172.21.15.123:0/526743093 conn(0x55b80cdec190 msgr2=0x55b80cdd8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa83bf80740 1 -- 172.21.15.123:0/526743093 shutdown_connections 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa83bf80740 1 -- 172.21.15.123:0/526743093 wait complete. 2024-04-24T05:47:50.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.106+0000 7fa83bf80740 20 client.5679 trim_cache size 0 max 16384 2024-04-24T05:47:50.112 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-24T05:47:50.112 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "6efb8524-c0d1-4076-b7c9-7485084c7ada", 2024-04-24T05:47:50.112 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2024-04-24T05:47:50.112 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 5493, 2024-04-24T05:47:50.112 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-04-24T05:47:50.112 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-04-24T05:47:50.112 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-04-24T05:47:50.112 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 6.5931575750000002, 2024-04-24T05:47:50.113 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 18, 2024-04-24T05:47:50.113 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 39, 2024-04-24T05:47:50.113 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 39, 2024-04-24T05:47:50.113 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 15.482441775 2024-04-24T05:47:50.113 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-24T05:47:50.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.107+0000 7fa839200640 1 -- 172.21.15.123:0/171053636 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa81006c8f0 msgr2=0x7fa81006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.107+0000 7fa839200640 1 --2- 172.21.15.123:0/171053636 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa81006c8f0 0x7fa81006edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa824009560 tx=0x7fa824008040 comp rx=0 tx=0).stop 2024-04-24T05:47:50.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.107+0000 7fa839200640 1 -- 172.21.15.123:0/171053636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406be60 msgr2=0x7fa8340cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.107+0000 7fa839200640 1 --2- 172.21.15.123:0/171053636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406be60 0x7fa8340cef90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa81c034040 tx=0x7fa81c004a40 comp rx=0 tx=0).stop 2024-04-24T05:47:50.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.107+0000 7fa839200640 1 -- 172.21.15.123:0/171053636 shutdown_connections 2024-04-24T05:47:50.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.107+0000 7fa839200640 1 --2- 172.21.15.123:0/171053636 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fa81006c8f0 0x7fa81006edb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.107+0000 7fa839200640 1 --2- 172.21.15.123:0/171053636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa834071e40 0x7fa8340cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.107+0000 7fa839200640 1 --2- 172.21.15.123:0/171053636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406be60 0x7fa8340cef90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.107+0000 7fa839200640 1 --2- 172.21.15.123:0/171053636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa83406ba00 0x7fa834149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.107+0000 7fa839200640 1 -- 172.21.15.123:0/171053636 >> 172.21.15.123:0/171053636 conn(0x7fa8340b1e70 msgr2=0x7fa8340b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:50.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.107+0000 7fa839200640 1 -- 172.21.15.123:0/171053636 shutdown_connections 2024-04-24T05:47:50.115 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.107+0000 7fa839200640 1 -- 172.21.15.123:0/171053636 wait complete. 2024-04-24T05:47:50.125 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-24T05:47:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.321+0000 7feeff000640 1 Processor -- start 2024-04-24T05:47:50.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.321+0000 7feeff000640 1 -- start start 2024-04-24T05:47:50.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feeff000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feef80ad860 0x7feef80af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feeff000640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feef80af780 0x7feef806bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feeff000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef815cdf0 con 0x7feef815a2a0 2024-04-24T05:47:50.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feeff000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7feef815a680 con 0x7feef80ad860 2024-04-24T05:47:50.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feeff000640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7feef815da00 con 0x7feef80af780 2024-04-24T05:47:50.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feef7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feef80ad860 0x7feef80af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feef7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7feef815a2a0 0x7feef80ad150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55140/0 (socket says 172.21.15.123:55140) 2024-04-24T05:47:50.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feef7400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feef80ad860 0x7feef80af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46412/0 (socket says 172.21.15.123:46412) 2024-04-24T05:47:50.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feef7e00640 1 -- 172.21.15.123:0/1531413876 learned_addr learned my addr 172.21.15.123:0/1531413876 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-24T05:47:50.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feefd000640 1 --2- 172.21.15.123:0/1531413876 >> v2:172.21.15.155:3300/0 conn(0x7feef80af780 0x7feef806bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feef7400640 1 -- 172.21.15.123:0/1531413876 >> v2:172.21.15.155:3300/0 conn(0x7feef80af780 msgr2=0x7feef806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:47:50.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feef7400640 1 --2- 172.21.15.123:0/1531413876 >> v2:172.21.15.155:3300/0 conn(0x7feef80af780 0x7feef806bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feef7400640 1 -- 172.21.15.123:0/1531413876 >> v1:172.21.15.123:6789/0 conn(0x7feef815a2a0 legacy=0x7feef80ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feef7400640 1 -- 172.21.15.123:0/1531413876 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef815d0d0 con 0x7feef80ad860 2024-04-24T05:47:50.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feefd000640 1 --2- 172.21.15.123:0/1531413876 >> v2:172.21.15.155:3300/0 conn(0x7feef80af780 0x7feef806bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:47:50.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.323+0000 7feef6a00640 1 -- 172.21.15.123:0/1531413876 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1217533713 0 0) 0x7feef815cdf0 con 0x7feef815a2a0 2024-04-24T05:47:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.325+0000 7feef7400640 1 --2- 172.21.15.123:0/1531413876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef80ad860 0x7feef80af240 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7feef8071bc0 tx=0x7feee402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8ccb4520bb29d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.325+0000 7feef6a00640 1 -- 172.21.15.123:0/1531413876 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4037070 con 0x7feef80ad860 2024-04-24T05:47:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.325+0000 7feef6a00640 1 -- 172.21.15.123:0/1531413876 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee4032440 con 0x7feef80ad860 2024-04-24T05:47:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.325+0000 7feef6a00640 1 -- 172.21.15.123:0/1531413876 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee402ae60 con 0x7feef80ad860 2024-04-24T05:47:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.325+0000 7feeff000640 1 -- 172.21.15.123:0/1531413876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef80ad860 msgr2=0x7feef80af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.325+0000 7feeff000640 1 --2- 172.21.15.123:0/1531413876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef80ad860 0x7feef80af240 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7feef8071bc0 tx=0x7feee402a110 comp rx=0 tx=0).stop 2024-04-24T05:47:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.326+0000 7feeff000640 1 -- 172.21.15.123:0/1531413876 shutdown_connections 2024-04-24T05:47:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.326+0000 7feeff000640 1 --2- 172.21.15.123:0/1531413876 >> v2:172.21.15.155:3300/0 conn(0x7feef80af780 0x7feef806bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.326+0000 7feeff000640 1 --2- 172.21.15.123:0/1531413876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef80ad860 0x7feef80af240 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.326+0000 7feeff000640 1 -- 172.21.15.123:0/1531413876 >> 172.21.15.123:0/1531413876 conn(0x7feef80b1e70 msgr2=0x7feef80b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.326+0000 7feeff000640 1 -- 172.21.15.123:0/1531413876 shutdown_connections 2024-04-24T05:47:50.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.326+0000 7feeff000640 1 -- 172.21.15.123:0/1531413876 wait complete. 2024-04-24T05:47:50.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.327+0000 7feeff000640 1 Processor -- start 2024-04-24T05:47:50.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.328+0000 7feeff000640 1 -- start start 2024-04-24T05:47:50.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.328+0000 7feeff000640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feef80ad860 0x7feef80c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.328+0000 7feeff000640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feef80af780 0x7feef80ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.329+0000 7feeff000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef815a2a0 0x7feef80cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.329+0000 7feeff000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feef80afb90 con 0x7feef815a2a0 2024-04-24T05:47:50.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.329+0000 7feeff000640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feef815d690 con 0x7feef80af780 2024-04-24T05:47:50.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.329+0000 7feeff000640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feef806c040 con 0x7feef80ad860 2024-04-24T05:47:50.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.329+0000 7feefd000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef815a2a0 0x7feef80cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.329+0000 7feefd000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef815a2a0 0x7feef80cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46414/0 (socket says 172.21.15.123:46414) 2024-04-24T05:47:50.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.329+0000 7feefd000640 1 -- 172.21.15.123:0/3602863424 learned_addr learned my addr 172.21.15.123:0/3602863424 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:47:50.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.329+0000 7feef7400640 1 --2- 172.21.15.123:0/3602863424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feef80af780 0x7feef80ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.329+0000 7feef7e00640 1 --2- 172.21.15.123:0/3602863424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feef80ad860 0x7feef80c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.329+0000 7feefd000640 1 -- 172.21.15.123:0/3602863424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feef80ad860 msgr2=0x7feef80c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.329+0000 7feefd000640 1 --2- 172.21.15.123:0/3602863424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feef80ad860 0x7feef80c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.329+0000 7feefd000640 1 -- 172.21.15.123:0/3602863424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feef80af780 msgr2=0x7feef80ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.329+0000 7feefd000640 1 --2- 172.21.15.123:0/3602863424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feef80af780 0x7feef80ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.329+0000 7feefd000640 1 -- 172.21.15.123:0/3602863424 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef8152700 con 0x7feef815a2a0 2024-04-24T05:47:50.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.330+0000 7feefd000640 1 --2- 172.21.15.123:0/3602863424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef815a2a0 0x7feef80cedf0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7feef000c5e0 tx=0x7feef000caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:50.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.330+0000 7feeefe00640 1 -- 172.21.15.123:0/3602863424 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0015070 con 0x7feef815a2a0 2024-04-24T05:47:50.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.330+0000 7feeefe00640 1 -- 172.21.15.123:0/3602863424 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef0011840 con 0x7feef815a2a0 2024-04-24T05:47:50.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.330+0000 7feeefe00640 1 -- 172.21.15.123:0/3602863424 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0011c30 con 0x7feef815a2a0 2024-04-24T05:47:50.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.330+0000 7feeff000640 1 -- 172.21.15.123:0/3602863424 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef81529e0 con 0x7feef815a2a0 2024-04-24T05:47:50.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.330+0000 7feeff000640 1 -- 172.21.15.123:0/3602863424 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef80c9a30 con 0x7feef815a2a0 2024-04-24T05:47:50.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.331+0000 7feeff000640 1 -- 172.21.15.123:0/3602863424 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef8074350 con 0x7feef815a2a0 2024-04-24T05:47:50.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.333+0000 7feeefe00640 1 -- 172.21.15.123:0/3602863424 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feef0018070 con 0x7feef815a2a0 2024-04-24T05:47:50.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.333+0000 7feeefe00640 1 --2- 172.21.15.123:0/3602863424 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feed806c8f0 0x7feed806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:47:50.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.334+0000 7feeefe00640 1 -- 172.21.15.123:0/3602863424 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7feef008bad0 con 0x7feef815a2a0 2024-04-24T05:47:50.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.334+0000 7feef7e00640 1 --2- 172.21.15.123:0/3602863424 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feed806c8f0 0x7feed806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:47:50.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.335+0000 7feef7e00640 1 --2- 172.21.15.123:0/3602863424 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feed806c8f0 0x7feed806edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feee80012e0 tx=0x7feee8005bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:47:50.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.339+0000 7feeefe00640 1 -- 172.21.15.123:0/3602863424 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feef8074350 con 0x7feef815a2a0 2024-04-24T05:47:50.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.566+0000 7feeff000640 1 -- 172.21.15.123:0/3602863424 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7feef8068a20 con 0x7feef815a2a0 2024-04-24T05:47:50.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.567+0000 7feeefe00640 1 -- 172.21.15.123:0/3602863424 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7feef0059e00 con 0x7feef815a2a0 2024-04-24T05:47:50.569 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T05:47:50.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.569+0000 7feeff000640 1 -- 172.21.15.123:0/3602863424 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feed806c8f0 msgr2=0x7feed806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.569+0000 7feeff000640 1 --2- 172.21.15.123:0/3602863424 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feed806c8f0 0x7feed806edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feee80012e0 tx=0x7feee8005bf0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.569+0000 7feeff000640 1 -- 172.21.15.123:0/3602863424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef815a2a0 msgr2=0x7feef80cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:47:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.569+0000 7feeff000640 1 --2- 172.21.15.123:0/3602863424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef815a2a0 0x7feef80cedf0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7feef000c5e0 tx=0x7feef000caa0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.570+0000 7feeff000640 1 -- 172.21.15.123:0/3602863424 shutdown_connections 2024-04-24T05:47:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.570+0000 7feeff000640 1 --2- 172.21.15.123:0/3602863424 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7feed806c8f0 0x7feed806edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.570+0000 7feeff000640 1 --2- 172.21.15.123:0/3602863424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feef815a2a0 0x7feef80cedf0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.570+0000 7feeff000640 1 --2- 172.21.15.123:0/3602863424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feef80af780 0x7feef80ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.570+0000 7feeff000640 1 --2- 172.21.15.123:0/3602863424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feef80ad860 0x7feef80c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:47:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.570+0000 7feeff000640 1 -- 172.21.15.123:0/3602863424 >> 172.21.15.123:0/3602863424 conn(0x7feef80b1e70 msgr2=0x7feef8154e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:47:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.570+0000 7feeff000640 1 -- 172.21.15.123:0/3602863424 shutdown_connections 2024-04-24T05:47:50.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50.570+0000 7feeff000640 1 -- 172.21.15.123:0/3602863424 wait complete. 2024-04-24T05:47:50.590 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T05:47:50.590 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T05:47:50.591 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-04-24T05:47:50.591 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T05:47:50.591 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T05:47:50.591 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T05:47:50.591 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T05:47:50.591 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T05:47:50.591 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T05:47:50.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:50.591 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-04-24T05:47:50.636 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T05:47:50.637 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T05:47:50.637 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T05:47:50.637 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:47:50.637 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-24T05:47:50.637 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:47:50.637 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:47:50.637 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-04-24T05:47:50.637 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-04-24T05:47:50.637 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:47:50.637 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-04-24T05:47:50.637 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-04-24T05:47:50.637 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T05:47:50.638 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-04-24T05:47:50.638 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T05:47:50.638 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-04-24T05:47:50.638 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T05:47:50.638 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42517sec preferred_lft 42517sec 2024-04-24T05:47:50.638 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-04-24T05:47:50.638 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:47:50.638 INFO:teuthology.orchestra.run.smithi123.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-24T05:47:50.638 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 4a:4a:d7:fd:35:cb brd ff:ff:ff:ff:ff:ff 2024-04-24T05:47:50.638 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-24T05:47:50.638 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:47:50.638 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::9041:44ff:fe5a:59c3/64 scope link 2024-04-24T05:47:50.638 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:47:50.639 INFO:teuthology.orchestra.run.smithi123.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T05:47:50.639 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 4a:4a:d7:fd:35:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:47:50.639 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::484a:d7ff:fefd:35cb/64 scope link 2024-04-24T05:47:50.639 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:47:50.639 INFO:teuthology.orchestra.run.smithi123.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-24T05:47:50.639 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 66:60:80:f8:89:ec brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T05:47:50.639 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::6460:80ff:fef8:89ec/64 scope link 2024-04-24T05:47:50.639 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-24T05:47:50.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:50.639 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T05:47:50.753 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-04-24T05:47:50.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:50.757 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-04-24T05:47:50.806 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-04-24T05:47:50.806 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T05:47:50.806 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T05:47:50.806 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T05:47:50.806 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T05:47:50.806 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T05:47:50.807 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-24T05:47:50.807 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T05:47:50.807 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T05:47:50.807 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T05:47:50.807 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-24T05:47:50.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T05:47:51.007 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:47:51.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:51.017 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-24T05:47:51.067 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T05:47:51.067 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-24T05:47:51.068 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T05:47:51.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:51.068 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-04-24T05:47:51.149 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T05:47:51.149 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T05:47:51.150 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T05:47:51.204 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T05:47:51.205 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T05:47:51.205 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T05:47:51.257 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-24T05:47:51.343 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T05:47:51.395 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-24T05:47:51.395 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-24T05:47:51.395 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-24T05:47:51.395 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-24T05:47:51.395 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-24T05:47:51.396 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-24T05:47:51.396 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-24T05:47:51.397 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-24T05:47:51.397 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-24T05:47:51.397 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-24T05:47:51.397 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-24T05:47:51.398 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-24T05:47:51.398 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T05:47:51.398 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-24T05:47:51.398 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-24T05:47:51.398 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-24T05:47:51.398 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-24T05:47:51.398 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-24T05:47:51.398 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-24T05:47:51.398 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T05:47:51.398 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T05:47:51.398 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T05:47:51.398 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T05:47:51.398 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-24T05:47:51.399 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-24T05:47:51.399 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T05:47:51.399 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T05:47:51.399 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T05:47:51.399 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T05:47:51.399 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-24T05:47:51.399 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:47:51.399 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:47:51.399 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T05:47:51.399 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T05:47:51.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:51.400 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-24T05:47:51.444 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T05:47:51.445 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T05:47:51.493 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-24T05:47:51.562 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T05:47:51.606 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-24T05:47:51.606 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-24T05:47:51.606 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-24T05:47:51.606 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-24T05:47:51.606 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-24T05:47:51.606 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-24T05:47:51.606 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-24T05:47:51.606 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-24T05:47:51.606 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-24T05:47:51.607 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-24T05:47:51.607 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-24T05:47:51.607 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-24T05:47:51.607 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T05:47:51.607 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-24T05:47:51.607 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-24T05:47:51.607 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-24T05:47:51.607 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-24T05:47:51.607 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-24T05:47:51.607 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-24T05:47:51.607 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T05:47:51.607 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T05:47:51.607 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T05:47:51.607 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T05:47:51.608 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-24T05:47:51.608 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-24T05:47:51.608 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-0 2024-04-24T05:47:51.608 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-1 2024-04-24T05:47:51.608 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-2 2024-04-24T05:47:51.608 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-3 2024-04-24T05:47:51.608 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-24T05:47:51.608 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:47:51.608 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T05:47:51.608 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T05:47:51.608 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-24T05:47:51.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:51.609 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-24T05:47:51.669 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:2024-04-24T05:47:51.668+0000 7eff50e18480 -1 init, newargv = 0x5580aac9b090 newargc=15 2024-04-24T05:47:51.669 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:ceph-fuse[50115]: starting ceph client 2024-04-24T05:47:51.680 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:ceph-fuse[50115]: starting fuse 2024-04-24T05:47:51.696 INFO:teuthology.orchestra.run.smithi123.stdout:41 2024-04-24T05:47:51.696 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-24T05:47:51.696 DEBUG:teuthology.orchestra.run.smithi123:> sudo stdin-killer -- python3 -c ' 2024-04-24T05:47:51.696 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2024-04-24T05:47:51.696 DEBUG:teuthology.orchestra.run.smithi123:> import re 2024-04-24T05:47:51.696 DEBUG:teuthology.orchestra.run.smithi123:> import os 2024-04-24T05:47:51.696 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2024-04-24T05:47:51.696 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2024-04-24T05:47:51.697 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2024-04-24T05:47:51.698 DEBUG:teuthology.orchestra.run.smithi123:> return f 2024-04-24T05:47:51.698 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T05:47:51.698 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-24T05:47:51.698 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.0")) 2024-04-24T05:47:51.698 DEBUG:teuthology.orchestra.run.smithi123:> ' 2024-04-24T05:47:51.881 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.0.50115.asok 2024-04-24T05:47:51.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:47:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T05:47:51.898 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50115.asok 2024-04-24T05:47:51.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:51.899 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50115.asok status 2024-04-24T05:47:52.116 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-24T05:47:52.116 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2024-04-24T05:47:52.116 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "a3c8a22bd5efe18ab230a8e2d072db8f381a0411", 2024-04-24T05:47:52.116 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T05:47:52.116 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "0", 2024-04-24T05:47:52.116 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2024-04-24T05:47:52.116 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T05:47:52.116 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "50115", 2024-04-24T05:47:52.116 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2024-04-24T05:47:52.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 5691, 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 5691 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3188177572 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:52.117 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-24T05:47:52.118 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-24T05:47:52.118 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-04-24T05:47:52.118 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3188177572 2024-04-24T05:47:52.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-24T05:47:52.118 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.5691 192.168.0.1:0/3188177572", 2024-04-24T05:47:52.118 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.1:0/3188177572", 2024-04-24T05:47:52.118 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2024-04-24T05:47:52.118 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 18, 2024-04-24T05:47:52.118 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 39, 2024-04-24T05:47:52.118 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2024-04-24T05:47:52.118 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2024-04-24T05:47:52.118 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2024-04-24T05:47:52.119 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-24T05:47:52.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:52.131 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2024-04-24T05:47:52.131 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T05:47:52.181 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2024-04-24T05:47:52.181 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T05:47:52.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:52.181 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T05:47:52.255 DEBUG:teuthology.orchestra.run.smithi123:> mktemp -d --suffix=xfstests-dev 2024-04-24T05:47:52.304 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.ejcLBa8fOwxfstests-dev 2024-04-24T05:47:52.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:52.304 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.ejcLBa8fOwxfstests-dev) 2024-04-24T05:47:52.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:52.370 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.ejcLBa8fOwxfstests-dev) 2024-04-24T05:47:52.423 INFO:teuthology.orchestra.run.smithi123.stderr:Cloning into '/tmp/tmp.ejcLBa8fOwxfstests-dev'... 2024-04-24T05:47:53.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:53.480 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-04-24T05:47:53.547 DEBUG:teuthology.orchestra.run.smithi123:> mktemp -d --suffix=test 2024-04-24T05:47:53.600 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.CHueoLyoRxtest 2024-04-24T05:47:53.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:47:53.600 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-04-24T05:47:53.654 DEBUG:teuthology.orchestra.run.smithi123:> mktemp -d --suffix=scratch 2024-04-24T05:47:53.699 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.eOr2Zr3UAsscratch 2024-04-24T05:47:53.699 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2024-04-24T05:47:53.699 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs 2024-04-24T05:47:53.939 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:47:53.947 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:47:54.516 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:52 ago on Wed 24 Apr 2024 05:43:02 AM UTC. 2024-04-24T05:47:54.605 INFO:teuthology.orchestra.run.smithi123.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T05:47:54.605 INFO:teuthology.orchestra.run.smithi123.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T05:47:54.606 INFO:teuthology.orchestra.run.smithi123.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2024-04-24T05:47:54.607 INFO:teuthology.orchestra.run.smithi123.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2024-04-24T05:47:54.608 INFO:teuthology.orchestra.run.smithi123.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2024-04-24T05:47:54.609 INFO:teuthology.orchestra.run.smithi123.stdout:Package e2fsprogs-1.46.5-5.el9.x86_64 is already installed. 2024-04-24T05:47:54.611 INFO:teuthology.orchestra.run.smithi123.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2024-04-24T05:47:54.612 INFO:teuthology.orchestra.run.smithi123.stdout:Package gcc-11.4.1-3.el9.x86_64 is already installed. 2024-04-24T05:47:54.614 INFO:teuthology.orchestra.run.smithi123.stdout:Package git-2.43.0-1.el9.x86_64 is already installed. 2024-04-24T05:47:54.616 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T05:47:54.617 INFO:teuthology.orchestra.run.smithi123.stdout:Package libaio-devel-0.3.111-13.el9.x86_64 is already installed. 2024-04-24T05:47:54.618 INFO:teuthology.orchestra.run.smithi123.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2024-04-24T05:47:54.619 INFO:teuthology.orchestra.run.smithi123.stdout:Package libuuid-devel-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T05:47:54.620 INFO:teuthology.orchestra.run.smithi123.stdout:Package lvm2-9:2.03.23-2.el9.x86_64 is already installed. 2024-04-24T05:47:54.621 INFO:teuthology.orchestra.run.smithi123.stdout:Package make-1:4.3-8.el9.x86_64 is already installed. 2024-04-24T05:47:54.622 INFO:teuthology.orchestra.run.smithi123.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2024-04-24T05:47:54.623 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-3.9.18-3.el9.x86_64 is already installed. 2024-04-24T05:47:54.624 INFO:teuthology.orchestra.run.smithi123.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2024-04-24T05:47:54.624 INFO:teuthology.orchestra.run.smithi123.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2024-04-24T05:47:54.627 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T05:47:54.675 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-24T05:47:54.676 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-24T05:47:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-24T05:47:54.676 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-24T05:47:54.676 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-24T05:47:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: dump x86_64 1:0.4-0.51.b47.el9 epel 88 k 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: fio x86_64 3.35-1.el9 appstream 6.2 M 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: gdbm-devel x86_64 1:1.19-4.el9 crb 59 k 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: indent x86_64 2.2.13-5.el9 epel 179 k 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-devel x86_64 5.14.0-437.el9 appstream 16 M 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: libcap-devel x86_64 2.48-9.el9 appstream 41 k 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: sqlite x86_64 3.34.1-7.el9 appstream 748 k 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: udftools x86_64 2.2-5.el9 appstream 149 k 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: gdbm x86_64 1:1.19-4.el9 crb 132 k 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: libnbd x86_64 1.18.1-3.el9 appstream 166 k 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: rmt x86_64 2:1.6-6.el9 baseos 46 k 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-http x86_64 3.35-1.el9 appstream 18 k 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-libaio x86_64 3.35-1.el9 appstream 16 k 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-nbd x86_64 3.35-1.el9 appstream 15 k 2024-04-24T05:47:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-rados x86_64 3.35-1.el9 appstream 16 k 2024-04-24T05:47:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-rbd x86_64 3.35-1.el9 appstream 16 k 2024-04-24T05:47:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-rdma x86_64 3.35-1.el9 appstream 21 k 2024-04-24T05:47:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:54.678 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-24T05:47:54.678 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-24T05:47:54.678 INFO:teuthology.orchestra.run.smithi123.stdout:Install 17 Packages 2024-04-24T05:47:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:47:54.679 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 24 M 2024-04-24T05:47:54.679 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 78 M 2024-04-24T05:47:54.679 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-24T05:47:55.311 INFO:teuthology.orchestra.run.smithi123.stdout:(1/17): rmt-1.6-6.el9.x86_64.rpm 324 kB/s | 46 kB 00:00 2024-04-24T05:47:55.736 INFO:teuthology.orchestra.run.smithi123.stdout:(2/17): fio-engine-http-3.35-1.el9.x86_64.rpm 32 kB/s | 18 kB 00:00 2024-04-24T05:47:55.886 INFO:teuthology.orchestra.run.smithi123.stdout:(3/17): fio-engine-libaio-3.35-1.el9.x86_64.rpm 27 kB/s | 16 kB 00:00 2024-04-24T05:47:56.329 INFO:teuthology.orchestra.run.smithi123.stdout:(4/17): fio-engine-nbd-3.35-1.el9.x86_64.rpm 25 kB/s | 15 kB 00:00 2024-04-24T05:47:56.470 INFO:teuthology.orchestra.run.smithi123.stdout:(5/17): fio-engine-rados-3.35-1.el9.x86_64.rpm 27 kB/s | 16 kB 00:00 2024-04-24T05:47:56.896 INFO:teuthology.orchestra.run.smithi123.stdout:(6/17): fio-engine-rbd-3.35-1.el9.x86_64.rpm 29 kB/s | 16 kB 00:00 2024-04-24T05:47:57.104 INFO:teuthology.orchestra.run.smithi123.stdout:(7/17): fio-engine-rdma-3.35-1.el9.x86_64.rpm 33 kB/s | 21 kB 00:00 2024-04-24T05:47:57.796 INFO:teuthology.orchestra.run.smithi123.stdout:(8/17): libcap-devel-2.48-9.el9.x86_64.rpm 59 kB/s | 41 kB 00:00 2024-04-24T05:47:58.713 INFO:teuthology.orchestra.run.smithi123.stdout:(9/17): libnbd-1.18.1-3.el9.x86_64.rpm 182 kB/s | 166 kB 00:00 2024-04-24T05:48:00.347 INFO:teuthology.orchestra.run.smithi123.stdout:(10/17): sqlite-3.34.1-7.el9.x86_64.rpm 459 kB/s | 748 kB 00:01 2024-04-24T05:48:01.247 INFO:teuthology.orchestra.run.smithi123.stdout:(11/17): udftools-2.2-5.el9.x86_64.rpm 166 kB/s | 149 kB 00:00 2024-04-24T05:48:02.089 INFO:teuthology.orchestra.run.smithi123.stdout:(12/17): gdbm-1.19-4.el9.x86_64.rpm 157 kB/s | 132 kB 00:00 2024-04-24T05:48:02.814 INFO:teuthology.orchestra.run.smithi123.stdout:(13/17): gdbm-devel-1.19-4.el9.x86_64.rpm 81 kB/s | 59 kB 00:00 2024-04-24T05:48:03.448 INFO:teuthology.orchestra.run.smithi123.stdout:(14/17): dump-0.4-0.51.b47.el9.x86_64.rpm 139 kB/s | 88 kB 00:00 2024-04-24T05:48:04.006 INFO:teuthology.orchestra.run.smithi123.stdout:(15/17): indent-2.2.13-5.el9.x86_64.rpm 322 kB/s | 179 kB 00:00 2024-04-24T05:48:04.615 INFO:teuthology.orchestra.run.smithi123.stdout:(16/17): fio-3.35-1.el9.x86_64.rpm 676 kB/s | 6.2 MB 00:09 2024-04-24T05:48:20.047 INFO:teuthology.orchestra.run.smithi123.stdout:(17/17): kernel-devel-5.14.0-437.el9.x86_64.rpm 700 kB/s | 16 MB 00:23 2024-04-24T05:48:20.047 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-24T05:48:20.048 INFO:teuthology.orchestra.run.smithi123.stdout:Total 957 kB/s | 24 MB 00:25 2024-04-24T05:48:20.752 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-24T05:48:20.860 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-24T05:48:20.860 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-24T05:48:21.658 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-24T05:48:21.658 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-24T05:48:22.817 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-24T05:48:22.911 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gdbm-1:1.19-4.el9.x86_64 1/17 2024-04-24T05:48:22.944 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libnbd-1.18.1-3.el9.x86_64 2/17 2024-04-24T05:48:22.978 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 3/17 2024-04-24T05:48:23.020 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 4/17 2024-04-24T05:48:23.074 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 5/17 2024-04-24T05:48:23.114 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 6/17 2024-04-24T05:48:23.148 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 7/17 2024-04-24T05:48:23.398 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 8/17 2024-04-24T05:48:23.453 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fio-3.35-1.el9.x86_64 9/17 2024-04-24T05:48:23.492 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rmt-2:1.6-6.el9.x86_64 10/17 2024-04-24T05:48:23.530 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : dump-1:0.4-0.51.b47.el9.x86_64 11/17 2024-04-24T05:48:23.608 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gdbm-devel-1:1.19-4.el9.x86_64 12/17 2024-04-24T05:48:23.667 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : indent-2.2.13-5.el9.x86_64 13/17 2024-04-24T05:48:23.728 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : udftools-2.2-5.el9.x86_64 14/17 2024-04-24T05:48:23.807 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 15/17 2024-04-24T05:48:24.074 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:48:24.082 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:48:35.365 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcap-devel-2.48-9.el9.x86_64 16/17 2024-04-24T05:48:35.742 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-devel-5.14.0-437.el9.x86_64 17/17 2024-04-24T05:48:54.209 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:48:54.215 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:49:24.339 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:49:24.349 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:49:26.440 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-devel-5.14.0-437.el9.x86_64 17/17 2024-04-24T05:49:26.441 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rmt-2:1.6-6.el9.x86_64 1/17 2024-04-24T05:49:26.441 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fio-3.35-1.el9.x86_64 2/17 2024-04-24T05:49:26.441 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 3/17 2024-04-24T05:49:26.441 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 4/17 2024-04-24T05:49:26.441 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 5/17 2024-04-24T05:49:26.441 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 6/17 2024-04-24T05:49:26.441 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 7/17 2024-04-24T05:49:26.441 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 8/17 2024-04-24T05:49:26.441 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-devel-5.14.0-437.el9.x86_64 9/17 2024-04-24T05:49:26.441 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcap-devel-2.48-9.el9.x86_64 10/17 2024-04-24T05:49:26.441 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libnbd-1.18.1-3.el9.x86_64 11/17 2024-04-24T05:49:26.441 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 12/17 2024-04-24T05:49:26.442 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : udftools-2.2-5.el9.x86_64 13/17 2024-04-24T05:49:26.442 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gdbm-1:1.19-4.el9.x86_64 14/17 2024-04-24T05:49:26.442 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gdbm-devel-1:1.19-4.el9.x86_64 15/17 2024-04-24T05:49:26.442 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : dump-1:0.4-0.51.b47.el9.x86_64 16/17 2024-04-24T05:49:26.964 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : indent-2.2.13-5.el9.x86_64 17/17 2024-04-24T05:49:26.964 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:49:26.964 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-24T05:49:26.964 INFO:teuthology.orchestra.run.smithi123.stdout: dump-1:0.4-0.51.b47.el9.x86_64 fio-3.35-1.el9.x86_64 2024-04-24T05:49:26.964 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-http-3.35-1.el9.x86_64 fio-engine-libaio-3.35-1.el9.x86_64 2024-04-24T05:49:26.964 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-nbd-3.35-1.el9.x86_64 fio-engine-rados-3.35-1.el9.x86_64 2024-04-24T05:49:26.964 INFO:teuthology.orchestra.run.smithi123.stdout: fio-engine-rbd-3.35-1.el9.x86_64 fio-engine-rdma-3.35-1.el9.x86_64 2024-04-24T05:49:26.964 INFO:teuthology.orchestra.run.smithi123.stdout: gdbm-1:1.19-4.el9.x86_64 gdbm-devel-1:1.19-4.el9.x86_64 2024-04-24T05:49:26.964 INFO:teuthology.orchestra.run.smithi123.stdout: indent-2.2.13-5.el9.x86_64 kernel-devel-5.14.0-437.el9.x86_64 2024-04-24T05:49:26.964 INFO:teuthology.orchestra.run.smithi123.stdout: libcap-devel-2.48-9.el9.x86_64 libnbd-1.18.1-3.el9.x86_64 2024-04-24T05:49:26.964 INFO:teuthology.orchestra.run.smithi123.stdout: rmt-2:1.6-6.el9.x86_64 sqlite-3.34.1-7.el9.x86_64 2024-04-24T05:49:26.965 INFO:teuthology.orchestra.run.smithi123.stdout: udftools-2.2-5.el9.x86_64 2024-04-24T05:49:26.965 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:49:26.965 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-24T05:49:27.393 DEBUG:teuthology.orchestra.run.smithi123:> sudo useradd -m fsgqa 2024-04-24T05:49:27.476 INFO:teuthology.orchestra.run.smithi123.stderr:useradd: user 'fsgqa' already exists 2024-04-24T05:49:27.479 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-24T05:49:27.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo groupadd fsgqa 2024-04-24T05:49:27.545 INFO:teuthology.orchestra.run.smithi123.stderr:groupadd: group 'fsgqa' already exists 2024-04-24T05:49:27.548 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-24T05:49:27.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo useradd fsgqa2 2024-04-24T05:49:28.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo useradd 123456-fsgqa 2024-04-24T05:49:28.833 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-04-24T05:49:29.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.015+0000 7f0fcd800640 1 Processor -- start 2024-04-24T05:49:29.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.015+0000 7f0fcd800640 1 -- start start 2024-04-24T05:49:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.017+0000 7f0fcd800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0fc815a2a0 0x7f0fc80ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:49:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.017+0000 7f0fcd800640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0fc80ad690 0x7f0fc80af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:49:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.017+0000 7f0fcd800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fc80af780 0x7f0fc8141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:49:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.017+0000 7f0fcd800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0fc815a680 con 0x7f0fc815a2a0 2024-04-24T05:49:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.017+0000 7f0fcd800640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0fc815da00 con 0x7f0fc80ad690 2024-04-24T05:49:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.017+0000 7f0fcd800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc80afbf0 con 0x7f0fc80af780 2024-04-24T05:49:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.017+0000 7f0fc6a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0fc815a2a0 0x7f0fc80ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:49:29.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.017+0000 7f0fc6a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0fc815a2a0 0x7f0fc80ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39386/0 (socket says 172.21.15.123:39386) 2024-04-24T05:49:29.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.017+0000 7f0fc6a00640 1 -- 172.21.15.123:0/867626548 learned_addr learned my addr 172.21.15.123:0/867626548 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:49:29.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.018+0000 7f0fc6a00640 1 -- 172.21.15.123:0/867626548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fc80af780 msgr2=0x7f0fc8141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:49:29.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.018+0000 7f0fc6a00640 1 --2- 172.21.15.123:0/867626548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fc80af780 0x7f0fc8141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.018+0000 7f0fc6a00640 1 -- 172.21.15.123:0/867626548 >> v2:172.21.15.155:3300/0 conn(0x7f0fc80ad690 msgr2=0x7f0fc80af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:49:29.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.018+0000 7f0fc6a00640 1 --2- 172.21.15.123:0/867626548 >> v2:172.21.15.155:3300/0 conn(0x7f0fc80ad690 0x7f0fc80af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.018+0000 7f0fc6a00640 1 -- 172.21.15.123:0/867626548 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc815cdf0 con 0x7f0fc815a2a0 2024-04-24T05:49:29.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.019+0000 7f0fc6a00640 1 --2- 172.21.15.123:0/867626548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc815a2a0 0x7f0fc80ad150 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80741d0 tx=0x7f0fb802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60bfce1a1dea8d00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:49:29.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.019+0000 7f0fc5600640 1 -- 172.21.15.123:0/867626548 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8037070 con 0x7f0fc815a2a0 2024-04-24T05:49:29.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.019+0000 7f0fc5600640 1 -- 172.21.15.123:0/867626548 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb8032440 con 0x7f0fc815a2a0 2024-04-24T05:49:29.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.019+0000 7f0fc5600640 1 -- 172.21.15.123:0/867626548 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8032810 con 0x7f0fc815a2a0 2024-04-24T05:49:29.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.020+0000 7f0fcd800640 1 -- 172.21.15.123:0/867626548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc815a2a0 msgr2=0x7f0fc80ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:49:29.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.020+0000 7f0fcd800640 1 --2- 172.21.15.123:0/867626548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc815a2a0 0x7f0fc80ad150 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80741d0 tx=0x7f0fb802a140 comp rx=0 tx=0).stop 2024-04-24T05:49:29.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.020+0000 7f0fcd800640 1 -- 172.21.15.123:0/867626548 shutdown_connections 2024-04-24T05:49:29.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.020+0000 7f0fcd800640 1 --2- 172.21.15.123:0/867626548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fc80af780 0x7f0fc8141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.020+0000 7f0fcd800640 1 --2- 172.21.15.123:0/867626548 >> v2:172.21.15.155:3300/0 conn(0x7f0fc80ad690 0x7f0fc80af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.020+0000 7f0fcd800640 1 --2- 172.21.15.123:0/867626548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc815a2a0 0x7f0fc80ad150 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.020+0000 7f0fcd800640 1 -- 172.21.15.123:0/867626548 >> 172.21.15.123:0/867626548 conn(0x7f0fc80b1e70 msgr2=0x7f0fc80b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:49:29.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.020+0000 7f0fcd800640 1 -- 172.21.15.123:0/867626548 shutdown_connections 2024-04-24T05:49:29.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.020+0000 7f0fcd800640 1 -- 172.21.15.123:0/867626548 wait complete. 2024-04-24T05:49:29.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.022+0000 7f0fcd800640 1 Processor -- start 2024-04-24T05:49:29.023 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.022+0000 7f0fcd800640 1 -- start start 2024-04-24T05:49:29.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fcd800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fc80ad690 0x7f0fc8073790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:49:29.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fcd800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc80af780 0x7f0fc815edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:49:29.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fcd800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fc815a2a0 0x7f0fc815f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:49:29.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fcd800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc815d690 con 0x7f0fc80af780 2024-04-24T05:49:29.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fcd800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc815d3b0 con 0x7f0fc815a2a0 2024-04-24T05:49:29.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fcd800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc81423c0 con 0x7f0fc80ad690 2024-04-24T05:49:29.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fc6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc80af780 0x7f0fc815edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:49:29.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fc6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc80af780 0x7f0fc815edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39402/0 (socket says 172.21.15.123:39402) 2024-04-24T05:49:29.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fc6000640 1 -- 172.21.15.123:0/3368280541 learned_addr learned my addr 172.21.15.123:0/3368280541 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:49:29.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fc6a00640 1 --2- 172.21.15.123:0/3368280541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fc80ad690 0x7f0fc8073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:49:29.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fc6000640 1 -- 172.21.15.123:0/3368280541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fc80ad690 msgr2=0x7f0fc8073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:49:29.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fc6000640 1 --2- 172.21.15.123:0/3368280541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fc80ad690 0x7f0fc8073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fc6000640 1 -- 172.21.15.123:0/3368280541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fc815a2a0 msgr2=0x7f0fc815f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:49:29.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fc7400640 1 --2- 172.21.15.123:0/3368280541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fc815a2a0 0x7f0fc815f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:49:29.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fc6000640 1 --2- 172.21.15.123:0/3368280541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fc815a2a0 0x7f0fc815f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.023+0000 7f0fc6000640 1 -- 172.21.15.123:0/3368280541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc80720c0 con 0x7f0fc80af780 2024-04-24T05:49:29.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.024+0000 7f0fc6000640 1 --2- 172.21.15.123:0/3368280541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc80af780 0x7f0fc815edf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400b4a0 tx=0x7f0fb400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:49:29.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.024+0000 7f0fbea00640 1 -- 172.21.15.123:0/3368280541 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4014070 con 0x7f0fc80af780 2024-04-24T05:49:29.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.024+0000 7f0fbea00640 1 -- 172.21.15.123:0/3368280541 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb4010610 con 0x7f0fc80af780 2024-04-24T05:49:29.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.024+0000 7f0fbea00640 1 -- 172.21.15.123:0/3368280541 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4010a00 con 0x7f0fc80af780 2024-04-24T05:49:29.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.024+0000 7f0fcd800640 1 -- 172.21.15.123:0/3368280541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc80723a0 con 0x7f0fc80af780 2024-04-24T05:49:29.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.024+0000 7f0fcd800640 1 -- 172.21.15.123:0/3368280541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc8072960 con 0x7f0fc80af780 2024-04-24T05:49:29.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.025+0000 7f0fcd800640 1 -- 172.21.15.123:0/3368280541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc8068a20 con 0x7f0fc80af780 2024-04-24T05:49:29.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.026+0000 7f0fbea00640 1 -- 172.21.15.123:0/3368280541 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fb4017070 con 0x7f0fc80af780 2024-04-24T05:49:29.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.028+0000 7f0fbea00640 1 --2- 172.21.15.123:0/3368280541 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0fa806c8f0 0x7f0fa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:49:29.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.028+0000 7f0fbea00640 1 -- 172.21.15.123:0/3368280541 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f0fb408b7a0 con 0x7f0fc80af780 2024-04-24T05:49:29.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.028+0000 7f0fc6a00640 1 --2- 172.21.15.123:0/3368280541 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0fa806c8f0 0x7f0fa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:49:29.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.029+0000 7f0fc6a00640 1 --2- 172.21.15.123:0/3368280541 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0fa806c8f0 0x7f0fa806edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8034000 tx=0x7f0fb802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:49:29.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.034+0000 7f0fbea00640 1 -- 172.21.15.123:0/3368280541 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0fb4059a10 con 0x7f0fc80af780 2024-04-24T05:49:29.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.310+0000 7f0fcd800640 1 -- 172.21.15.123:0/3368280541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0fc8074e70 con 0x7f0fc80af780 2024-04-24T05:49:29.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.311+0000 7f0fbea00640 1 -- 172.21.15.123:0/3368280541 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1289 (secure 0 0 0) 0x7f0fb4059310 con 0x7f0fc80af780 2024-04-24T05:49:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:49:29.313 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"6efb8524-c0d1-4076-b7c9-7485084c7ada","modified":"2024-04-24T05:44:54.802840Z","created":"2024-04-24T05:44:54.802840Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3301","nonce":0},{"type":"v1","addr":"172.21.15.155:6790","nonce":0}]},"addr":"172.21.15.155:6790/0","public_addr":"172.21.15.155:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-24T05:49:29.313 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-04-24T05:49:29.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.314+0000 7f0fcd800640 1 -- 172.21.15.123:0/3368280541 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0fa806c8f0 msgr2=0x7f0fa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:49:29.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.314+0000 7f0fcd800640 1 --2- 172.21.15.123:0/3368280541 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0fa806c8f0 0x7f0fa806edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8034000 tx=0x7f0fb802e000 comp rx=0 tx=0).stop 2024-04-24T05:49:29.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.314+0000 7f0fcd800640 1 -- 172.21.15.123:0/3368280541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc80af780 msgr2=0x7f0fc815edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:49:29.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.314+0000 7f0fcd800640 1 --2- 172.21.15.123:0/3368280541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc80af780 0x7f0fc815edf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400b4a0 tx=0x7f0fb400b960 comp rx=0 tx=0).stop 2024-04-24T05:49:29.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.314+0000 7f0fcd800640 1 -- 172.21.15.123:0/3368280541 shutdown_connections 2024-04-24T05:49:29.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.314+0000 7f0fcd800640 1 --2- 172.21.15.123:0/3368280541 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7f0fa806c8f0 0x7f0fa806edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.314+0000 7f0fcd800640 1 --2- 172.21.15.123:0/3368280541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fc815a2a0 0x7f0fc815f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.314+0000 7f0fcd800640 1 --2- 172.21.15.123:0/3368280541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fc80af780 0x7f0fc815edf0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.314+0000 7f0fcd800640 1 --2- 172.21.15.123:0/3368280541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fc80ad690 0x7f0fc8073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.314+0000 7f0fcd800640 1 -- 172.21.15.123:0/3368280541 >> 172.21.15.123:0/3368280541 conn(0x7f0fc80b1e70 msgr2=0x7f0fc80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:49:29.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.314+0000 7f0fcd800640 1 -- 172.21.15.123:0/3368280541 shutdown_connections 2024-04-24T05:49:29.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.315+0000 7f0fcd800640 1 -- 172.21.15.123:0/3368280541 wait complete. 2024-04-24T05:49:29.340 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2024-04-24T05:49:29.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.513+0000 7fda40800640 1 Processor -- start 2024-04-24T05:49:29.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.513+0000 7fda40800640 1 -- start start 2024-04-24T05:49:29.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.515+0000 7fda40800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fda3815ebf0 0x7fda3806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:49:29.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.515+0000 7fda40800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380d35f0 0x7fda380b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:49:29.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.515+0000 7fda40800640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda380d3af0 con 0x7fda380d1500 2024-04-24T05:49:29.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.515+0000 7fda40800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fda380b6030 con 0x7fda3815ebf0 2024-04-24T05:49:29.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.515+0000 7fda40800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda380b6310 con 0x7fda380d35f0 2024-04-24T05:49:29.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.515+0000 7fda3de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fda3815ebf0 0x7fda3806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:49:29.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.515+0000 7fda3de00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fda3815ebf0 0x7fda3806ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39406/0 (socket says 172.21.15.123:39406) 2024-04-24T05:49:29.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.515+0000 7fda3de00640 1 -- 172.21.15.123:0/1105322337 learned_addr learned my addr 172.21.15.123:0/1105322337 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:49:29.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.515+0000 7fda3de00640 1 -- 172.21.15.123:0/1105322337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380d35f0 msgr2=0x7fda380b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:49:29.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.515+0000 7fda3d400640 1 --2- 172.21.15.123:0/1105322337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380d35f0 0x7fda380b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:49:29.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.515+0000 7fda3de00640 1 --2- 172.21.15.123:0/1105322337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380d35f0 0x7fda380b67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.515+0000 7fda3de00640 1 -- 172.21.15.123:0/1105322337 >> v1:172.21.15.155:6789/0 conn(0x7fda380d1500 legacy=0x7fda381625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T05:49:29.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.515+0000 7fda3de00640 1 -- 172.21.15.123:0/1105322337 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda380d3dd0 con 0x7fda3815ebf0 2024-04-24T05:49:29.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.515+0000 7fda3d400640 1 --2- 172.21.15.123:0/1105322337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380d35f0 0x7fda380b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:49:29.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.516+0000 7fda3de00640 1 --2- 172.21.15.123:0/1105322337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda3815ebf0 0x7fda3806ed20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009870 tx=0x7fda2c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b2d6260749b0bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:49:29.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.517+0000 7fda3ca00640 1 -- 172.21.15.123:0/1105322337 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c037070 con 0x7fda3815ebf0 2024-04-24T05:49:29.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.517+0000 7fda3ca00640 1 -- 172.21.15.123:0/1105322337 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2c032440 con 0x7fda3815ebf0 2024-04-24T05:49:29.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.517+0000 7fda3ca00640 1 -- 172.21.15.123:0/1105322337 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c032810 con 0x7fda3815ebf0 2024-04-24T05:49:29.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.518+0000 7fda40800640 1 -- 172.21.15.123:0/1105322337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda3815ebf0 msgr2=0x7fda3806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:49:29.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.518+0000 7fda40800640 1 --2- 172.21.15.123:0/1105322337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda3815ebf0 0x7fda3806ed20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009870 tx=0x7fda2c02a210 comp rx=0 tx=0).stop 2024-04-24T05:49:29.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.518+0000 7fda40800640 1 -- 172.21.15.123:0/1105322337 shutdown_connections 2024-04-24T05:49:29.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.518+0000 7fda40800640 1 --2- 172.21.15.123:0/1105322337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380d35f0 0x7fda380b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.518+0000 7fda40800640 1 --2- 172.21.15.123:0/1105322337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda3815ebf0 0x7fda3806ed20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.518+0000 7fda40800640 1 -- 172.21.15.123:0/1105322337 >> 172.21.15.123:0/1105322337 conn(0x7fda380b9570 msgr2=0x7fda380b9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:49:29.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.518+0000 7fda40800640 1 -- 172.21.15.123:0/1105322337 shutdown_connections 2024-04-24T05:49:29.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.518+0000 7fda40800640 1 -- 172.21.15.123:0/1105322337 wait complete. 2024-04-24T05:49:29.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.520+0000 7fda40800640 1 Processor -- start 2024-04-24T05:49:29.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.520+0000 7fda40800640 1 -- start start 2024-04-24T05:49:29.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda40800640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380d1500 0x7fda38154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:49:29.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda40800640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda380d35f0 0x7fda3806d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:49:29.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda40800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda3815ebf0 0x7fda380cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:49:29.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda40800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda3806f3e0 con 0x7fda3815ebf0 2024-04-24T05:49:29.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda40800640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda3806f260 con 0x7fda380d35f0 2024-04-24T05:49:29.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda40800640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda380b6ce0 con 0x7fda380d1500 2024-04-24T05:49:29.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda3e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda3815ebf0 0x7fda380cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:49:29.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda3e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda3815ebf0 0x7fda380cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39412/0 (socket says 172.21.15.123:39412) 2024-04-24T05:49:29.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda3e800640 1 -- 172.21.15.123:0/572505303 learned_addr learned my addr 172.21.15.123:0/572505303 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-24T05:49:29.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda3d400640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda380d35f0 0x7fda3806d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:49:29.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda3de00640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380d1500 0x7fda38154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:49:29.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda3e800640 1 -- 172.21.15.123:0/572505303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380d1500 msgr2=0x7fda38154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:49:29.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda3e800640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380d1500 0x7fda38154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda3e800640 1 -- 172.21.15.123:0/572505303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda380d35f0 msgr2=0x7fda3806d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:49:29.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda3e800640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda380d35f0 0x7fda3806d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda3e800640 1 -- 172.21.15.123:0/572505303 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3806c5e0 con 0x7fda3815ebf0 2024-04-24T05:49:29.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.521+0000 7fda3de00640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380d1500 0x7fda38154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T05:49:29.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.522+0000 7fda3e800640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda3815ebf0 0x7fda380cd690 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fda3400c780 tx=0x7fda3400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:49:29.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.522+0000 7fda32000640 1 -- 172.21.15.123:0/572505303 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34017070 con 0x7fda3815ebf0 2024-04-24T05:49:29.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.522+0000 7fda32000640 1 -- 172.21.15.123:0/572505303 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda34003da0 con 0x7fda3815ebf0 2024-04-24T05:49:29.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.522+0000 7fda40800640 1 -- 172.21.15.123:0/572505303 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3806c8c0 con 0x7fda3815ebf0 2024-04-24T05:49:29.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.522+0000 7fda32000640 1 -- 172.21.15.123:0/572505303 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda340052b0 con 0x7fda3815ebf0 2024-04-24T05:49:29.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.522+0000 7fda40800640 1 -- 172.21.15.123:0/572505303 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3806ce80 con 0x7fda3815ebf0 2024-04-24T05:49:29.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.523+0000 7fda40800640 1 -- 172.21.15.123:0/572505303 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3808ca80 con 0x7fda3815ebf0 2024-04-24T05:49:29.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.524+0000 7fda32000640 1 -- 172.21.15.123:0/572505303 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda3401a070 con 0x7fda3815ebf0 2024-04-24T05:49:29.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.525+0000 7fda32000640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda1c06c8f0 0x7fda1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T05:49:29.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.525+0000 7fda32000640 1 -- 172.21.15.123:0/572505303 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fda3408b3d0 con 0x7fda3815ebf0 2024-04-24T05:49:29.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.525+0000 7fda3de00640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda1c06c8f0 0x7fda1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T05:49:29.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.526+0000 7fda3de00640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda1c06c8f0 0x7fda1c06edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fda2c034000 tx=0x7fda2c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T05:49:29.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.531+0000 7fda32000640 1 -- 172.21.15.123:0/572505303 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda3808ca80 con 0x7fda3815ebf0 2024-04-24T05:49:29.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.809+0000 7fda40800640 1 -- 172.21.15.123:0/572505303 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) -- 0x7fda38068a20 con 0x7fda3815ebf0 2024-04-24T05:49:29.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.810+0000 7fda32000640 1 -- 172.21.15.123:0/572505303 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v15) ==== 82+0+151 (secure 0 0 0) 0x7fda340595b0 con 0x7fda3815ebf0 2024-04-24T05:49:29.812 INFO:teuthology.orchestra.run.smithi123.stdout:[client.admin] 2024-04-24T05:49:29.812 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBXnChmsw/jCBAAnhJKi1uIOmvadAg0OBipng== 2024-04-24T05:49:29.812 INFO:teuthology.orchestra.run.smithi123.stdout: caps mds = "allow *" 2024-04-24T05:49:29.812 INFO:teuthology.orchestra.run.smithi123.stdout: caps mgr = "allow *" 2024-04-24T05:49:29.812 INFO:teuthology.orchestra.run.smithi123.stdout: caps mon = "allow *" 2024-04-24T05:49:29.812 INFO:teuthology.orchestra.run.smithi123.stdout: caps osd = "allow *" 2024-04-24T05:49:29.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.813+0000 7fda40800640 1 -- 172.21.15.123:0/572505303 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda1c06c8f0 msgr2=0x7fda1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:49:29.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.813+0000 7fda40800640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda1c06c8f0 0x7fda1c06edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fda2c034000 tx=0x7fda2c02e000 comp rx=0 tx=0).stop 2024-04-24T05:49:29.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.813+0000 7fda40800640 1 -- 172.21.15.123:0/572505303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda3815ebf0 msgr2=0x7fda380cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T05:49:29.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.813+0000 7fda40800640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda3815ebf0 0x7fda380cd690 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fda3400c780 tx=0x7fda3400cc40 comp rx=0 tx=0).stop 2024-04-24T05:49:29.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.813+0000 7fda40800640 1 -- 172.21.15.123:0/572505303 shutdown_connections 2024-04-24T05:49:29.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.813+0000 7fda40800640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.123:6832/1503718323,v1:172.21.15.123:6833/1503718323] conn(0x7fda1c06c8f0 0x7fda1c06edb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.813+0000 7fda40800640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda3815ebf0 0x7fda380cd690 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.813+0000 7fda40800640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda380d35f0 0x7fda3806d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.813+0000 7fda40800640 1 --2- 172.21.15.123:0/572505303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380d1500 0x7fda38154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T05:49:29.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.813+0000 7fda40800640 1 -- 172.21.15.123:0/572505303 >> 172.21.15.123:0/572505303 conn(0x7fda380b9570 msgr2=0x7fda380b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T05:49:29.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.813+0000 7fda40800640 1 -- 172.21.15.123:0/572505303 shutdown_connections 2024-04-24T05:49:29.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-24T05:49:29.813+0000 7fda40800640 1 -- 172.21.15.123:0/572505303 wait complete. 2024-04-24T05:49:29.832 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-04-24T05:49:29.871 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.Pe1GHR3xfL 2024-04-24T05:49:29.871 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:49:29.871 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/tmp/tmp.Pe1GHR3xfL 2024-04-24T05:49:29.952 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:49:29.952 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/tmp/tmp.ejcLBa8fOwxfstests-dev/local.config 2024-04-24T05:49:30.051 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.CHueoLyoRxtest export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.eOr2Zr3UAsscratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.123:6789 -k /tmp/tmp.Pe1GHR3xfL --client_mountpoint /test" 2024-04-24T05:49:30.051 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-24T05:49:30.051 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/tmp/tmp.ejcLBa8fOwxfstests-dev/ceph.exclude 2024-04-24T05:49:30.156 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo make) 2024-04-24T05:49:30.278 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize -cfi 2024-04-24T05:49:30.812 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-24T05:49:30.812 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file './config.guess' 2024-04-24T05:49:30.822 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file './config.sub' 2024-04-24T05:49:30.831 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file './install-sh' 2024-04-24T05:49:30.842 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file './ltmain.sh' 2024-04-24T05:49:30.904 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-24T05:49:30.904 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-24T05:49:30.917 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-24T05:49:30.930 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-24T05:49:30.943 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-24T05:49:30.956 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-24T05:49:30.972 INFO:teuthology.orchestra.run.smithi123.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-24T05:49:30.975 INFO:teuthology.orchestra.run.smithi123.stdout:cp include/install-sh . 2024-04-24T05:49:30.978 INFO:teuthology.orchestra.run.smithi123.stdout:aclocal -I m4 2024-04-24T05:49:32.507 INFO:teuthology.orchestra.run.smithi123.stdout:autoheader 2024-04-24T05:49:32.918 INFO:teuthology.orchestra.run.smithi123.stdout:autoconf 2024-04-24T05:49:33.172 INFO:teuthology.orchestra.run.smithi123.stdout:./configure \ 2024-04-24T05:49:33.172 INFO:teuthology.orchestra.run.smithi123.stdout: --libexecdir=/usr/lib \ 2024-04-24T05:49:33.173 INFO:teuthology.orchestra.run.smithi123.stdout: --exec_prefix=/var/lib 2024-04-24T05:49:33.823 INFO:teuthology.orchestra.run.smithi123.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-24T05:49:33.824 INFO:teuthology.orchestra.run.smithi123.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-24T05:49:33.828 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to print strings... printf 2024-04-24T05:49:33.829 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gcc... gcc 2024-04-24T05:49:34.033 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether the C compiler works... yes 2024-04-24T05:49:34.033 INFO:teuthology.orchestra.run.smithi123.stdout:checking for C compiler default output file name... a.out 2024-04-24T05:49:34.103 INFO:teuthology.orchestra.run.smithi123.stdout:checking for suffix of executables... 2024-04-24T05:49:34.226 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether we are cross compiling... no 2024-04-24T05:49:34.276 INFO:teuthology.orchestra.run.smithi123.stdout:checking for suffix of object files... o 2024-04-24T05:49:34.320 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether we are using the GNU C compiler... yes 2024-04-24T05:49:34.368 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether gcc accepts -g... yes 2024-04-24T05:49:34.471 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-24T05:49:34.480 INFO:teuthology.orchestra.run.smithi123.stdout:checking for a sed that does not truncate output... /bin/sed 2024-04-24T05:49:34.488 INFO:teuthology.orchestra.run.smithi123.stdout:checking for grep that handles long lines and -e... /bin/grep 2024-04-24T05:49:34.492 INFO:teuthology.orchestra.run.smithi123.stdout:checking for egrep... /bin/grep -E 2024-04-24T05:49:34.496 INFO:teuthology.orchestra.run.smithi123.stdout:checking for fgrep... /bin/grep -F 2024-04-24T05:49:34.509 INFO:teuthology.orchestra.run.smithi123.stdout:checking for ld used by gcc... /bin/ld 2024-04-24T05:49:34.514 INFO:teuthology.orchestra.run.smithi123.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2024-04-24T05:49:34.530 INFO:teuthology.orchestra.run.smithi123.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2024-04-24T05:49:35.582 INFO:teuthology.orchestra.run.smithi123.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2024-04-24T05:49:35.582 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether ln -s works... yes 2024-04-24T05:49:35.620 INFO:teuthology.orchestra.run.smithi123.stdout:checking the maximum length of command line arguments... 1572864 2024-04-24T05:49:35.631 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-24T05:49:35.631 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-24T05:49:35.631 INFO:teuthology.orchestra.run.smithi123.stdout:checking for /bin/ld option to reload object files... -r 2024-04-24T05:49:35.631 INFO:teuthology.orchestra.run.smithi123.stdout:checking for objdump... objdump 2024-04-24T05:49:35.632 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to recognize dependent libraries... pass_all 2024-04-24T05:49:35.632 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dlltool... no 2024-04-24T05:49:35.632 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-24T05:49:35.632 INFO:teuthology.orchestra.run.smithi123.stdout:checking for ar... ar 2024-04-24T05:49:35.727 INFO:teuthology.orchestra.run.smithi123.stdout:checking for archiver @FILE support... @ 2024-04-24T05:49:35.728 INFO:teuthology.orchestra.run.smithi123.stdout:checking for strip... strip 2024-04-24T05:49:35.729 INFO:teuthology.orchestra.run.smithi123.stdout:checking for ranlib... ranlib 2024-04-24T05:49:35.730 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gawk... gawk 2024-04-24T05:49:35.900 INFO:teuthology.orchestra.run.smithi123.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2024-04-24T05:49:35.903 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sysroot... no 2024-04-24T05:49:35.914 INFO:teuthology.orchestra.run.smithi123.stdout:checking for a working dd... /bin/dd 2024-04-24T05:49:35.924 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2024-04-24T05:49:35.963 INFO:teuthology.orchestra.run.smithi123.stdout:checking for mt... no 2024-04-24T05:49:35.971 INFO:teuthology.orchestra.run.smithi123.stdout:checking if : is a manifest tool... no 2024-04-24T05:49:36.074 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to run the C preprocessor... gcc -E 2024-04-24T05:49:36.423 INFO:teuthology.orchestra.run.smithi123.stdout:checking for ANSI C header files... yes 2024-04-24T05:49:36.483 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/types.h... yes 2024-04-24T05:49:36.557 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/stat.h... yes 2024-04-24T05:49:36.625 INFO:teuthology.orchestra.run.smithi123.stdout:checking for stdlib.h... yes 2024-04-24T05:49:36.695 INFO:teuthology.orchestra.run.smithi123.stdout:checking for string.h... yes 2024-04-24T05:49:36.768 INFO:teuthology.orchestra.run.smithi123.stdout:checking for memory.h... yes 2024-04-24T05:49:36.838 INFO:teuthology.orchestra.run.smithi123.stdout:checking for strings.h... yes 2024-04-24T05:49:36.906 INFO:teuthology.orchestra.run.smithi123.stdout:checking for inttypes.h... yes 2024-04-24T05:49:36.976 INFO:teuthology.orchestra.run.smithi123.stdout:checking for stdint.h... yes 2024-04-24T05:49:37.064 INFO:teuthology.orchestra.run.smithi123.stdout:checking for unistd.h... yes 2024-04-24T05:49:37.135 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dlfcn.h... yes 2024-04-24T05:49:37.146 INFO:teuthology.orchestra.run.smithi123.stdout:checking for objdir... .libs 2024-04-24T05:49:37.322 INFO:teuthology.orchestra.run.smithi123.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-24T05:49:37.322 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-24T05:49:37.373 INFO:teuthology.orchestra.run.smithi123.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-24T05:49:37.431 INFO:teuthology.orchestra.run.smithi123.stdout:checking if gcc static flag -static works... no 2024-04-24T05:49:37.507 INFO:teuthology.orchestra.run.smithi123.stdout:checking if gcc supports -c -o file.o... yes 2024-04-24T05:49:37.507 INFO:teuthology.orchestra.run.smithi123.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-24T05:49:37.528 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-24T05:49:37.593 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether -lc should be explicitly linked in... no 2024-04-24T05:49:37.733 INFO:teuthology.orchestra.run.smithi123.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-24T05:49:37.733 INFO:teuthology.orchestra.run.smithi123.stdout:checking how to hardcode library paths into programs... immediate 2024-04-24T05:49:37.737 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether stripping libraries is possible... yes 2024-04-24T05:49:37.737 INFO:teuthology.orchestra.run.smithi123.stdout:checking if libtool supports shared libraries... yes 2024-04-24T05:49:37.737 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether to build shared libraries... yes 2024-04-24T05:49:37.737 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether to build static libraries... yes 2024-04-24T05:49:37.756 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gcc... (cached) gcc 2024-04-24T05:49:37.808 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-24T05:49:37.808 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-24T05:49:37.809 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-24T05:49:37.809 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gmake... /usr/bin/gmake 2024-04-24T05:49:37.810 INFO:teuthology.orchestra.run.smithi123.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-24T05:49:37.810 INFO:teuthology.orchestra.run.smithi123.stdout:checking for tar... /bin/tar 2024-04-24T05:49:37.810 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gzip... /bin/gzip 2024-04-24T05:49:37.850 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether gcc -MM is supported... yes 2024-04-24T05:49:37.851 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sort... /bin/sort 2024-04-24T05:49:37.852 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether ln -s works... yes 2024-04-24T05:49:37.853 INFO:teuthology.orchestra.run.smithi123.stdout:checking for rpm... /bin/rpm 2024-04-24T05:49:37.865 INFO:teuthology.orchestra.run.smithi123.stdout:checking for rpmbuild... no 2024-04-24T05:49:37.939 INFO:teuthology.orchestra.run.smithi123.stdout:checking assert.h usability... yes 2024-04-24T05:49:37.965 INFO:teuthology.orchestra.run.smithi123.stdout:checking assert.h presence... yes 2024-04-24T05:49:37.965 INFO:teuthology.orchestra.run.smithi123.stdout:checking for assert.h... yes 2024-04-24T05:49:38.038 INFO:teuthology.orchestra.run.smithi123.stdout:checking bstring.h usability... no 2024-04-24T05:49:38.074 INFO:teuthology.orchestra.run.smithi123.stdout:checking bstring.h presence... no 2024-04-24T05:49:38.074 INFO:teuthology.orchestra.run.smithi123.stdout:checking for bstring.h... no 2024-04-24T05:49:38.141 INFO:teuthology.orchestra.run.smithi123.stdout:checking libgen.h usability... yes 2024-04-24T05:49:38.167 INFO:teuthology.orchestra.run.smithi123.stdout:checking libgen.h presence... yes 2024-04-24T05:49:38.167 INFO:teuthology.orchestra.run.smithi123.stdout:checking for libgen.h... yes 2024-04-24T05:49:38.246 INFO:teuthology.orchestra.run.smithi123.stdout:checking dirent.h usability... yes 2024-04-24T05:49:38.274 INFO:teuthology.orchestra.run.smithi123.stdout:checking dirent.h presence... yes 2024-04-24T05:49:38.274 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dirent.h... yes 2024-04-24T05:49:38.380 INFO:teuthology.orchestra.run.smithi123.stdout:checking errno.h usability... yes 2024-04-24T05:49:38.409 INFO:teuthology.orchestra.run.smithi123.stdout:checking errno.h presence... yes 2024-04-24T05:49:38.410 INFO:teuthology.orchestra.run.smithi123.stdout:checking for errno.h... yes 2024-04-24T05:49:38.485 INFO:teuthology.orchestra.run.smithi123.stdout:checking malloc.h usability... yes 2024-04-24T05:49:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:checking malloc.h presence... yes 2024-04-24T05:49:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:checking for malloc.h... yes 2024-04-24T05:49:38.590 INFO:teuthology.orchestra.run.smithi123.stdout:checking uuid.h usability... no 2024-04-24T05:49:38.631 INFO:teuthology.orchestra.run.smithi123.stdout:checking uuid.h presence... no 2024-04-24T05:49:38.632 INFO:teuthology.orchestra.run.smithi123.stdout:checking for uuid.h... no 2024-04-24T05:49:38.705 INFO:teuthology.orchestra.run.smithi123.stdout:checking uuid/uuid.h usability... yes 2024-04-24T05:49:38.737 INFO:teuthology.orchestra.run.smithi123.stdout:checking uuid/uuid.h presence... yes 2024-04-24T05:49:38.737 INFO:teuthology.orchestra.run.smithi123.stdout:checking for uuid/uuid.h... yes 2024-04-24T05:49:38.813 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/uuid.h usability... no 2024-04-24T05:49:38.850 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/uuid.h presence... no 2024-04-24T05:49:38.850 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/uuid.h... no 2024-04-24T05:49:38.918 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/file.h usability... yes 2024-04-24T05:49:38.948 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/file.h presence... yes 2024-04-24T05:49:38.948 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/file.h... yes 2024-04-24T05:49:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/fcntl.h usability... yes 2024-04-24T05:49:39.054 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/fcntl.h presence... yes 2024-04-24T05:49:39.055 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/fcntl.h... yes 2024-04-24T05:49:39.133 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/syssgi.h usability... no 2024-04-24T05:49:39.171 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/syssgi.h presence... no 2024-04-24T05:49:39.171 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/syssgi.h... no 2024-04-24T05:49:39.266 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/param.h usability... yes 2024-04-24T05:49:39.302 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/param.h presence... yes 2024-04-24T05:49:39.302 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/param.h... yes 2024-04-24T05:49:39.322 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/stat.h... (cached) yes 2024-04-24T05:49:39.401 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/statvfs.h usability... yes 2024-04-24T05:49:39.427 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/statvfs.h presence... yes 2024-04-24T05:49:39.428 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/statvfs.h... yes 2024-04-24T05:49:39.503 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/time.h usability... yes 2024-04-24T05:49:39.530 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/time.h presence... yes 2024-04-24T05:49:39.530 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/time.h... yes 2024-04-24T05:49:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/ioctl.h usability... yes 2024-04-24T05:49:39.643 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/ioctl.h presence... yes 2024-04-24T05:49:39.644 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/ioctl.h... yes 2024-04-24T05:49:39.728 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/wait.h usability... yes 2024-04-24T05:49:39.760 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/wait.h presence... yes 2024-04-24T05:49:39.760 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/wait.h... yes 2024-04-24T05:49:39.777 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/types.h... (cached) yes 2024-04-24T05:49:39.791 INFO:teuthology.orchestra.run.smithi123.stdout:checking for strings.h... (cached) yes 2024-04-24T05:49:39.861 INFO:teuthology.orchestra.run.smithi123.stdout:checking err.h usability... yes 2024-04-24T05:49:39.886 INFO:teuthology.orchestra.run.smithi123.stdout:checking err.h presence... yes 2024-04-24T05:49:39.886 INFO:teuthology.orchestra.run.smithi123.stdout:checking for err.h... yes 2024-04-24T05:49:39.970 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/falloc.h usability... yes 2024-04-24T05:49:39.995 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/falloc.h presence... yes 2024-04-24T05:49:39.995 INFO:teuthology.orchestra.run.smithi123.stdout:checking for linux/falloc.h... yes 2024-04-24T05:49:40.070 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-24T05:49:40.111 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-24T05:49:40.111 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-24T05:49:40.186 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-24T05:49:40.229 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-24T05:49:40.229 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-24T05:49:40.301 INFO:teuthology.orchestra.run.smithi123.stdout:checking xfs/platform_defs.h usability... no 2024-04-24T05:49:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:checking xfs/platform_defs.h presence... no 2024-04-24T05:49:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:checking for xfs/platform_defs.h... no 2024-04-24T05:49:40.406 INFO:teuthology.orchestra.run.smithi123.stdout:checking btrfs/ioctl.h usability... no 2024-04-24T05:49:40.443 INFO:teuthology.orchestra.run.smithi123.stdout:checking btrfs/ioctl.h presence... no 2024-04-24T05:49:40.443 INFO:teuthology.orchestra.run.smithi123.stdout:checking for btrfs/ioctl.h... no 2024-04-24T05:49:40.510 INFO:teuthology.orchestra.run.smithi123.stdout:checking cifs/ioctl.h usability... no 2024-04-24T05:49:40.546 INFO:teuthology.orchestra.run.smithi123.stdout:checking cifs/ioctl.h presence... no 2024-04-24T05:49:40.546 INFO:teuthology.orchestra.run.smithi123.stdout:checking for cifs/ioctl.h... no 2024-04-24T05:49:40.623 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/mman.h usability... yes 2024-04-24T05:49:40.649 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/mman.h presence... yes 2024-04-24T05:49:40.649 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/mman.h... yes 2024-04-24T05:49:40.724 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/ext4.h usability... no 2024-04-24T05:49:40.764 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/ext4.h presence... no 2024-04-24T05:49:40.764 INFO:teuthology.orchestra.run.smithi123.stdout:checking for linux/ext4.h... no 2024-04-24T05:49:40.811 INFO:teuthology.orchestra.run.smithi123.stdout:checking for xfs/xfs_log_format.h... no 2024-04-24T05:49:40.916 INFO:teuthology.orchestra.run.smithi123.stdout:checking for uuid_compare... no 2024-04-24T05:49:40.990 INFO:teuthology.orchestra.run.smithi123.stdout:checking for uuid_compare in -luuid... yes 2024-04-24T05:49:41.116 INFO:teuthology.orchestra.run.smithi123.stdout:checking for xfs/xfs.h... yes 2024-04-24T05:49:41.167 INFO:teuthology.orchestra.run.smithi123.stdout:checking for xfs/libxfs.h... no 2024-04-24T05:49:41.226 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-24T05:49:41.301 INFO:teuthology.orchestra.run.smithi123.stdout:checking for xfs/xqm.h... yes 2024-04-24T05:49:41.408 INFO:teuthology.orchestra.run.smithi123.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-24T05:49:41.471 INFO:teuthology.orchestra.run.smithi123.stdout:checking xfs/handle.h usability... yes 2024-04-24T05:49:41.497 INFO:teuthology.orchestra.run.smithi123.stdout:checking xfs/handle.h presence... yes 2024-04-24T05:49:41.497 INFO:teuthology.orchestra.run.smithi123.stdout:checking for xfs/handle.h... yes 2024-04-24T05:49:41.587 INFO:teuthology.orchestra.run.smithi123.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-24T05:49:41.658 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/acl.h usability... yes 2024-04-24T05:49:41.689 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/acl.h presence... yes 2024-04-24T05:49:41.689 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/acl.h... yes 2024-04-24T05:49:41.758 INFO:teuthology.orchestra.run.smithi123.stdout:checking acl/libacl.h usability... yes 2024-04-24T05:49:41.789 INFO:teuthology.orchestra.run.smithi123.stdout:checking acl/libacl.h presence... yes 2024-04-24T05:49:41.789 INFO:teuthology.orchestra.run.smithi123.stdout:checking for acl/libacl.h... yes 2024-04-24T05:49:41.872 INFO:teuthology.orchestra.run.smithi123.stdout:checking for acl_init in -lacl... yes 2024-04-24T05:49:41.938 INFO:teuthology.orchestra.run.smithi123.stdout:checking gdbm-ndbm.h usability... no 2024-04-24T05:49:41.979 INFO:teuthology.orchestra.run.smithi123.stdout:checking gdbm-ndbm.h presence... no 2024-04-24T05:49:41.979 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gdbm-ndbm.h... no 2024-04-24T05:49:42.043 INFO:teuthology.orchestra.run.smithi123.stdout:checking gdbm/ndbm.h usability... yes 2024-04-24T05:49:42.073 INFO:teuthology.orchestra.run.smithi123.stdout:checking gdbm/ndbm.h presence... yes 2024-04-24T05:49:42.073 INFO:teuthology.orchestra.run.smithi123.stdout:checking for gdbm/ndbm.h... yes 2024-04-24T05:49:42.163 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_open in -lgdbm... no 2024-04-24T05:49:42.256 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_fetch in -lgdbm... no 2024-04-24T05:49:42.345 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_store in -lgdbm... no 2024-04-24T05:49:42.434 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_close in -lgdbm... no 2024-04-24T05:49:42.533 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_open in -lgdbm_compat... yes 2024-04-24T05:49:42.613 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_fetch in -lgdbm_compat... yes 2024-04-24T05:49:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_store in -lgdbm_compat... yes 2024-04-24T05:49:42.777 INFO:teuthology.orchestra.run.smithi123.stdout:checking for dbm_close in -lgdbm_compat... yes 2024-04-24T05:49:42.832 INFO:teuthology.orchestra.run.smithi123.stdout:checking libaio.h usability... yes 2024-04-24T05:49:42.863 INFO:teuthology.orchestra.run.smithi123.stdout:checking libaio.h presence... yes 2024-04-24T05:49:42.863 INFO:teuthology.orchestra.run.smithi123.stdout:checking for libaio.h... yes 2024-04-24T05:49:42.927 INFO:teuthology.orchestra.run.smithi123.stdout:checking liburing.h usability... no 2024-04-24T05:49:42.968 INFO:teuthology.orchestra.run.smithi123.stdout:checking liburing.h presence... no 2024-04-24T05:49:42.968 INFO:teuthology.orchestra.run.smithi123.stdout:checking for liburing.h... no 2024-04-24T05:49:43.039 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/capability.h usability... yes 2024-04-24T05:49:43.070 INFO:teuthology.orchestra.run.smithi123.stdout:checking sys/capability.h presence... yes 2024-04-24T05:49:43.071 INFO:teuthology.orchestra.run.smithi123.stdout:checking for sys/capability.h... yes 2024-04-24T05:49:43.146 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/fiemap.h usability... yes 2024-04-24T05:49:43.172 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/fiemap.h presence... yes 2024-04-24T05:49:43.172 INFO:teuthology.orchestra.run.smithi123.stdout:checking for linux/fiemap.h... yes 2024-04-24T05:49:43.258 INFO:teuthology.orchestra.run.smithi123.stdout:checking for fallocate... yes 2024-04-24T05:49:43.345 INFO:teuthology.orchestra.run.smithi123.stdout:checking for open_by_handle_at... yes 2024-04-24T05:49:43.411 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/fs.h usability... yes 2024-04-24T05:49:43.440 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/fs.h presence... yes 2024-04-24T05:49:43.440 INFO:teuthology.orchestra.run.smithi123.stdout:checking for linux/fs.h... yes 2024-04-24T05:49:43.505 INFO:teuthology.orchestra.run.smithi123.stdout:checking btrfsutil.h usability... no 2024-04-24T05:49:43.549 INFO:teuthology.orchestra.run.smithi123.stdout:checking btrfsutil.h presence... no 2024-04-24T05:49:43.549 INFO:teuthology.orchestra.run.smithi123.stdout:checking for btrfsutil.h... no 2024-04-24T05:49:43.645 INFO:teuthology.orchestra.run.smithi123.stdout:checking for copy_file_range... yes 2024-04-24T05:49:43.733 INFO:teuthology.orchestra.run.smithi123.stdout:checking for SEEK_DATA... yes 2024-04-24T05:49:43.848 INFO:teuthology.orchestra.run.smithi123.stdout:checking for BMV_OF_SHARED... yes 2024-04-24T05:49:43.937 INFO:teuthology.orchestra.run.smithi123.stdout:checking for nftw... yes 2024-04-24T05:49:44.030 INFO:teuthology.orchestra.run.smithi123.stdout:checking for RLIMIT_NOFILE... yes 2024-04-24T05:49:44.109 INFO:teuthology.orchestra.run.smithi123.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-24T05:49:44.194 INFO:teuthology.orchestra.run.smithi123.stdout:checking for FICLONE... yes 2024-04-24T05:49:44.282 INFO:teuthology.orchestra.run.smithi123.stdout:checking for renameat2... yes 2024-04-24T05:49:44.371 INFO:teuthology.orchestra.run.smithi123.stdout:checking for reallocarray... yes 2024-04-24T05:49:44.467 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct mount_attr... yes 2024-04-24T05:49:44.571 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-24T05:49:44.669 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-24T05:49:44.772 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-24T05:49:44.878 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-24T05:49:44.987 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-24T05:49:45.096 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-24T05:49:45.198 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-24T05:49:45.266 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/btrfs.h usability... yes 2024-04-24T05:49:45.295 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/btrfs.h presence... yes 2024-04-24T05:49:45.295 INFO:teuthology.orchestra.run.smithi123.stdout:checking for linux/btrfs.h... yes 2024-04-24T05:49:45.380 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-24T05:49:45.406 INFO:teuthology.orchestra.run.smithi123.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-24T05:49:45.406 INFO:teuthology.orchestra.run.smithi123.stdout:checking for linux/btrfs_tree.h... yes 2024-04-24T05:49:45.468 INFO:teuthology.orchestra.run.smithi123.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-24T05:49:45.534 INFO:teuthology.orchestra.run.smithi123.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-24T05:49:45.572 INFO:teuthology.orchestra.run.smithi123.stdout:configure: creating ./config.status 2024-04-24T05:49:46.742 INFO:teuthology.orchestra.run.smithi123.stdout:config.status: creating include/builddefs 2024-04-24T05:49:46.783 INFO:teuthology.orchestra.run.smithi123.stdout:config.status: creating include/config.h 2024-04-24T05:49:46.819 INFO:teuthology.orchestra.run.smithi123.stdout:config.status: executing libtool commands 2024-04-24T05:49:46.931 INFO:teuthology.orchestra.run.smithi123.stdout:Building include 2024-04-24T05:49:46.941 INFO:teuthology.orchestra.run.smithi123.stdout:Building lib 2024-04-24T05:49:46.950 INFO:teuthology.orchestra.run.smithi123.stdout: [LTDEP] 2024-04-24T05:49:47.271 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dataascii.lo 2024-04-24T05:49:47.505 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] databin.lo 2024-04-24T05:49:47.804 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] datapid.lo 2024-04-24T05:49:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] file_lock.lo 2024-04-24T05:49:48.354 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] forker.lo 2024-04-24T05:49:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] pattern.lo 2024-04-24T05:49:48.865 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] open_flags.lo 2024-04-24T05:49:49.160 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] random_range.lo 2024-04-24T05:49:49.546 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] string_to_tokens.lo 2024-04-24T05:49:49.784 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] str_to_bytes.lo 2024-04-24T05:49:50.073 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] tlibio.lo 2024-04-24T05:49:50.444 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] write_log.lo 2024-04-24T05:49:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] random.lo 2024-04-24T05:49:51.077 INFO:teuthology.orchestra.run.smithi123.stdout: [LD] 2024-04-24T05:49:51.254 INFO:teuthology.orchestra.run.smithi123.stdout:Building ltp 2024-04-24T05:49:51.262 INFO:teuthology.orchestra.run.smithi123.stdout: [DEP] 2024-04-24T05:49:51.567 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] doio 2024-04-24T05:49:52.466 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fsstress 2024-04-24T05:49:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fsx 2024-04-24T05:49:54.487 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:49:54.495 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:49:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] growfiles 2024-04-24T05:49:55.687 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] iogen 2024-04-24T05:49:56.238 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-stress 2024-04-24T05:49:56.815 INFO:teuthology.orchestra.run.smithi123.stdout:Building src 2024-04-24T05:49:56.829 INFO:teuthology.orchestra.run.smithi123.stdout: [DEP] 2024-04-24T05:50:00.620 INFO:teuthology.orchestra.run.smithi123.stdout: [DEP] 2024-04-24T05:50:00.693 INFO:teuthology.orchestra.run.smithi123.stdout: [DEP] 2024-04-24T05:50:00.715 INFO:teuthology.orchestra.run.smithi123.stdout: [DEP] 2024-04-24T05:50:01.212 INFO:teuthology.orchestra.run.smithi123.stdout: [DEP] 2024-04-24T05:50:01.267 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dirstress 2024-04-24T05:50:01.693 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fill 2024-04-24T05:50:02.005 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fill2 2024-04-24T05:50:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] getpagesize 2024-04-24T05:50:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] holes 2024-04-24T05:50:03.056 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] lstat64 2024-04-24T05:50:03.419 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] nametest 2024-04-24T05:50:03.846 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] permname 2024-04-24T05:50:04.208 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] randholes 2024-04-24T05:50:04.644 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] runas 2024-04-24T05:50:04.980 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] truncfile 2024-04-24T05:50:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] usemem 2024-04-24T05:50:05.654 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] mmapcat 2024-04-24T05:50:05.946 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] append_reader 2024-04-24T05:50:06.243 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] append_writer 2024-04-24T05:50:06.541 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dirperf 2024-04-24T05:50:06.943 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] metaperf 2024-04-24T05:50:07.377 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] devzero 2024-04-24T05:50:07.730 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] feature 2024-04-24T05:50:08.100 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] alloc 2024-04-24T05:50:08.485 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fault 2024-04-24T05:50:08.830 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fstest 2024-04-24T05:50:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_access_root 2024-04-24T05:50:09.541 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] godown 2024-04-24T05:50:09.891 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] resvtest 2024-04-24T05:50:10.249 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] writemod 2024-04-24T05:50:10.556 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] writev_on_pagefault 2024-04-24T05:50:10.882 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] makeextents 2024-04-24T05:50:11.237 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] itrash 2024-04-24T05:50:11.539 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] rename 2024-04-24T05:50:11.866 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] multi_open_unlink 2024-04-24T05:50:12.199 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] unwritten_sync 2024-04-24T05:50:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] genhashnames 2024-04-24T05:50:12.924 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_holes 2024-04-24T05:50:13.228 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_writev 2024-04-24T05:50:13.513 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_truncate_cmtime 2024-04-24T05:50:13.839 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dirhash_collide 2024-04-24T05:50:14.194 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_rename_overwrite 2024-04-24T05:50:14.488 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] holetest 2024-04-24T05:50:14.899 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_truncate_self 2024-04-24T05:50:15.216 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] af_unix 2024-04-24T05:50:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_stale_pmd 2024-04-24T05:50:15.883 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_cow_race 2024-04-24T05:50:16.212 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_fallocate 2024-04-24T05:50:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fsync-err 2024-04-24T05:50:16.894 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_write_ro 2024-04-24T05:50:17.197 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-24T05:50:17.546 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-24T05:50:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_ofd_locks 2024-04-24T05:50:18.215 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_collision 2024-04-24T05:50:18.591 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] mmap-write-concurrent 2024-04-24T05:50:18.937 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_get_file_time 2024-04-24T05:50:19.283 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_create_short_dirs 2024-04-24T05:50:19.596 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_create_long_dirs 2024-04-24T05:50:19.941 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_enospc 2024-04-24T05:50:20.304 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_writev_overlap 2024-04-24T05:50:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] checkpoint_journal 2024-04-24T05:50:20.964 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] mmap-rw-fault 2024-04-24T05:50:21.289 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] allocstale 2024-04-24T05:50:21.606 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_cow_memory_failure 2024-04-24T05:50:21.978 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fake-dump-rootino 2024-04-24T05:50:22.288 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dio-buf-fault 2024-04-24T05:50:22.598 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] rewinddir-test 2024-04-24T05:50:22.906 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] readdir-while-renames 2024-04-24T05:50:23.239 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] xfsctl 2024-04-24T05:50:23.617 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] bstat 2024-04-24T05:50:23.999 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mtab 2024-04-24T05:50:24.343 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] getdevicesize 2024-04-24T05:50:24.616 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:50:24.625 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:50:24.668 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] preallo_rw_pattern_reader 2024-04-24T05:50:24.944 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] preallo_rw_pattern_writer 2024-04-24T05:50:25.252 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] ftrunc 2024-04-24T05:50:25.567 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] trunc 2024-04-24T05:50:25.900 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fs_perms 2024-04-24T05:50:26.232 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] testx 2024-04-24T05:50:26.533 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] looptest 2024-04-24T05:50:26.908 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] locktest 2024-04-24T05:50:27.394 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] unwritten_mmap 2024-04-24T05:50:27.737 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] bulkstat_unlink_test 2024-04-24T05:50:28.101 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] deduperace 2024-04-24T05:50:28.468 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] bulkstat_unlink_test_modified 2024-04-24T05:50:28.864 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_dir_offset 2024-04-24T05:50:29.176 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_futimens 2024-04-24T05:50:29.483 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_immutable 2024-04-24T05:50:30.323 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] stale_handle 2024-04-24T05:50:30.673 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] pwrite_mmap_blocked 2024-04-24T05:50:30.981 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_dir_offset2 2024-04-24T05:50:31.358 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] seek_sanity_test 2024-04-24T05:50:32.007 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] seek_copy_test 2024-04-24T05:50:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_readdir_1 2024-04-24T05:50:32.635 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_readdir_2 2024-04-24T05:50:32.940 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fsync-tester 2024-04-24T05:50:33.368 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] nsexec 2024-04-24T05:50:33.707 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] cloner 2024-04-24T05:50:34.025 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] renameat2 2024-04-24T05:50:34.347 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_getcwd 2024-04-24T05:50:34.661 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] e4compact 2024-04-24T05:50:35.014 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] test-nextquota 2024-04-24T05:50:35.363 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] punch-alternating 2024-04-24T05:50:35.690 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-24T05:50:36.052 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] listxattr 2024-04-24T05:50:36.349 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dio-interleaved 2024-04-24T05:50:36.674 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_dir_type 2024-04-24T05:50:36.991 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dio-invalidate-cache 2024-04-24T05:50:37.376 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] stat_test 2024-04-24T05:50:37.859 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_encrypted_d_revalidate 2024-04-24T05:50:38.203 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] attr_replace_test 2024-04-24T05:50:38.526 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] swapon 2024-04-24T05:50:38.839 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] mkswap 2024-04-24T05:50:39.165 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_attr_corruption 2024-04-24T05:50:39.487 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_open_tmpfiles 2024-04-24T05:50:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fscrypt-crypt-util 2024-04-24T05:50:40.637 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] bulkstat_null_ocount 2024-04-24T05:50:40.968 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] splice-test 2024-04-24T05:50:41.305 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] chprojid_fail 2024-04-24T05:50:41.640 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] detached_mounts_propagation 2024-04-24T05:50:41.990 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] ext4_resize 2024-04-24T05:50:42.274 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_readdir_3 2024-04-24T05:50:42.652 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] splice2pipe 2024-04-24T05:50:42.998 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] uuid_ioctl 2024-04-24T05:50:43.317 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-24T05:50:43.645 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fiemap-fault 2024-04-24T05:50:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fiemap-tester 2024-04-24T05:50:44.380 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_stripealign 2024-04-24T05:50:44.677 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] open_by_handle 2024-04-24T05:50:45.085 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] dbtest 2024-04-24T05:50:45.536 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_mmap_dio 2024-04-24T05:50:45.854 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] xfsfind 2024-04-24T05:50:46.214 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] t_reflink_read_race 2024-04-24T05:50:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] fssum 2024-04-24T05:50:47.190 INFO:teuthology.orchestra.run.smithi123.stdout:Building log-writes 2024-04-24T05:50:47.205 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] replay-log 2024-04-24T05:50:47.707 INFO:teuthology.orchestra.run.smithi123.stdout:Building perf 2024-04-24T05:50:47.718 INFO:teuthology.orchestra.run.smithi123.stdout:Building aio-dio-regress 2024-04-24T05:50:47.735 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aiocp 2024-04-24T05:50:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-24T05:50:48.445 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-append-write-read-race 2024-04-24T05:50:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-cow-race 2024-04-24T05:50:49.118 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-cycle-write 2024-04-24T05:50:49.453 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-eof-race 2024-04-24T05:50:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-extend-stat 2024-04-24T05:50:50.147 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-fcntl-race 2024-04-24T05:50:50.481 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-hole-filling-race 2024-04-24T05:50:50.787 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-invalidate-failure 2024-04-24T05:50:51.092 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-invalidate-readahead 2024-04-24T05:50:51.443 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aiodio_sparse2 2024-04-24T05:50:51.835 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-subblock-eof-read 2024-04-24T05:50:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-dio-write-verify 2024-04-24T05:50:52.496 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-24T05:50:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-24T05:50:53.070 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] aio-last-ref-held-by-io 2024-04-24T05:50:53.404 INFO:teuthology.orchestra.run.smithi123.stdout:Building vfs 2024-04-24T05:50:53.419 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] vfstest 2024-04-24T05:50:54.755 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:50:54.762 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:50:57.339 INFO:teuthology.orchestra.run.smithi123.stdout: [CC] mount-idmapped 2024-04-24T05:50:57.915 INFO:teuthology.orchestra.run.smithi123.stdout:Building m4 2024-04-24T05:50:57.926 INFO:teuthology.orchestra.run.smithi123.stdout:Building common 2024-04-24T05:50:57.936 INFO:teuthology.orchestra.run.smithi123.stdout:Building tests 2024-04-24T05:50:57.950 INFO:teuthology.orchestra.run.smithi123.stdout:Building btrfs 2024-04-24T05:50:57.965 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/btrfs/group.list 2024-04-24T05:50:58.187 INFO:teuthology.orchestra.run.smithi123.stdout:Building ceph 2024-04-24T05:50:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/ceph/group.list 2024-04-24T05:50:58.254 INFO:teuthology.orchestra.run.smithi123.stdout:Building cifs 2024-04-24T05:50:58.268 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/cifs/group.list 2024-04-24T05:50:58.314 INFO:teuthology.orchestra.run.smithi123.stdout:Building ext4 2024-04-24T05:50:58.329 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/ext4/group.list 2024-04-24T05:50:58.471 INFO:teuthology.orchestra.run.smithi123.stdout:Building f2fs 2024-04-24T05:50:58.485 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/f2fs/group.list 2024-04-24T05:50:58.551 INFO:teuthology.orchestra.run.smithi123.stdout:Building generic 2024-04-24T05:50:58.568 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/generic/group.list 2024-04-24T05:50:58.938 INFO:teuthology.orchestra.run.smithi123.stdout:Building nfs 2024-04-24T05:50:58.953 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/nfs/group.list 2024-04-24T05:50:59.002 INFO:teuthology.orchestra.run.smithi123.stdout:Building ocfs2 2024-04-24T05:50:59.015 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/ocfs2/group.list 2024-04-24T05:50:59.059 INFO:teuthology.orchestra.run.smithi123.stdout:Building overlay 2024-04-24T05:50:59.073 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/overlay/group.list 2024-04-24T05:50:59.219 INFO:teuthology.orchestra.run.smithi123.stdout:Building perf 2024-04-24T05:50:59.233 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/perf/group.list 2024-04-24T05:50:59.275 INFO:teuthology.orchestra.run.smithi123.stdout:Building selftest 2024-04-24T05:50:59.289 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/selftest/group.list 2024-04-24T05:50:59.336 INFO:teuthology.orchestra.run.smithi123.stdout:Building shared 2024-04-24T05:50:59.350 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/shared/group.list 2024-04-24T05:50:59.416 INFO:teuthology.orchestra.run.smithi123.stdout:Building tmpfs 2024-04-24T05:50:59.430 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/tmpfs/group.list 2024-04-24T05:50:59.479 INFO:teuthology.orchestra.run.smithi123.stdout:Building udf 2024-04-24T05:50:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/udf/group.list 2024-04-24T05:50:59.533 INFO:teuthology.orchestra.run.smithi123.stdout:Building xfs 2024-04-24T05:50:59.550 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/xfs/group.list 2024-04-24T05:50:59.888 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo make install) 2024-04-24T05:51:00.002 INFO:teuthology.orchestra.run.smithi123.stdout:Building include 2024-04-24T05:51:00.012 INFO:teuthology.orchestra.run.smithi123.stdout:Building lib 2024-04-24T05:51:00.021 INFO:teuthology.orchestra.run.smithi123.stdout:Building ltp 2024-04-24T05:51:00.036 INFO:teuthology.orchestra.run.smithi123.stdout:Building src 2024-04-24T05:51:00.114 INFO:teuthology.orchestra.run.smithi123.stdout:Building log-writes 2024-04-24T05:51:00.123 INFO:teuthology.orchestra.run.smithi123.stdout:Building perf 2024-04-24T05:51:00.132 INFO:teuthology.orchestra.run.smithi123.stdout:Building aio-dio-regress 2024-04-24T05:51:00.142 INFO:teuthology.orchestra.run.smithi123.stdout:Building vfs 2024-04-24T05:51:00.152 INFO:teuthology.orchestra.run.smithi123.stdout:Building m4 2024-04-24T05:51:00.163 INFO:teuthology.orchestra.run.smithi123.stdout:Building common 2024-04-24T05:51:00.173 INFO:teuthology.orchestra.run.smithi123.stdout:Building tests 2024-04-24T05:51:00.189 INFO:teuthology.orchestra.run.smithi123.stdout:Building btrfs 2024-04-24T05:51:00.205 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/btrfs/group.list 2024-04-24T05:51:00.441 INFO:teuthology.orchestra.run.smithi123.stdout:Building ceph 2024-04-24T05:51:00.456 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/ceph/group.list 2024-04-24T05:51:00.510 INFO:teuthology.orchestra.run.smithi123.stdout:Building cifs 2024-04-24T05:51:00.525 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/cifs/group.list 2024-04-24T05:51:00.569 INFO:teuthology.orchestra.run.smithi123.stdout:Building ext4 2024-04-24T05:51:00.581 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/ext4/group.list 2024-04-24T05:51:00.721 INFO:teuthology.orchestra.run.smithi123.stdout:Building f2fs 2024-04-24T05:51:00.735 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/f2fs/group.list 2024-04-24T05:51:00.799 INFO:teuthology.orchestra.run.smithi123.stdout:Building generic 2024-04-24T05:51:00.817 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/generic/group.list 2024-04-24T05:51:01.175 INFO:teuthology.orchestra.run.smithi123.stdout:Building nfs 2024-04-24T05:51:01.189 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/nfs/group.list 2024-04-24T05:51:01.242 INFO:teuthology.orchestra.run.smithi123.stdout:Building ocfs2 2024-04-24T05:51:01.256 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/ocfs2/group.list 2024-04-24T05:51:01.306 INFO:teuthology.orchestra.run.smithi123.stdout:Building overlay 2024-04-24T05:51:01.321 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/overlay/group.list 2024-04-24T05:51:01.467 INFO:teuthology.orchestra.run.smithi123.stdout:Building perf 2024-04-24T05:51:01.479 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/perf/group.list 2024-04-24T05:51:01.518 INFO:teuthology.orchestra.run.smithi123.stdout:Building selftest 2024-04-24T05:51:01.533 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/selftest/group.list 2024-04-24T05:51:01.579 INFO:teuthology.orchestra.run.smithi123.stdout:Building shared 2024-04-24T05:51:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/shared/group.list 2024-04-24T05:51:01.655 INFO:teuthology.orchestra.run.smithi123.stdout:Building tmpfs 2024-04-24T05:51:01.669 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/tmpfs/group.list 2024-04-24T05:51:01.717 INFO:teuthology.orchestra.run.smithi123.stdout:Building udf 2024-04-24T05:51:01.731 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/udf/group.list 2024-04-24T05:51:01.772 INFO:teuthology.orchestra.run.smithi123.stdout:Building xfs 2024-04-24T05:51:01.788 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/xfs/group.list 2024-04-24T05:51:02.143 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-04-24T05:51:02.148 INFO:teuthology.orchestra.run.smithi123.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-24T05:51:02.148 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-04-24T05:51:02.153 INFO:teuthology.orchestra.run.smithi123.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-24T05:51:02.153 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-04-24T05:51:02.158 INFO:teuthology.orchestra.run.smithi123.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-04-24T05:51:02.189 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-04-24T05:51:02.430 INFO:teuthology.orchestra.run.smithi123.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-04-24T05:51:02.463 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-04-24T05:51:02.507 INFO:teuthology.orchestra.run.smithi123.stdout:Building log-writes 2024-04-24T05:51:02.517 INFO:teuthology.orchestra.run.smithi123.stdout:Building perf 2024-04-24T05:51:02.529 INFO:teuthology.orchestra.run.smithi123.stdout:Building aio-dio-regress 2024-04-24T05:51:02.541 INFO:teuthology.orchestra.run.smithi123.stdout:Building vfs 2024-04-24T05:51:02.549 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake -C log-writes install 2024-04-24T05:51:02.553 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-04-24T05:51:02.585 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-04-24T05:51:02.617 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake -C perf install 2024-04-24T05:51:02.621 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-04-24T05:51:02.653 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-04-24T05:51:02.732 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-04-24T05:51:02.736 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-04-24T05:51:02.769 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-04-24T05:51:02.970 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake -C vfs install 2024-04-24T05:51:02.974 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-04-24T05:51:03.006 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-04-24T05:51:03.050 INFO:teuthology.orchestra.run.smithi123.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-04-24T05:51:03.082 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault fiemap-tester t_stripealign open_by_handle dbtest t_mmap_dio xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-04-24T05:51:07.495 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-04-24T05:51:07.833 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-04-24T05:51:07.895 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-04-24T05:51:07.898 INFO:teuthology.orchestra.run.smithi123.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-24T05:51:07.899 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-04-24T05:51:07.901 INFO:teuthology.orchestra.run.smithi123.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-04-24T05:51:07.932 INFO:teuthology.orchestra.run.smithi123.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-04-24T05:51:08.495 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-04-24T05:51:08.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/btrfs/ install 2024-04-24T05:51:08.506 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/btrfs/group.list 2024-04-24T05:51:08.742 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-04-24T05:51:08.775 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 320 330 /var/lib/xfstests/tests/btrfs 2024-04-24T05:51:12.286 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-04-24T05:51:12.319 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 320.out 330.out /var/lib/xfstests/tests/btrfs 2024-04-24T05:51:15.499 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/ceph/ install 2024-04-24T05:51:15.505 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/ceph/group.list 2024-04-24T05:51:15.550 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-04-24T05:51:15.578 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-04-24T05:51:15.639 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-04-24T05:51:15.665 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-04-24T05:51:15.723 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/cifs/ install 2024-04-24T05:51:15.728 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/cifs/group.list 2024-04-24T05:51:15.761 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-04-24T05:51:15.786 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-04-24T05:51:15.810 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-04-24T05:51:15.834 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-04-24T05:51:15.860 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/ext4/ install 2024-04-24T05:51:15.865 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/ext4/group.list 2024-04-24T05:51:15.973 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-04-24T05:51:15.996 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-04-24T05:51:16.697 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-04-24T05:51:16.732 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-04-24T05:51:17.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/f2fs/ install 2024-04-24T05:51:17.455 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/f2fs/group.list 2024-04-24T05:51:17.518 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-04-24T05:51:17.551 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-04-24T05:51:17.594 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-04-24T05:51:17.626 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-04-24T05:51:17.670 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/generic/ install 2024-04-24T05:51:17.678 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/generic/group.list 2024-04-24T05:51:18.024 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-04-24T05:51:18.057 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 741 742 743 /var/lib/xfstests/tests/generic 2024-04-24T05:51:24.886 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:51:24.891 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:51:25.711 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-04-24T05:51:25.744 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 741.out 742.out 743.out /var/lib/xfstests/tests/generic 2024-04-24T05:51:33.707 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/nfs/ install 2024-04-24T05:51:33.713 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/nfs/group.list 2024-04-24T05:51:33.762 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-04-24T05:51:33.794 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-04-24T05:51:33.827 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-04-24T05:51:33.858 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-04-24T05:51:33.890 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/ocfs2/ install 2024-04-24T05:51:33.897 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/ocfs2/group.list 2024-04-24T05:51:33.942 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-04-24T05:51:33.972 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-04-24T05:51:34.003 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-04-24T05:51:34.036 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-04-24T05:51:34.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/overlay/ install 2024-04-24T05:51:34.076 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/overlay/group.list 2024-04-24T05:51:34.217 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-04-24T05:51:34.247 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-04-24T05:51:35.470 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-04-24T05:51:35.503 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-04-24T05:51:36.658 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/perf/ install 2024-04-24T05:51:36.665 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/perf/group.list 2024-04-24T05:51:36.704 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-04-24T05:51:36.736 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-04-24T05:51:36.768 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-04-24T05:51:36.800 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-04-24T05:51:36.832 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/selftest/ install 2024-04-24T05:51:36.840 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/selftest/group.list 2024-04-24T05:51:36.883 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-04-24T05:51:36.913 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-04-24T05:51:36.992 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-04-24T05:51:37.022 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-04-24T05:51:37.099 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/shared/ install 2024-04-24T05:51:37.106 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/shared/group.list 2024-04-24T05:51:37.164 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-04-24T05:51:37.196 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-04-24T05:51:37.248 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-04-24T05:51:37.280 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-04-24T05:51:37.331 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/tmpfs/ install 2024-04-24T05:51:37.338 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/tmpfs/group.list 2024-04-24T05:51:37.382 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-04-24T05:51:37.413 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-04-24T05:51:37.445 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-04-24T05:51:37.477 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-04-24T05:51:37.509 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/udf/ install 2024-04-24T05:51:37.515 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/udf/group.list 2024-04-24T05:51:37.554 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-04-24T05:51:37.585 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-04-24T05:51:37.618 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-04-24T05:51:37.650 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-04-24T05:51:37.685 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/xfs/ install 2024-04-24T05:51:37.693 INFO:teuthology.orchestra.run.smithi123.stdout: [GROUP] /tmp/tmp.ejcLBa8fOwxfstests-dev/tests/xfs/group.list 2024-04-24T05:51:38.039 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-04-24T05:51:38.071 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-04-24T05:51:45.441 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-04-24T05:51:45.475 INFO:teuthology.orchestra.run.smithi123.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-04-24T05:51:53.365 INFO:teuthology.orchestra.run.smithi123.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-04-24T05:51:53.397 INFO:teuthology.orchestra.run.smithi123.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-04-24T05:51:53.430 INFO:teuthology.orchestra.run.smithi123.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-04-24T05:51:53.461 INFO:teuthology.orchestra.run.smithi123.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-04-24T05:51:53.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T05:51:53.499 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec ls tests/generic) 2024-04-24T05:51:53.551 INFO:teuthology.orchestra.run.smithi123.stdout:001 2024-04-24T05:51:53.552 INFO:teuthology.orchestra.run.smithi123.stdout:001.out 2024-04-24T05:51:53.552 INFO:teuthology.orchestra.run.smithi123.stdout:002 2024-04-24T05:51:53.552 INFO:teuthology.orchestra.run.smithi123.stdout:002.out 2024-04-24T05:51:53.552 INFO:teuthology.orchestra.run.smithi123.stdout:003 2024-04-24T05:51:53.552 INFO:teuthology.orchestra.run.smithi123.stdout:003.out 2024-04-24T05:51:53.552 INFO:teuthology.orchestra.run.smithi123.stdout:004 2024-04-24T05:51:53.552 INFO:teuthology.orchestra.run.smithi123.stdout:004.out 2024-04-24T05:51:53.552 INFO:teuthology.orchestra.run.smithi123.stdout:005 2024-04-24T05:51:53.552 INFO:teuthology.orchestra.run.smithi123.stdout:005.out 2024-04-24T05:51:53.552 INFO:teuthology.orchestra.run.smithi123.stdout:006 2024-04-24T05:51:53.552 INFO:teuthology.orchestra.run.smithi123.stdout:006.out 2024-04-24T05:51:53.552 INFO:teuthology.orchestra.run.smithi123.stdout:007 2024-04-24T05:51:53.552 INFO:teuthology.orchestra.run.smithi123.stdout:007.out 2024-04-24T05:51:53.553 INFO:teuthology.orchestra.run.smithi123.stdout:008 2024-04-24T05:51:53.553 INFO:teuthology.orchestra.run.smithi123.stdout:008.out 2024-04-24T05:51:53.553 INFO:teuthology.orchestra.run.smithi123.stdout:009 2024-04-24T05:51:53.553 INFO:teuthology.orchestra.run.smithi123.stdout:009.out 2024-04-24T05:51:53.553 INFO:teuthology.orchestra.run.smithi123.stdout:010 2024-04-24T05:51:53.553 INFO:teuthology.orchestra.run.smithi123.stdout:010.out 2024-04-24T05:51:53.553 INFO:teuthology.orchestra.run.smithi123.stdout:011 2024-04-24T05:51:53.553 INFO:teuthology.orchestra.run.smithi123.stdout:011.out 2024-04-24T05:51:53.553 INFO:teuthology.orchestra.run.smithi123.stdout:012 2024-04-24T05:51:53.553 INFO:teuthology.orchestra.run.smithi123.stdout:012.out 2024-04-24T05:51:53.553 INFO:teuthology.orchestra.run.smithi123.stdout:013 2024-04-24T05:51:53.553 INFO:teuthology.orchestra.run.smithi123.stdout:013.out 2024-04-24T05:51:53.553 INFO:teuthology.orchestra.run.smithi123.stdout:014 2024-04-24T05:51:53.554 INFO:teuthology.orchestra.run.smithi123.stdout:014.out 2024-04-24T05:51:53.554 INFO:teuthology.orchestra.run.smithi123.stdout:015 2024-04-24T05:51:53.554 INFO:teuthology.orchestra.run.smithi123.stdout:015.out 2024-04-24T05:51:53.554 INFO:teuthology.orchestra.run.smithi123.stdout:016 2024-04-24T05:51:53.554 INFO:teuthology.orchestra.run.smithi123.stdout:016.out 2024-04-24T05:51:53.554 INFO:teuthology.orchestra.run.smithi123.stdout:017 2024-04-24T05:51:53.554 INFO:teuthology.orchestra.run.smithi123.stdout:017.out 2024-04-24T05:51:53.554 INFO:teuthology.orchestra.run.smithi123.stdout:018 2024-04-24T05:51:53.554 INFO:teuthology.orchestra.run.smithi123.stdout:018.out 2024-04-24T05:51:53.554 INFO:teuthology.orchestra.run.smithi123.stdout:019 2024-04-24T05:51:53.554 INFO:teuthology.orchestra.run.smithi123.stdout:019.out 2024-04-24T05:51:53.554 INFO:teuthology.orchestra.run.smithi123.stdout:020 2024-04-24T05:51:53.554 INFO:teuthology.orchestra.run.smithi123.stdout:020.out 2024-04-24T05:51:53.555 INFO:teuthology.orchestra.run.smithi123.stdout:021 2024-04-24T05:51:53.555 INFO:teuthology.orchestra.run.smithi123.stdout:021.out 2024-04-24T05:51:53.555 INFO:teuthology.orchestra.run.smithi123.stdout:022 2024-04-24T05:51:53.555 INFO:teuthology.orchestra.run.smithi123.stdout:022.out 2024-04-24T05:51:53.555 INFO:teuthology.orchestra.run.smithi123.stdout:023 2024-04-24T05:51:53.555 INFO:teuthology.orchestra.run.smithi123.stdout:023.out 2024-04-24T05:51:53.555 INFO:teuthology.orchestra.run.smithi123.stdout:024 2024-04-24T05:51:53.555 INFO:teuthology.orchestra.run.smithi123.stdout:024.out 2024-04-24T05:51:53.555 INFO:teuthology.orchestra.run.smithi123.stdout:025 2024-04-24T05:51:53.555 INFO:teuthology.orchestra.run.smithi123.stdout:025.out 2024-04-24T05:51:53.555 INFO:teuthology.orchestra.run.smithi123.stdout:026 2024-04-24T05:51:53.555 INFO:teuthology.orchestra.run.smithi123.stdout:026.out 2024-04-24T05:51:53.555 INFO:teuthology.orchestra.run.smithi123.stdout:027 2024-04-24T05:51:53.555 INFO:teuthology.orchestra.run.smithi123.stdout:027.out 2024-04-24T05:51:53.556 INFO:teuthology.orchestra.run.smithi123.stdout:028 2024-04-24T05:51:53.556 INFO:teuthology.orchestra.run.smithi123.stdout:028.out 2024-04-24T05:51:53.556 INFO:teuthology.orchestra.run.smithi123.stdout:029 2024-04-24T05:51:53.556 INFO:teuthology.orchestra.run.smithi123.stdout:029.out 2024-04-24T05:51:53.556 INFO:teuthology.orchestra.run.smithi123.stdout:030 2024-04-24T05:51:53.556 INFO:teuthology.orchestra.run.smithi123.stdout:030.out 2024-04-24T05:51:53.556 INFO:teuthology.orchestra.run.smithi123.stdout:031 2024-04-24T05:51:53.556 INFO:teuthology.orchestra.run.smithi123.stdout:031.out 2024-04-24T05:51:53.556 INFO:teuthology.orchestra.run.smithi123.stdout:032 2024-04-24T05:51:53.556 INFO:teuthology.orchestra.run.smithi123.stdout:032.out 2024-04-24T05:51:53.556 INFO:teuthology.orchestra.run.smithi123.stdout:033 2024-04-24T05:51:53.556 INFO:teuthology.orchestra.run.smithi123.stdout:033.out 2024-04-24T05:51:53.556 INFO:teuthology.orchestra.run.smithi123.stdout:034 2024-04-24T05:51:53.556 INFO:teuthology.orchestra.run.smithi123.stdout:034.out 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:035 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:035.cfg 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:035.out.default 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:035.out.nfs 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:036 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:036.out 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:037 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:037.out 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:038 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:038.out 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:039 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:039.out 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:040 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:040.out 2024-04-24T05:51:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:041 2024-04-24T05:51:53.558 INFO:teuthology.orchestra.run.smithi123.stdout:041.out 2024-04-24T05:51:53.558 INFO:teuthology.orchestra.run.smithi123.stdout:042 2024-04-24T05:51:53.558 INFO:teuthology.orchestra.run.smithi123.stdout:042.out 2024-04-24T05:51:53.558 INFO:teuthology.orchestra.run.smithi123.stdout:043 2024-04-24T05:51:53.558 INFO:teuthology.orchestra.run.smithi123.stdout:043.out 2024-04-24T05:51:53.558 INFO:teuthology.orchestra.run.smithi123.stdout:044 2024-04-24T05:51:53.558 INFO:teuthology.orchestra.run.smithi123.stdout:044.out 2024-04-24T05:51:53.558 INFO:teuthology.orchestra.run.smithi123.stdout:045 2024-04-24T05:51:53.558 INFO:teuthology.orchestra.run.smithi123.stdout:045.out 2024-04-24T05:51:53.558 INFO:teuthology.orchestra.run.smithi123.stdout:046 2024-04-24T05:51:53.558 INFO:teuthology.orchestra.run.smithi123.stdout:046.out 2024-04-24T05:51:53.558 INFO:teuthology.orchestra.run.smithi123.stdout:047 2024-04-24T05:51:53.558 INFO:teuthology.orchestra.run.smithi123.stdout:047.out 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:048 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:048.out 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:049 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:049.out 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:050 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:050.cfg 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:050.out.default 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:050.out.nojournal 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:050.out.xfsquota 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:051 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:051.out 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:052 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:052.out 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:053 2024-04-24T05:51:53.559 INFO:teuthology.orchestra.run.smithi123.stdout:053.out 2024-04-24T05:51:53.560 INFO:teuthology.orchestra.run.smithi123.stdout:054 2024-04-24T05:51:53.560 INFO:teuthology.orchestra.run.smithi123.stdout:054.out 2024-04-24T05:51:53.560 INFO:teuthology.orchestra.run.smithi123.stdout:055 2024-04-24T05:51:53.560 INFO:teuthology.orchestra.run.smithi123.stdout:055.out 2024-04-24T05:51:53.560 INFO:teuthology.orchestra.run.smithi123.stdout:056 2024-04-24T05:51:53.560 INFO:teuthology.orchestra.run.smithi123.stdout:056.out 2024-04-24T05:51:53.560 INFO:teuthology.orchestra.run.smithi123.stdout:057 2024-04-24T05:51:53.560 INFO:teuthology.orchestra.run.smithi123.stdout:057.out 2024-04-24T05:51:53.560 INFO:teuthology.orchestra.run.smithi123.stdout:058 2024-04-24T05:51:53.560 INFO:teuthology.orchestra.run.smithi123.stdout:058.out 2024-04-24T05:51:53.560 INFO:teuthology.orchestra.run.smithi123.stdout:059 2024-04-24T05:51:53.560 INFO:teuthology.orchestra.run.smithi123.stdout:059.out 2024-04-24T05:51:53.560 INFO:teuthology.orchestra.run.smithi123.stdout:060 2024-04-24T05:51:53.561 INFO:teuthology.orchestra.run.smithi123.stdout:060.out 2024-04-24T05:51:53.561 INFO:teuthology.orchestra.run.smithi123.stdout:061 2024-04-24T05:51:53.561 INFO:teuthology.orchestra.run.smithi123.stdout:061.out 2024-04-24T05:51:53.561 INFO:teuthology.orchestra.run.smithi123.stdout:062 2024-04-24T05:51:53.561 INFO:teuthology.orchestra.run.smithi123.stdout:062.out 2024-04-24T05:51:53.561 INFO:teuthology.orchestra.run.smithi123.stdout:063 2024-04-24T05:51:53.561 INFO:teuthology.orchestra.run.smithi123.stdout:063.out 2024-04-24T05:51:53.561 INFO:teuthology.orchestra.run.smithi123.stdout:064 2024-04-24T05:51:53.561 INFO:teuthology.orchestra.run.smithi123.stdout:064.out 2024-04-24T05:51:53.561 INFO:teuthology.orchestra.run.smithi123.stdout:065 2024-04-24T05:51:53.561 INFO:teuthology.orchestra.run.smithi123.stdout:065.out 2024-04-24T05:51:53.561 INFO:teuthology.orchestra.run.smithi123.stdout:066 2024-04-24T05:51:53.561 INFO:teuthology.orchestra.run.smithi123.stdout:066.out 2024-04-24T05:51:53.561 INFO:teuthology.orchestra.run.smithi123.stdout:067 2024-04-24T05:51:53.562 INFO:teuthology.orchestra.run.smithi123.stdout:067.out 2024-04-24T05:51:53.562 INFO:teuthology.orchestra.run.smithi123.stdout:068 2024-04-24T05:51:53.562 INFO:teuthology.orchestra.run.smithi123.stdout:068.out 2024-04-24T05:51:53.562 INFO:teuthology.orchestra.run.smithi123.stdout:069 2024-04-24T05:51:53.562 INFO:teuthology.orchestra.run.smithi123.stdout:069.out 2024-04-24T05:51:53.562 INFO:teuthology.orchestra.run.smithi123.stdout:070 2024-04-24T05:51:53.562 INFO:teuthology.orchestra.run.smithi123.stdout:070.out 2024-04-24T05:51:53.562 INFO:teuthology.orchestra.run.smithi123.stdout:071 2024-04-24T05:51:53.562 INFO:teuthology.orchestra.run.smithi123.stdout:071.out 2024-04-24T05:51:53.562 INFO:teuthology.orchestra.run.smithi123.stdout:072 2024-04-24T05:51:53.562 INFO:teuthology.orchestra.run.smithi123.stdout:072.out 2024-04-24T05:51:53.562 INFO:teuthology.orchestra.run.smithi123.stdout:073 2024-04-24T05:51:53.562 INFO:teuthology.orchestra.run.smithi123.stdout:073.out 2024-04-24T05:51:53.562 INFO:teuthology.orchestra.run.smithi123.stdout:074 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:074.out 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:075 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:075.out 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:076 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:076.out 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:077 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:077.out 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:078 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:078.out 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:079 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:079.out 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:080 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:080.out 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:081 2024-04-24T05:51:53.563 INFO:teuthology.orchestra.run.smithi123.stdout:081.out 2024-04-24T05:51:53.564 INFO:teuthology.orchestra.run.smithi123.stdout:082 2024-04-24T05:51:53.564 INFO:teuthology.orchestra.run.smithi123.stdout:082.out 2024-04-24T05:51:53.564 INFO:teuthology.orchestra.run.smithi123.stdout:083 2024-04-24T05:51:53.564 INFO:teuthology.orchestra.run.smithi123.stdout:083.out 2024-04-24T05:51:53.564 INFO:teuthology.orchestra.run.smithi123.stdout:084 2024-04-24T05:51:53.564 INFO:teuthology.orchestra.run.smithi123.stdout:084.out 2024-04-24T05:51:53.564 INFO:teuthology.orchestra.run.smithi123.stdout:085 2024-04-24T05:51:53.564 INFO:teuthology.orchestra.run.smithi123.stdout:085.out 2024-04-24T05:51:53.564 INFO:teuthology.orchestra.run.smithi123.stdout:086 2024-04-24T05:51:53.564 INFO:teuthology.orchestra.run.smithi123.stdout:086.out 2024-04-24T05:51:53.564 INFO:teuthology.orchestra.run.smithi123.stdout:087 2024-04-24T05:51:53.564 INFO:teuthology.orchestra.run.smithi123.stdout:087.out 2024-04-24T05:51:53.564 INFO:teuthology.orchestra.run.smithi123.stdout:088 2024-04-24T05:51:53.564 INFO:teuthology.orchestra.run.smithi123.stdout:088.out 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:089 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:089.out 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:090 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:090.out 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:091 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:091.out 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:092 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:092.out 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:093 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:093.out 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:094 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:094.out 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:095 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:095.out 2024-04-24T05:51:53.565 INFO:teuthology.orchestra.run.smithi123.stdout:096 2024-04-24T05:51:53.566 INFO:teuthology.orchestra.run.smithi123.stdout:096.out 2024-04-24T05:51:53.566 INFO:teuthology.orchestra.run.smithi123.stdout:097 2024-04-24T05:51:53.566 INFO:teuthology.orchestra.run.smithi123.stdout:097.out 2024-04-24T05:51:53.566 INFO:teuthology.orchestra.run.smithi123.stdout:098 2024-04-24T05:51:53.566 INFO:teuthology.orchestra.run.smithi123.stdout:098.out 2024-04-24T05:51:53.566 INFO:teuthology.orchestra.run.smithi123.stdout:099 2024-04-24T05:51:53.566 INFO:teuthology.orchestra.run.smithi123.stdout:099.out 2024-04-24T05:51:53.566 INFO:teuthology.orchestra.run.smithi123.stdout:100 2024-04-24T05:51:53.566 INFO:teuthology.orchestra.run.smithi123.stdout:100.out 2024-04-24T05:51:53.566 INFO:teuthology.orchestra.run.smithi123.stdout:101 2024-04-24T05:51:53.566 INFO:teuthology.orchestra.run.smithi123.stdout:101.out 2024-04-24T05:51:53.566 INFO:teuthology.orchestra.run.smithi123.stdout:102 2024-04-24T05:51:53.566 INFO:teuthology.orchestra.run.smithi123.stdout:102.out 2024-04-24T05:51:53.567 INFO:teuthology.orchestra.run.smithi123.stdout:103 2024-04-24T05:51:53.567 INFO:teuthology.orchestra.run.smithi123.stdout:103.out 2024-04-24T05:51:53.567 INFO:teuthology.orchestra.run.smithi123.stdout:104 2024-04-24T05:51:53.567 INFO:teuthology.orchestra.run.smithi123.stdout:104.out 2024-04-24T05:51:53.567 INFO:teuthology.orchestra.run.smithi123.stdout:105 2024-04-24T05:51:53.567 INFO:teuthology.orchestra.run.smithi123.stdout:105.out 2024-04-24T05:51:53.567 INFO:teuthology.orchestra.run.smithi123.stdout:106 2024-04-24T05:51:53.567 INFO:teuthology.orchestra.run.smithi123.stdout:106.out 2024-04-24T05:51:53.567 INFO:teuthology.orchestra.run.smithi123.stdout:107 2024-04-24T05:51:53.567 INFO:teuthology.orchestra.run.smithi123.stdout:107.out 2024-04-24T05:51:53.567 INFO:teuthology.orchestra.run.smithi123.stdout:108 2024-04-24T05:51:53.567 INFO:teuthology.orchestra.run.smithi123.stdout:108.out 2024-04-24T05:51:53.567 INFO:teuthology.orchestra.run.smithi123.stdout:109 2024-04-24T05:51:53.567 INFO:teuthology.orchestra.run.smithi123.stdout:109.out 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:110 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:110.out 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:111 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:111.out 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:112 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:112.out 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:113 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:113.out 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:114 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:114.out 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:115 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:115.out 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:116 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:116.out 2024-04-24T05:51:53.568 INFO:teuthology.orchestra.run.smithi123.stdout:117 2024-04-24T05:51:53.569 INFO:teuthology.orchestra.run.smithi123.stdout:117.out 2024-04-24T05:51:53.569 INFO:teuthology.orchestra.run.smithi123.stdout:118 2024-04-24T05:51:53.569 INFO:teuthology.orchestra.run.smithi123.stdout:118.out 2024-04-24T05:51:53.569 INFO:teuthology.orchestra.run.smithi123.stdout:119 2024-04-24T05:51:53.569 INFO:teuthology.orchestra.run.smithi123.stdout:119.out 2024-04-24T05:51:53.569 INFO:teuthology.orchestra.run.smithi123.stdout:120 2024-04-24T05:51:53.569 INFO:teuthology.orchestra.run.smithi123.stdout:120.out 2024-04-24T05:51:53.569 INFO:teuthology.orchestra.run.smithi123.stdout:121 2024-04-24T05:51:53.569 INFO:teuthology.orchestra.run.smithi123.stdout:121.out 2024-04-24T05:51:53.569 INFO:teuthology.orchestra.run.smithi123.stdout:122 2024-04-24T05:51:53.569 INFO:teuthology.orchestra.run.smithi123.stdout:122.out 2024-04-24T05:51:53.569 INFO:teuthology.orchestra.run.smithi123.stdout:123 2024-04-24T05:51:53.569 INFO:teuthology.orchestra.run.smithi123.stdout:123.out 2024-04-24T05:51:53.569 INFO:teuthology.orchestra.run.smithi123.stdout:124 2024-04-24T05:51:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:124.out 2024-04-24T05:51:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:125 2024-04-24T05:51:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:125.out 2024-04-24T05:51:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:126 2024-04-24T05:51:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:126.out 2024-04-24T05:51:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:127 2024-04-24T05:51:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:127.out 2024-04-24T05:51:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:128 2024-04-24T05:51:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:128.out 2024-04-24T05:51:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:129 2024-04-24T05:51:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:129.out 2024-04-24T05:51:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:130 2024-04-24T05:51:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:130.out 2024-04-24T05:51:53.570 INFO:teuthology.orchestra.run.smithi123.stdout:131 2024-04-24T05:51:53.571 INFO:teuthology.orchestra.run.smithi123.stdout:131.out 2024-04-24T05:51:53.571 INFO:teuthology.orchestra.run.smithi123.stdout:132 2024-04-24T05:51:53.571 INFO:teuthology.orchestra.run.smithi123.stdout:132.out 2024-04-24T05:51:53.571 INFO:teuthology.orchestra.run.smithi123.stdout:133 2024-04-24T05:51:53.571 INFO:teuthology.orchestra.run.smithi123.stdout:133.out 2024-04-24T05:51:53.571 INFO:teuthology.orchestra.run.smithi123.stdout:134 2024-04-24T05:51:53.571 INFO:teuthology.orchestra.run.smithi123.stdout:134.out 2024-04-24T05:51:53.571 INFO:teuthology.orchestra.run.smithi123.stdout:135 2024-04-24T05:51:53.571 INFO:teuthology.orchestra.run.smithi123.stdout:135.out 2024-04-24T05:51:53.571 INFO:teuthology.orchestra.run.smithi123.stdout:136 2024-04-24T05:51:53.571 INFO:teuthology.orchestra.run.smithi123.stdout:136.out 2024-04-24T05:51:53.571 INFO:teuthology.orchestra.run.smithi123.stdout:137 2024-04-24T05:51:53.571 INFO:teuthology.orchestra.run.smithi123.stdout:137.out 2024-04-24T05:51:53.571 INFO:teuthology.orchestra.run.smithi123.stdout:138 2024-04-24T05:51:53.572 INFO:teuthology.orchestra.run.smithi123.stdout:138.out 2024-04-24T05:51:53.573 INFO:teuthology.orchestra.run.smithi123.stdout:139 2024-04-24T05:51:53.573 INFO:teuthology.orchestra.run.smithi123.stdout:139.out 2024-04-24T05:51:53.573 INFO:teuthology.orchestra.run.smithi123.stdout:140 2024-04-24T05:51:53.573 INFO:teuthology.orchestra.run.smithi123.stdout:140.out 2024-04-24T05:51:53.574 INFO:teuthology.orchestra.run.smithi123.stdout:141 2024-04-24T05:51:53.574 INFO:teuthology.orchestra.run.smithi123.stdout:141.out 2024-04-24T05:51:53.574 INFO:teuthology.orchestra.run.smithi123.stdout:142 2024-04-24T05:51:53.574 INFO:teuthology.orchestra.run.smithi123.stdout:142.out 2024-04-24T05:51:53.574 INFO:teuthology.orchestra.run.smithi123.stdout:143 2024-04-24T05:51:53.574 INFO:teuthology.orchestra.run.smithi123.stdout:143.out 2024-04-24T05:51:53.574 INFO:teuthology.orchestra.run.smithi123.stdout:144 2024-04-24T05:51:53.574 INFO:teuthology.orchestra.run.smithi123.stdout:144.out 2024-04-24T05:51:53.574 INFO:teuthology.orchestra.run.smithi123.stdout:145 2024-04-24T05:51:53.574 INFO:teuthology.orchestra.run.smithi123.stdout:145.out 2024-04-24T05:51:53.575 INFO:teuthology.orchestra.run.smithi123.stdout:146 2024-04-24T05:51:53.575 INFO:teuthology.orchestra.run.smithi123.stdout:146.out 2024-04-24T05:51:53.575 INFO:teuthology.orchestra.run.smithi123.stdout:147 2024-04-24T05:51:53.575 INFO:teuthology.orchestra.run.smithi123.stdout:147.out 2024-04-24T05:51:53.575 INFO:teuthology.orchestra.run.smithi123.stdout:148 2024-04-24T05:51:53.575 INFO:teuthology.orchestra.run.smithi123.stdout:148.out 2024-04-24T05:51:53.575 INFO:teuthology.orchestra.run.smithi123.stdout:149 2024-04-24T05:51:53.575 INFO:teuthology.orchestra.run.smithi123.stdout:149.out 2024-04-24T05:51:53.575 INFO:teuthology.orchestra.run.smithi123.stdout:150 2024-04-24T05:51:53.575 INFO:teuthology.orchestra.run.smithi123.stdout:150.out 2024-04-24T05:51:53.575 INFO:teuthology.orchestra.run.smithi123.stdout:151 2024-04-24T05:51:53.576 INFO:teuthology.orchestra.run.smithi123.stdout:151.out 2024-04-24T05:51:53.576 INFO:teuthology.orchestra.run.smithi123.stdout:152 2024-04-24T05:51:53.576 INFO:teuthology.orchestra.run.smithi123.stdout:152.out 2024-04-24T05:51:53.576 INFO:teuthology.orchestra.run.smithi123.stdout:153 2024-04-24T05:51:53.576 INFO:teuthology.orchestra.run.smithi123.stdout:153.out 2024-04-24T05:51:53.576 INFO:teuthology.orchestra.run.smithi123.stdout:154 2024-04-24T05:51:53.576 INFO:teuthology.orchestra.run.smithi123.stdout:154.out 2024-04-24T05:51:53.576 INFO:teuthology.orchestra.run.smithi123.stdout:155 2024-04-24T05:51:53.576 INFO:teuthology.orchestra.run.smithi123.stdout:155.out 2024-04-24T05:51:53.576 INFO:teuthology.orchestra.run.smithi123.stdout:156 2024-04-24T05:51:53.576 INFO:teuthology.orchestra.run.smithi123.stdout:156.out 2024-04-24T05:51:53.577 INFO:teuthology.orchestra.run.smithi123.stdout:157 2024-04-24T05:51:53.577 INFO:teuthology.orchestra.run.smithi123.stdout:157.out 2024-04-24T05:51:53.577 INFO:teuthology.orchestra.run.smithi123.stdout:158 2024-04-24T05:51:53.577 INFO:teuthology.orchestra.run.smithi123.stdout:158.out 2024-04-24T05:51:53.577 INFO:teuthology.orchestra.run.smithi123.stdout:159 2024-04-24T05:51:53.577 INFO:teuthology.orchestra.run.smithi123.stdout:159.out 2024-04-24T05:51:53.577 INFO:teuthology.orchestra.run.smithi123.stdout:160 2024-04-24T05:51:53.577 INFO:teuthology.orchestra.run.smithi123.stdout:160.out 2024-04-24T05:51:53.577 INFO:teuthology.orchestra.run.smithi123.stdout:161 2024-04-24T05:51:53.577 INFO:teuthology.orchestra.run.smithi123.stdout:161.out 2024-04-24T05:51:53.577 INFO:teuthology.orchestra.run.smithi123.stdout:162 2024-04-24T05:51:53.578 INFO:teuthology.orchestra.run.smithi123.stdout:162.out 2024-04-24T05:51:53.578 INFO:teuthology.orchestra.run.smithi123.stdout:163 2024-04-24T05:51:53.578 INFO:teuthology.orchestra.run.smithi123.stdout:163.out 2024-04-24T05:51:53.578 INFO:teuthology.orchestra.run.smithi123.stdout:164 2024-04-24T05:51:53.578 INFO:teuthology.orchestra.run.smithi123.stdout:164.out 2024-04-24T05:51:53.578 INFO:teuthology.orchestra.run.smithi123.stdout:165 2024-04-24T05:51:53.578 INFO:teuthology.orchestra.run.smithi123.stdout:165.out 2024-04-24T05:51:53.578 INFO:teuthology.orchestra.run.smithi123.stdout:166 2024-04-24T05:51:53.578 INFO:teuthology.orchestra.run.smithi123.stdout:166.out 2024-04-24T05:51:53.578 INFO:teuthology.orchestra.run.smithi123.stdout:167 2024-04-24T05:51:53.578 INFO:teuthology.orchestra.run.smithi123.stdout:167.out 2024-04-24T05:51:53.579 INFO:teuthology.orchestra.run.smithi123.stdout:168 2024-04-24T05:51:53.579 INFO:teuthology.orchestra.run.smithi123.stdout:168.out 2024-04-24T05:51:53.579 INFO:teuthology.orchestra.run.smithi123.stdout:169 2024-04-24T05:51:53.579 INFO:teuthology.orchestra.run.smithi123.stdout:169.out 2024-04-24T05:51:53.579 INFO:teuthology.orchestra.run.smithi123.stdout:170 2024-04-24T05:51:53.579 INFO:teuthology.orchestra.run.smithi123.stdout:170.out 2024-04-24T05:51:53.579 INFO:teuthology.orchestra.run.smithi123.stdout:171 2024-04-24T05:51:53.579 INFO:teuthology.orchestra.run.smithi123.stdout:171.out 2024-04-24T05:51:53.579 INFO:teuthology.orchestra.run.smithi123.stdout:172 2024-04-24T05:51:53.579 INFO:teuthology.orchestra.run.smithi123.stdout:172.out 2024-04-24T05:51:53.579 INFO:teuthology.orchestra.run.smithi123.stdout:173 2024-04-24T05:51:53.579 INFO:teuthology.orchestra.run.smithi123.stdout:173.out 2024-04-24T05:51:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:174 2024-04-24T05:51:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:174.out 2024-04-24T05:51:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:175 2024-04-24T05:51:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:175.out 2024-04-24T05:51:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:176 2024-04-24T05:51:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:176.out 2024-04-24T05:51:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:177 2024-04-24T05:51:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:177.out 2024-04-24T05:51:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:178 2024-04-24T05:51:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:178.out 2024-04-24T05:51:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:179 2024-04-24T05:51:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:179.out 2024-04-24T05:51:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:180 2024-04-24T05:51:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:180.out 2024-04-24T05:51:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:181 2024-04-24T05:51:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:181.out 2024-04-24T05:51:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:182 2024-04-24T05:51:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:182.out 2024-04-24T05:51:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:183 2024-04-24T05:51:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:183.out 2024-04-24T05:51:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:184 2024-04-24T05:51:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:184.out 2024-04-24T05:51:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:185 2024-04-24T05:51:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:185.out 2024-04-24T05:51:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:186 2024-04-24T05:51:53.581 INFO:teuthology.orchestra.run.smithi123.stdout:186.out 2024-04-24T05:51:53.582 INFO:teuthology.orchestra.run.smithi123.stdout:187 2024-04-24T05:51:53.582 INFO:teuthology.orchestra.run.smithi123.stdout:187.out 2024-04-24T05:51:53.582 INFO:teuthology.orchestra.run.smithi123.stdout:188 2024-04-24T05:51:53.582 INFO:teuthology.orchestra.run.smithi123.stdout:188.out 2024-04-24T05:51:53.582 INFO:teuthology.orchestra.run.smithi123.stdout:189 2024-04-24T05:51:53.582 INFO:teuthology.orchestra.run.smithi123.stdout:189.out 2024-04-24T05:51:53.582 INFO:teuthology.orchestra.run.smithi123.stdout:190 2024-04-24T05:51:53.582 INFO:teuthology.orchestra.run.smithi123.stdout:190.out 2024-04-24T05:51:53.582 INFO:teuthology.orchestra.run.smithi123.stdout:191 2024-04-24T05:51:53.582 INFO:teuthology.orchestra.run.smithi123.stdout:191.out 2024-04-24T05:51:53.582 INFO:teuthology.orchestra.run.smithi123.stdout:192 2024-04-24T05:51:53.583 INFO:teuthology.orchestra.run.smithi123.stdout:192.out 2024-04-24T05:51:53.583 INFO:teuthology.orchestra.run.smithi123.stdout:193 2024-04-24T05:51:53.583 INFO:teuthology.orchestra.run.smithi123.stdout:193.out 2024-04-24T05:51:53.583 INFO:teuthology.orchestra.run.smithi123.stdout:194 2024-04-24T05:51:53.583 INFO:teuthology.orchestra.run.smithi123.stdout:194.out 2024-04-24T05:51:53.583 INFO:teuthology.orchestra.run.smithi123.stdout:195 2024-04-24T05:51:53.583 INFO:teuthology.orchestra.run.smithi123.stdout:195.out 2024-04-24T05:51:53.583 INFO:teuthology.orchestra.run.smithi123.stdout:196 2024-04-24T05:51:53.583 INFO:teuthology.orchestra.run.smithi123.stdout:196.out 2024-04-24T05:51:53.583 INFO:teuthology.orchestra.run.smithi123.stdout:197 2024-04-24T05:51:53.583 INFO:teuthology.orchestra.run.smithi123.stdout:197.out 2024-04-24T05:51:53.583 INFO:teuthology.orchestra.run.smithi123.stdout:198 2024-04-24T05:51:53.583 INFO:teuthology.orchestra.run.smithi123.stdout:198.out 2024-04-24T05:51:53.583 INFO:teuthology.orchestra.run.smithi123.stdout:199 2024-04-24T05:51:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:199.out 2024-04-24T05:51:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:200 2024-04-24T05:51:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:200.out 2024-04-24T05:51:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:201 2024-04-24T05:51:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:201.out 2024-04-24T05:51:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:202 2024-04-24T05:51:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:202.out 2024-04-24T05:51:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:203 2024-04-24T05:51:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:203.out 2024-04-24T05:51:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:204 2024-04-24T05:51:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:204.out 2024-04-24T05:51:53.584 INFO:teuthology.orchestra.run.smithi123.stdout:205 2024-04-24T05:51:53.585 INFO:teuthology.orchestra.run.smithi123.stdout:205.out 2024-04-24T05:51:53.585 INFO:teuthology.orchestra.run.smithi123.stdout:206 2024-04-24T05:51:53.585 INFO:teuthology.orchestra.run.smithi123.stdout:206.out 2024-04-24T05:51:53.585 INFO:teuthology.orchestra.run.smithi123.stdout:207 2024-04-24T05:51:53.585 INFO:teuthology.orchestra.run.smithi123.stdout:207.out 2024-04-24T05:51:53.585 INFO:teuthology.orchestra.run.smithi123.stdout:208 2024-04-24T05:51:53.585 INFO:teuthology.orchestra.run.smithi123.stdout:208.out 2024-04-24T05:51:53.585 INFO:teuthology.orchestra.run.smithi123.stdout:209 2024-04-24T05:51:53.585 INFO:teuthology.orchestra.run.smithi123.stdout:209.out 2024-04-24T05:51:53.585 INFO:teuthology.orchestra.run.smithi123.stdout:210 2024-04-24T05:51:53.585 INFO:teuthology.orchestra.run.smithi123.stdout:210.out 2024-04-24T05:51:53.585 INFO:teuthology.orchestra.run.smithi123.stdout:211 2024-04-24T05:51:53.585 INFO:teuthology.orchestra.run.smithi123.stdout:211.out 2024-04-24T05:51:53.586 INFO:teuthology.orchestra.run.smithi123.stdout:212 2024-04-24T05:51:53.586 INFO:teuthology.orchestra.run.smithi123.stdout:212.out 2024-04-24T05:51:53.586 INFO:teuthology.orchestra.run.smithi123.stdout:213 2024-04-24T05:51:53.586 INFO:teuthology.orchestra.run.smithi123.stdout:213.out 2024-04-24T05:51:53.586 INFO:teuthology.orchestra.run.smithi123.stdout:214 2024-04-24T05:51:53.586 INFO:teuthology.orchestra.run.smithi123.stdout:214.out 2024-04-24T05:51:53.586 INFO:teuthology.orchestra.run.smithi123.stdout:215 2024-04-24T05:51:53.586 INFO:teuthology.orchestra.run.smithi123.stdout:215.out 2024-04-24T05:51:53.586 INFO:teuthology.orchestra.run.smithi123.stdout:216 2024-04-24T05:51:53.586 INFO:teuthology.orchestra.run.smithi123.stdout:216.out 2024-04-24T05:51:53.586 INFO:teuthology.orchestra.run.smithi123.stdout:217 2024-04-24T05:51:53.587 INFO:teuthology.orchestra.run.smithi123.stdout:217.out 2024-04-24T05:51:53.587 INFO:teuthology.orchestra.run.smithi123.stdout:218 2024-04-24T05:51:53.587 INFO:teuthology.orchestra.run.smithi123.stdout:218.out 2024-04-24T05:51:53.587 INFO:teuthology.orchestra.run.smithi123.stdout:219 2024-04-24T05:51:53.587 INFO:teuthology.orchestra.run.smithi123.stdout:219.out 2024-04-24T05:51:53.587 INFO:teuthology.orchestra.run.smithi123.stdout:220 2024-04-24T05:51:53.587 INFO:teuthology.orchestra.run.smithi123.stdout:220.out 2024-04-24T05:51:53.587 INFO:teuthology.orchestra.run.smithi123.stdout:221 2024-04-24T05:51:53.587 INFO:teuthology.orchestra.run.smithi123.stdout:221.out 2024-04-24T05:51:53.587 INFO:teuthology.orchestra.run.smithi123.stdout:222 2024-04-24T05:51:53.587 INFO:teuthology.orchestra.run.smithi123.stdout:222.out 2024-04-24T05:51:53.587 INFO:teuthology.orchestra.run.smithi123.stdout:223 2024-04-24T05:51:53.588 INFO:teuthology.orchestra.run.smithi123.stdout:223.out 2024-04-24T05:51:53.588 INFO:teuthology.orchestra.run.smithi123.stdout:224 2024-04-24T05:51:53.588 INFO:teuthology.orchestra.run.smithi123.stdout:224.out 2024-04-24T05:51:53.588 INFO:teuthology.orchestra.run.smithi123.stdout:225 2024-04-24T05:51:53.588 INFO:teuthology.orchestra.run.smithi123.stdout:225.out 2024-04-24T05:51:53.588 INFO:teuthology.orchestra.run.smithi123.stdout:226 2024-04-24T05:51:53.588 INFO:teuthology.orchestra.run.smithi123.stdout:226.out 2024-04-24T05:51:53.588 INFO:teuthology.orchestra.run.smithi123.stdout:227 2024-04-24T05:51:53.588 INFO:teuthology.orchestra.run.smithi123.stdout:227.out 2024-04-24T05:51:53.588 INFO:teuthology.orchestra.run.smithi123.stdout:228 2024-04-24T05:51:53.588 INFO:teuthology.orchestra.run.smithi123.stdout:228.out 2024-04-24T05:51:53.588 INFO:teuthology.orchestra.run.smithi123.stdout:229 2024-04-24T05:51:53.589 INFO:teuthology.orchestra.run.smithi123.stdout:229.out 2024-04-24T05:51:53.589 INFO:teuthology.orchestra.run.smithi123.stdout:230 2024-04-24T05:51:53.589 INFO:teuthology.orchestra.run.smithi123.stdout:230.out 2024-04-24T05:51:53.589 INFO:teuthology.orchestra.run.smithi123.stdout:231 2024-04-24T05:51:53.589 INFO:teuthology.orchestra.run.smithi123.stdout:231.out 2024-04-24T05:51:53.589 INFO:teuthology.orchestra.run.smithi123.stdout:232 2024-04-24T05:51:53.589 INFO:teuthology.orchestra.run.smithi123.stdout:232.out 2024-04-24T05:51:53.589 INFO:teuthology.orchestra.run.smithi123.stdout:233 2024-04-24T05:51:53.589 INFO:teuthology.orchestra.run.smithi123.stdout:233.out 2024-04-24T05:51:53.589 INFO:teuthology.orchestra.run.smithi123.stdout:234 2024-04-24T05:51:53.589 INFO:teuthology.orchestra.run.smithi123.stdout:234.out 2024-04-24T05:51:53.589 INFO:teuthology.orchestra.run.smithi123.stdout:235 2024-04-24T05:51:53.589 INFO:teuthology.orchestra.run.smithi123.stdout:235.out 2024-04-24T05:51:53.590 INFO:teuthology.orchestra.run.smithi123.stdout:236 2024-04-24T05:51:53.590 INFO:teuthology.orchestra.run.smithi123.stdout:236.out 2024-04-24T05:51:53.590 INFO:teuthology.orchestra.run.smithi123.stdout:237 2024-04-24T05:51:53.590 INFO:teuthology.orchestra.run.smithi123.stdout:237.out 2024-04-24T05:51:53.590 INFO:teuthology.orchestra.run.smithi123.stdout:238 2024-04-24T05:51:53.590 INFO:teuthology.orchestra.run.smithi123.stdout:238.out 2024-04-24T05:51:53.590 INFO:teuthology.orchestra.run.smithi123.stdout:239 2024-04-24T05:51:53.590 INFO:teuthology.orchestra.run.smithi123.stdout:239.out 2024-04-24T05:51:53.590 INFO:teuthology.orchestra.run.smithi123.stdout:240 2024-04-24T05:51:53.590 INFO:teuthology.orchestra.run.smithi123.stdout:240.out 2024-04-24T05:51:53.590 INFO:teuthology.orchestra.run.smithi123.stdout:241 2024-04-24T05:51:53.590 INFO:teuthology.orchestra.run.smithi123.stdout:241.out 2024-04-24T05:51:53.590 INFO:teuthology.orchestra.run.smithi123.stdout:242 2024-04-24T05:51:53.591 INFO:teuthology.orchestra.run.smithi123.stdout:242.out 2024-04-24T05:51:53.591 INFO:teuthology.orchestra.run.smithi123.stdout:243 2024-04-24T05:51:53.591 INFO:teuthology.orchestra.run.smithi123.stdout:243.out 2024-04-24T05:51:53.591 INFO:teuthology.orchestra.run.smithi123.stdout:244 2024-04-24T05:51:53.591 INFO:teuthology.orchestra.run.smithi123.stdout:244.out 2024-04-24T05:51:53.591 INFO:teuthology.orchestra.run.smithi123.stdout:245 2024-04-24T05:51:53.591 INFO:teuthology.orchestra.run.smithi123.stdout:245.out 2024-04-24T05:51:53.591 INFO:teuthology.orchestra.run.smithi123.stdout:246 2024-04-24T05:51:53.591 INFO:teuthology.orchestra.run.smithi123.stdout:246.out 2024-04-24T05:51:53.591 INFO:teuthology.orchestra.run.smithi123.stdout:247 2024-04-24T05:51:53.591 INFO:teuthology.orchestra.run.smithi123.stdout:247.out 2024-04-24T05:51:53.591 INFO:teuthology.orchestra.run.smithi123.stdout:248 2024-04-24T05:51:53.591 INFO:teuthology.orchestra.run.smithi123.stdout:248.out 2024-04-24T05:51:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:249 2024-04-24T05:51:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:249.out 2024-04-24T05:51:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:250 2024-04-24T05:51:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:250.out 2024-04-24T05:51:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:251 2024-04-24T05:51:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:251.out 2024-04-24T05:51:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:252 2024-04-24T05:51:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:252.out 2024-04-24T05:51:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:253 2024-04-24T05:51:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:253.out 2024-04-24T05:51:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:254 2024-04-24T05:51:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:254.out 2024-04-24T05:51:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:255 2024-04-24T05:51:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:255.out 2024-04-24T05:51:53.593 INFO:teuthology.orchestra.run.smithi123.stdout:256 2024-04-24T05:51:53.593 INFO:teuthology.orchestra.run.smithi123.stdout:256.out 2024-04-24T05:51:53.593 INFO:teuthology.orchestra.run.smithi123.stdout:257 2024-04-24T05:51:53.593 INFO:teuthology.orchestra.run.smithi123.stdout:257.out 2024-04-24T05:51:53.593 INFO:teuthology.orchestra.run.smithi123.stdout:258 2024-04-24T05:51:53.593 INFO:teuthology.orchestra.run.smithi123.stdout:258.out 2024-04-24T05:51:53.593 INFO:teuthology.orchestra.run.smithi123.stdout:259 2024-04-24T05:51:53.593 INFO:teuthology.orchestra.run.smithi123.stdout:259.out 2024-04-24T05:51:53.593 INFO:teuthology.orchestra.run.smithi123.stdout:260 2024-04-24T05:51:53.593 INFO:teuthology.orchestra.run.smithi123.stdout:260.out 2024-04-24T05:51:53.593 INFO:teuthology.orchestra.run.smithi123.stdout:261 2024-04-24T05:51:53.593 INFO:teuthology.orchestra.run.smithi123.stdout:261.out 2024-04-24T05:51:53.594 INFO:teuthology.orchestra.run.smithi123.stdout:262 2024-04-24T05:51:53.594 INFO:teuthology.orchestra.run.smithi123.stdout:262.out 2024-04-24T05:51:53.594 INFO:teuthology.orchestra.run.smithi123.stdout:263 2024-04-24T05:51:53.594 INFO:teuthology.orchestra.run.smithi123.stdout:263.out 2024-04-24T05:51:53.594 INFO:teuthology.orchestra.run.smithi123.stdout:264 2024-04-24T05:51:53.594 INFO:teuthology.orchestra.run.smithi123.stdout:264.out 2024-04-24T05:51:53.594 INFO:teuthology.orchestra.run.smithi123.stdout:265 2024-04-24T05:51:53.594 INFO:teuthology.orchestra.run.smithi123.stdout:265.out 2024-04-24T05:51:53.594 INFO:teuthology.orchestra.run.smithi123.stdout:266 2024-04-24T05:51:53.594 INFO:teuthology.orchestra.run.smithi123.stdout:266.out 2024-04-24T05:51:53.594 INFO:teuthology.orchestra.run.smithi123.stdout:267 2024-04-24T05:51:53.594 INFO:teuthology.orchestra.run.smithi123.stdout:267.out 2024-04-24T05:51:53.595 INFO:teuthology.orchestra.run.smithi123.stdout:268 2024-04-24T05:51:53.595 INFO:teuthology.orchestra.run.smithi123.stdout:268.out 2024-04-24T05:51:53.595 INFO:teuthology.orchestra.run.smithi123.stdout:269 2024-04-24T05:51:53.595 INFO:teuthology.orchestra.run.smithi123.stdout:269.out 2024-04-24T05:51:53.595 INFO:teuthology.orchestra.run.smithi123.stdout:270 2024-04-24T05:51:53.595 INFO:teuthology.orchestra.run.smithi123.stdout:270.out 2024-04-24T05:51:53.595 INFO:teuthology.orchestra.run.smithi123.stdout:271 2024-04-24T05:51:53.595 INFO:teuthology.orchestra.run.smithi123.stdout:271.out 2024-04-24T05:51:53.595 INFO:teuthology.orchestra.run.smithi123.stdout:272 2024-04-24T05:51:53.595 INFO:teuthology.orchestra.run.smithi123.stdout:272.out 2024-04-24T05:51:53.595 INFO:teuthology.orchestra.run.smithi123.stdout:273 2024-04-24T05:51:53.595 INFO:teuthology.orchestra.run.smithi123.stdout:273.out 2024-04-24T05:51:53.595 INFO:teuthology.orchestra.run.smithi123.stdout:274 2024-04-24T05:51:53.596 INFO:teuthology.orchestra.run.smithi123.stdout:274.out 2024-04-24T05:51:53.596 INFO:teuthology.orchestra.run.smithi123.stdout:275 2024-04-24T05:51:53.596 INFO:teuthology.orchestra.run.smithi123.stdout:275.out 2024-04-24T05:51:53.596 INFO:teuthology.orchestra.run.smithi123.stdout:276 2024-04-24T05:51:53.596 INFO:teuthology.orchestra.run.smithi123.stdout:276.out 2024-04-24T05:51:53.596 INFO:teuthology.orchestra.run.smithi123.stdout:277 2024-04-24T05:51:53.596 INFO:teuthology.orchestra.run.smithi123.stdout:277.out 2024-04-24T05:51:53.596 INFO:teuthology.orchestra.run.smithi123.stdout:278 2024-04-24T05:51:53.596 INFO:teuthology.orchestra.run.smithi123.stdout:278.out 2024-04-24T05:51:53.596 INFO:teuthology.orchestra.run.smithi123.stdout:279 2024-04-24T05:51:53.596 INFO:teuthology.orchestra.run.smithi123.stdout:279.out 2024-04-24T05:51:53.597 INFO:teuthology.orchestra.run.smithi123.stdout:280 2024-04-24T05:51:53.597 INFO:teuthology.orchestra.run.smithi123.stdout:280.out 2024-04-24T05:51:53.597 INFO:teuthology.orchestra.run.smithi123.stdout:281 2024-04-24T05:51:53.597 INFO:teuthology.orchestra.run.smithi123.stdout:281.out 2024-04-24T05:51:53.597 INFO:teuthology.orchestra.run.smithi123.stdout:282 2024-04-24T05:51:53.597 INFO:teuthology.orchestra.run.smithi123.stdout:282.out 2024-04-24T05:51:53.597 INFO:teuthology.orchestra.run.smithi123.stdout:283 2024-04-24T05:51:53.597 INFO:teuthology.orchestra.run.smithi123.stdout:283.out 2024-04-24T05:51:53.597 INFO:teuthology.orchestra.run.smithi123.stdout:284 2024-04-24T05:51:53.597 INFO:teuthology.orchestra.run.smithi123.stdout:284.out 2024-04-24T05:51:53.597 INFO:teuthology.orchestra.run.smithi123.stdout:285 2024-04-24T05:51:53.597 INFO:teuthology.orchestra.run.smithi123.stdout:285.out 2024-04-24T05:51:53.598 INFO:teuthology.orchestra.run.smithi123.stdout:286 2024-04-24T05:51:53.598 INFO:teuthology.orchestra.run.smithi123.stdout:286.out 2024-04-24T05:51:53.598 INFO:teuthology.orchestra.run.smithi123.stdout:287 2024-04-24T05:51:53.598 INFO:teuthology.orchestra.run.smithi123.stdout:287.out 2024-04-24T05:51:53.598 INFO:teuthology.orchestra.run.smithi123.stdout:288 2024-04-24T05:51:53.598 INFO:teuthology.orchestra.run.smithi123.stdout:288.out 2024-04-24T05:51:53.598 INFO:teuthology.orchestra.run.smithi123.stdout:289 2024-04-24T05:51:53.598 INFO:teuthology.orchestra.run.smithi123.stdout:289.out 2024-04-24T05:51:53.598 INFO:teuthology.orchestra.run.smithi123.stdout:290 2024-04-24T05:51:53.598 INFO:teuthology.orchestra.run.smithi123.stdout:290.out 2024-04-24T05:51:53.598 INFO:teuthology.orchestra.run.smithi123.stdout:291 2024-04-24T05:51:53.598 INFO:teuthology.orchestra.run.smithi123.stdout:291.out 2024-04-24T05:51:53.599 INFO:teuthology.orchestra.run.smithi123.stdout:292 2024-04-24T05:51:53.599 INFO:teuthology.orchestra.run.smithi123.stdout:292.out 2024-04-24T05:51:53.599 INFO:teuthology.orchestra.run.smithi123.stdout:293 2024-04-24T05:51:53.599 INFO:teuthology.orchestra.run.smithi123.stdout:293.out 2024-04-24T05:51:53.599 INFO:teuthology.orchestra.run.smithi123.stdout:294 2024-04-24T05:51:53.599 INFO:teuthology.orchestra.run.smithi123.stdout:294.out 2024-04-24T05:51:53.599 INFO:teuthology.orchestra.run.smithi123.stdout:295 2024-04-24T05:51:53.599 INFO:teuthology.orchestra.run.smithi123.stdout:295.out 2024-04-24T05:51:53.599 INFO:teuthology.orchestra.run.smithi123.stdout:296 2024-04-24T05:51:53.599 INFO:teuthology.orchestra.run.smithi123.stdout:296.out 2024-04-24T05:51:53.599 INFO:teuthology.orchestra.run.smithi123.stdout:297 2024-04-24T05:51:53.599 INFO:teuthology.orchestra.run.smithi123.stdout:297.out 2024-04-24T05:51:53.599 INFO:teuthology.orchestra.run.smithi123.stdout:298 2024-04-24T05:51:53.600 INFO:teuthology.orchestra.run.smithi123.stdout:298.out 2024-04-24T05:51:53.600 INFO:teuthology.orchestra.run.smithi123.stdout:299 2024-04-24T05:51:53.600 INFO:teuthology.orchestra.run.smithi123.stdout:299.out 2024-04-24T05:51:53.600 INFO:teuthology.orchestra.run.smithi123.stdout:300 2024-04-24T05:51:53.600 INFO:teuthology.orchestra.run.smithi123.stdout:300.out 2024-04-24T05:51:53.600 INFO:teuthology.orchestra.run.smithi123.stdout:301 2024-04-24T05:51:53.600 INFO:teuthology.orchestra.run.smithi123.stdout:301.out 2024-04-24T05:51:53.600 INFO:teuthology.orchestra.run.smithi123.stdout:302 2024-04-24T05:51:53.600 INFO:teuthology.orchestra.run.smithi123.stdout:302.out 2024-04-24T05:51:53.600 INFO:teuthology.orchestra.run.smithi123.stdout:303 2024-04-24T05:51:53.600 INFO:teuthology.orchestra.run.smithi123.stdout:303.out 2024-04-24T05:51:53.600 INFO:teuthology.orchestra.run.smithi123.stdout:304 2024-04-24T05:51:53.600 INFO:teuthology.orchestra.run.smithi123.stdout:304.out 2024-04-24T05:51:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:305 2024-04-24T05:51:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:305.out 2024-04-24T05:51:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:306 2024-04-24T05:51:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:306.out 2024-04-24T05:51:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:307 2024-04-24T05:51:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:307.out 2024-04-24T05:51:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:308 2024-04-24T05:51:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:308.out 2024-04-24T05:51:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:309 2024-04-24T05:51:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:309.out 2024-04-24T05:51:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:310 2024-04-24T05:51:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:310.out 2024-04-24T05:51:53.601 INFO:teuthology.orchestra.run.smithi123.stdout:311 2024-04-24T05:51:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:311.out 2024-04-24T05:51:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:312 2024-04-24T05:51:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:312.out 2024-04-24T05:51:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:313 2024-04-24T05:51:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:313.out 2024-04-24T05:51:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:314 2024-04-24T05:51:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:314.out 2024-04-24T05:51:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:315 2024-04-24T05:51:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:315.out 2024-04-24T05:51:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:316 2024-04-24T05:51:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:316.out 2024-04-24T05:51:53.602 INFO:teuthology.orchestra.run.smithi123.stdout:317 2024-04-24T05:51:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:317.out 2024-04-24T05:51:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:318 2024-04-24T05:51:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:318.out 2024-04-24T05:51:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:319 2024-04-24T05:51:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:319.out 2024-04-24T05:51:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:320 2024-04-24T05:51:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:320.out 2024-04-24T05:51:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:321 2024-04-24T05:51:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:321.out 2024-04-24T05:51:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:322 2024-04-24T05:51:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:322.out 2024-04-24T05:51:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:323 2024-04-24T05:51:53.603 INFO:teuthology.orchestra.run.smithi123.stdout:323.out 2024-04-24T05:51:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:324 2024-04-24T05:51:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:324.out 2024-04-24T05:51:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:325 2024-04-24T05:51:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:325.out 2024-04-24T05:51:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:326 2024-04-24T05:51:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:326.out 2024-04-24T05:51:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:327 2024-04-24T05:51:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:327.out 2024-04-24T05:51:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:328 2024-04-24T05:51:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:328.out 2024-04-24T05:51:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:329 2024-04-24T05:51:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:329.out 2024-04-24T05:51:53.604 INFO:teuthology.orchestra.run.smithi123.stdout:330 2024-04-24T05:51:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:330.out 2024-04-24T05:51:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:331 2024-04-24T05:51:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:331.out 2024-04-24T05:51:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:332 2024-04-24T05:51:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:332.out 2024-04-24T05:51:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:333 2024-04-24T05:51:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:333.out 2024-04-24T05:51:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:334 2024-04-24T05:51:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:334.out 2024-04-24T05:51:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:335 2024-04-24T05:51:53.605 INFO:teuthology.orchestra.run.smithi123.stdout:335.out 2024-04-24T05:51:53.606 INFO:teuthology.orchestra.run.smithi123.stdout:336 2024-04-24T05:51:53.606 INFO:teuthology.orchestra.run.smithi123.stdout:336.out 2024-04-24T05:51:53.606 INFO:teuthology.orchestra.run.smithi123.stdout:337 2024-04-24T05:51:53.606 INFO:teuthology.orchestra.run.smithi123.stdout:337.out 2024-04-24T05:51:53.606 INFO:teuthology.orchestra.run.smithi123.stdout:338 2024-04-24T05:51:53.606 INFO:teuthology.orchestra.run.smithi123.stdout:338.out 2024-04-24T05:51:53.606 INFO:teuthology.orchestra.run.smithi123.stdout:339 2024-04-24T05:51:53.606 INFO:teuthology.orchestra.run.smithi123.stdout:339.out 2024-04-24T05:51:53.606 INFO:teuthology.orchestra.run.smithi123.stdout:340 2024-04-24T05:51:53.606 INFO:teuthology.orchestra.run.smithi123.stdout:340.out 2024-04-24T05:51:53.606 INFO:teuthology.orchestra.run.smithi123.stdout:341 2024-04-24T05:51:53.607 INFO:teuthology.orchestra.run.smithi123.stdout:341.out 2024-04-24T05:51:53.607 INFO:teuthology.orchestra.run.smithi123.stdout:342 2024-04-24T05:51:53.607 INFO:teuthology.orchestra.run.smithi123.stdout:342.out 2024-04-24T05:51:53.607 INFO:teuthology.orchestra.run.smithi123.stdout:343 2024-04-24T05:51:53.607 INFO:teuthology.orchestra.run.smithi123.stdout:343.out 2024-04-24T05:51:53.607 INFO:teuthology.orchestra.run.smithi123.stdout:344 2024-04-24T05:51:53.607 INFO:teuthology.orchestra.run.smithi123.stdout:344.out 2024-04-24T05:51:53.607 INFO:teuthology.orchestra.run.smithi123.stdout:345 2024-04-24T05:51:53.607 INFO:teuthology.orchestra.run.smithi123.stdout:345.out 2024-04-24T05:51:53.607 INFO:teuthology.orchestra.run.smithi123.stdout:346 2024-04-24T05:51:53.607 INFO:teuthology.orchestra.run.smithi123.stdout:346.out 2024-04-24T05:51:53.607 INFO:teuthology.orchestra.run.smithi123.stdout:347 2024-04-24T05:51:53.607 INFO:teuthology.orchestra.run.smithi123.stdout:347.out 2024-04-24T05:51:53.608 INFO:teuthology.orchestra.run.smithi123.stdout:348 2024-04-24T05:51:53.608 INFO:teuthology.orchestra.run.smithi123.stdout:348.out 2024-04-24T05:51:53.608 INFO:teuthology.orchestra.run.smithi123.stdout:349 2024-04-24T05:51:53.608 INFO:teuthology.orchestra.run.smithi123.stdout:349.out 2024-04-24T05:51:53.608 INFO:teuthology.orchestra.run.smithi123.stdout:350 2024-04-24T05:51:53.608 INFO:teuthology.orchestra.run.smithi123.stdout:350.out 2024-04-24T05:51:53.608 INFO:teuthology.orchestra.run.smithi123.stdout:351 2024-04-24T05:51:53.608 INFO:teuthology.orchestra.run.smithi123.stdout:351.out 2024-04-24T05:51:53.608 INFO:teuthology.orchestra.run.smithi123.stdout:352 2024-04-24T05:51:53.608 INFO:teuthology.orchestra.run.smithi123.stdout:352.out 2024-04-24T05:51:53.608 INFO:teuthology.orchestra.run.smithi123.stdout:353 2024-04-24T05:51:53.608 INFO:teuthology.orchestra.run.smithi123.stdout:353.out 2024-04-24T05:51:53.608 INFO:teuthology.orchestra.run.smithi123.stdout:354 2024-04-24T05:51:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:354.out 2024-04-24T05:51:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:355 2024-04-24T05:51:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:355.out 2024-04-24T05:51:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:356 2024-04-24T05:51:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:356.out 2024-04-24T05:51:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:357 2024-04-24T05:51:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:357.out 2024-04-24T05:51:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:358 2024-04-24T05:51:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:358.out 2024-04-24T05:51:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:359 2024-04-24T05:51:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:359.out 2024-04-24T05:51:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:360 2024-04-24T05:51:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:360.out 2024-04-24T05:51:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:361 2024-04-24T05:51:53.610 INFO:teuthology.orchestra.run.smithi123.stdout:361.out 2024-04-24T05:51:53.610 INFO:teuthology.orchestra.run.smithi123.stdout:362 2024-04-24T05:51:53.610 INFO:teuthology.orchestra.run.smithi123.stdout:362.out 2024-04-24T05:51:53.610 INFO:teuthology.orchestra.run.smithi123.stdout:363 2024-04-24T05:51:53.610 INFO:teuthology.orchestra.run.smithi123.stdout:363.out 2024-04-24T05:51:53.610 INFO:teuthology.orchestra.run.smithi123.stdout:364 2024-04-24T05:51:53.610 INFO:teuthology.orchestra.run.smithi123.stdout:364.out 2024-04-24T05:51:53.610 INFO:teuthology.orchestra.run.smithi123.stdout:365 2024-04-24T05:51:53.610 INFO:teuthology.orchestra.run.smithi123.stdout:365.out 2024-04-24T05:51:53.610 INFO:teuthology.orchestra.run.smithi123.stdout:366 2024-04-24T05:51:53.610 INFO:teuthology.orchestra.run.smithi123.stdout:366.out 2024-04-24T05:51:53.610 INFO:teuthology.orchestra.run.smithi123.stdout:367 2024-04-24T05:51:53.611 INFO:teuthology.orchestra.run.smithi123.stdout:367.out 2024-04-24T05:51:53.611 INFO:teuthology.orchestra.run.smithi123.stdout:368 2024-04-24T05:51:53.611 INFO:teuthology.orchestra.run.smithi123.stdout:368.out 2024-04-24T05:51:53.611 INFO:teuthology.orchestra.run.smithi123.stdout:369 2024-04-24T05:51:53.611 INFO:teuthology.orchestra.run.smithi123.stdout:369.out 2024-04-24T05:51:53.611 INFO:teuthology.orchestra.run.smithi123.stdout:370 2024-04-24T05:51:53.611 INFO:teuthology.orchestra.run.smithi123.stdout:370.out 2024-04-24T05:51:53.611 INFO:teuthology.orchestra.run.smithi123.stdout:371 2024-04-24T05:51:53.611 INFO:teuthology.orchestra.run.smithi123.stdout:371.out 2024-04-24T05:51:53.611 INFO:teuthology.orchestra.run.smithi123.stdout:372 2024-04-24T05:51:53.611 INFO:teuthology.orchestra.run.smithi123.stdout:372.out 2024-04-24T05:51:53.611 INFO:teuthology.orchestra.run.smithi123.stdout:373 2024-04-24T05:51:53.611 INFO:teuthology.orchestra.run.smithi123.stdout:373.out 2024-04-24T05:51:53.612 INFO:teuthology.orchestra.run.smithi123.stdout:374 2024-04-24T05:51:53.612 INFO:teuthology.orchestra.run.smithi123.stdout:374.out 2024-04-24T05:51:53.612 INFO:teuthology.orchestra.run.smithi123.stdout:375 2024-04-24T05:51:53.612 INFO:teuthology.orchestra.run.smithi123.stdout:375.out 2024-04-24T05:51:53.612 INFO:teuthology.orchestra.run.smithi123.stdout:376 2024-04-24T05:51:53.612 INFO:teuthology.orchestra.run.smithi123.stdout:376.out 2024-04-24T05:51:53.612 INFO:teuthology.orchestra.run.smithi123.stdout:377 2024-04-24T05:51:53.612 INFO:teuthology.orchestra.run.smithi123.stdout:377.out 2024-04-24T05:51:53.612 INFO:teuthology.orchestra.run.smithi123.stdout:378 2024-04-24T05:51:53.612 INFO:teuthology.orchestra.run.smithi123.stdout:378.out 2024-04-24T05:51:53.612 INFO:teuthology.orchestra.run.smithi123.stdout:379 2024-04-24T05:51:53.612 INFO:teuthology.orchestra.run.smithi123.stdout:379.out 2024-04-24T05:51:53.613 INFO:teuthology.orchestra.run.smithi123.stdout:380 2024-04-24T05:51:53.613 INFO:teuthology.orchestra.run.smithi123.stdout:380.out 2024-04-24T05:51:53.613 INFO:teuthology.orchestra.run.smithi123.stdout:381 2024-04-24T05:51:53.613 INFO:teuthology.orchestra.run.smithi123.stdout:381.out 2024-04-24T05:51:53.613 INFO:teuthology.orchestra.run.smithi123.stdout:382 2024-04-24T05:51:53.613 INFO:teuthology.orchestra.run.smithi123.stdout:382.out 2024-04-24T05:51:53.613 INFO:teuthology.orchestra.run.smithi123.stdout:383 2024-04-24T05:51:53.613 INFO:teuthology.orchestra.run.smithi123.stdout:383.out 2024-04-24T05:51:53.613 INFO:teuthology.orchestra.run.smithi123.stdout:384 2024-04-24T05:51:53.613 INFO:teuthology.orchestra.run.smithi123.stdout:384.out 2024-04-24T05:51:53.613 INFO:teuthology.orchestra.run.smithi123.stdout:385 2024-04-24T05:51:53.613 INFO:teuthology.orchestra.run.smithi123.stdout:385.out 2024-04-24T05:51:53.614 INFO:teuthology.orchestra.run.smithi123.stdout:386 2024-04-24T05:51:53.614 INFO:teuthology.orchestra.run.smithi123.stdout:386.out 2024-04-24T05:51:53.614 INFO:teuthology.orchestra.run.smithi123.stdout:387 2024-04-24T05:51:53.614 INFO:teuthology.orchestra.run.smithi123.stdout:387.out 2024-04-24T05:51:53.614 INFO:teuthology.orchestra.run.smithi123.stdout:388 2024-04-24T05:51:53.614 INFO:teuthology.orchestra.run.smithi123.stdout:388.out 2024-04-24T05:51:53.614 INFO:teuthology.orchestra.run.smithi123.stdout:389 2024-04-24T05:51:53.614 INFO:teuthology.orchestra.run.smithi123.stdout:389.out 2024-04-24T05:51:53.614 INFO:teuthology.orchestra.run.smithi123.stdout:390 2024-04-24T05:51:53.614 INFO:teuthology.orchestra.run.smithi123.stdout:390.out 2024-04-24T05:51:53.614 INFO:teuthology.orchestra.run.smithi123.stdout:391 2024-04-24T05:51:53.614 INFO:teuthology.orchestra.run.smithi123.stdout:391.out 2024-04-24T05:51:53.615 INFO:teuthology.orchestra.run.smithi123.stdout:392 2024-04-24T05:51:53.615 INFO:teuthology.orchestra.run.smithi123.stdout:392.out 2024-04-24T05:51:53.615 INFO:teuthology.orchestra.run.smithi123.stdout:393 2024-04-24T05:51:53.615 INFO:teuthology.orchestra.run.smithi123.stdout:393.out 2024-04-24T05:51:53.615 INFO:teuthology.orchestra.run.smithi123.stdout:394 2024-04-24T05:51:53.615 INFO:teuthology.orchestra.run.smithi123.stdout:394.out 2024-04-24T05:51:53.615 INFO:teuthology.orchestra.run.smithi123.stdout:395 2024-04-24T05:51:53.615 INFO:teuthology.orchestra.run.smithi123.stdout:395.out 2024-04-24T05:51:53.615 INFO:teuthology.orchestra.run.smithi123.stdout:396 2024-04-24T05:51:53.615 INFO:teuthology.orchestra.run.smithi123.stdout:396.out 2024-04-24T05:51:53.615 INFO:teuthology.orchestra.run.smithi123.stdout:397 2024-04-24T05:51:53.616 INFO:teuthology.orchestra.run.smithi123.stdout:397.out 2024-04-24T05:51:53.616 INFO:teuthology.orchestra.run.smithi123.stdout:398 2024-04-24T05:51:53.616 INFO:teuthology.orchestra.run.smithi123.stdout:398.out 2024-04-24T05:51:53.616 INFO:teuthology.orchestra.run.smithi123.stdout:399 2024-04-24T05:51:53.616 INFO:teuthology.orchestra.run.smithi123.stdout:399.out 2024-04-24T05:51:53.616 INFO:teuthology.orchestra.run.smithi123.stdout:400 2024-04-24T05:51:53.616 INFO:teuthology.orchestra.run.smithi123.stdout:400.out 2024-04-24T05:51:53.616 INFO:teuthology.orchestra.run.smithi123.stdout:401 2024-04-24T05:51:53.616 INFO:teuthology.orchestra.run.smithi123.stdout:401.out 2024-04-24T05:51:53.616 INFO:teuthology.orchestra.run.smithi123.stdout:402 2024-04-24T05:51:53.616 INFO:teuthology.orchestra.run.smithi123.stdout:402.out 2024-04-24T05:51:53.617 INFO:teuthology.orchestra.run.smithi123.stdout:403 2024-04-24T05:51:53.617 INFO:teuthology.orchestra.run.smithi123.stdout:403.out 2024-04-24T05:51:53.617 INFO:teuthology.orchestra.run.smithi123.stdout:404 2024-04-24T05:51:53.617 INFO:teuthology.orchestra.run.smithi123.stdout:404.out 2024-04-24T05:51:53.617 INFO:teuthology.orchestra.run.smithi123.stdout:405 2024-04-24T05:51:53.617 INFO:teuthology.orchestra.run.smithi123.stdout:405.out 2024-04-24T05:51:53.617 INFO:teuthology.orchestra.run.smithi123.stdout:406 2024-04-24T05:51:53.617 INFO:teuthology.orchestra.run.smithi123.stdout:406.out 2024-04-24T05:51:53.617 INFO:teuthology.orchestra.run.smithi123.stdout:407 2024-04-24T05:51:53.617 INFO:teuthology.orchestra.run.smithi123.stdout:407.out 2024-04-24T05:51:53.617 INFO:teuthology.orchestra.run.smithi123.stdout:408 2024-04-24T05:51:53.618 INFO:teuthology.orchestra.run.smithi123.stdout:408.out 2024-04-24T05:51:53.618 INFO:teuthology.orchestra.run.smithi123.stdout:409 2024-04-24T05:51:53.618 INFO:teuthology.orchestra.run.smithi123.stdout:409.out 2024-04-24T05:51:53.618 INFO:teuthology.orchestra.run.smithi123.stdout:410 2024-04-24T05:51:53.618 INFO:teuthology.orchestra.run.smithi123.stdout:410.out 2024-04-24T05:51:53.618 INFO:teuthology.orchestra.run.smithi123.stdout:411 2024-04-24T05:51:53.618 INFO:teuthology.orchestra.run.smithi123.stdout:411.out 2024-04-24T05:51:53.618 INFO:teuthology.orchestra.run.smithi123.stdout:412 2024-04-24T05:51:53.618 INFO:teuthology.orchestra.run.smithi123.stdout:412.out 2024-04-24T05:51:53.618 INFO:teuthology.orchestra.run.smithi123.stdout:413 2024-04-24T05:51:53.618 INFO:teuthology.orchestra.run.smithi123.stdout:413.out 2024-04-24T05:51:53.619 INFO:teuthology.orchestra.run.smithi123.stdout:414 2024-04-24T05:51:53.619 INFO:teuthology.orchestra.run.smithi123.stdout:414.out 2024-04-24T05:51:53.619 INFO:teuthology.orchestra.run.smithi123.stdout:415 2024-04-24T05:51:53.619 INFO:teuthology.orchestra.run.smithi123.stdout:415.out 2024-04-24T05:51:53.619 INFO:teuthology.orchestra.run.smithi123.stdout:416 2024-04-24T05:51:53.619 INFO:teuthology.orchestra.run.smithi123.stdout:416.out 2024-04-24T05:51:53.619 INFO:teuthology.orchestra.run.smithi123.stdout:417 2024-04-24T05:51:53.619 INFO:teuthology.orchestra.run.smithi123.stdout:417.out 2024-04-24T05:51:53.619 INFO:teuthology.orchestra.run.smithi123.stdout:418 2024-04-24T05:51:53.619 INFO:teuthology.orchestra.run.smithi123.stdout:418.out 2024-04-24T05:51:53.620 INFO:teuthology.orchestra.run.smithi123.stdout:419 2024-04-24T05:51:53.620 INFO:teuthology.orchestra.run.smithi123.stdout:419.out 2024-04-24T05:51:53.620 INFO:teuthology.orchestra.run.smithi123.stdout:420 2024-04-24T05:51:53.620 INFO:teuthology.orchestra.run.smithi123.stdout:420.out 2024-04-24T05:51:53.620 INFO:teuthology.orchestra.run.smithi123.stdout:421 2024-04-24T05:51:53.620 INFO:teuthology.orchestra.run.smithi123.stdout:421.out 2024-04-24T05:51:53.620 INFO:teuthology.orchestra.run.smithi123.stdout:422 2024-04-24T05:51:53.620 INFO:teuthology.orchestra.run.smithi123.stdout:422.out 2024-04-24T05:51:53.620 INFO:teuthology.orchestra.run.smithi123.stdout:423 2024-04-24T05:51:53.620 INFO:teuthology.orchestra.run.smithi123.stdout:423.out 2024-04-24T05:51:53.620 INFO:teuthology.orchestra.run.smithi123.stdout:424 2024-04-24T05:51:53.620 INFO:teuthology.orchestra.run.smithi123.stdout:424.out 2024-04-24T05:51:53.621 INFO:teuthology.orchestra.run.smithi123.stdout:425 2024-04-24T05:51:53.621 INFO:teuthology.orchestra.run.smithi123.stdout:425.out 2024-04-24T05:51:53.621 INFO:teuthology.orchestra.run.smithi123.stdout:426 2024-04-24T05:51:53.621 INFO:teuthology.orchestra.run.smithi123.stdout:426.out 2024-04-24T05:51:53.621 INFO:teuthology.orchestra.run.smithi123.stdout:427 2024-04-24T05:51:53.621 INFO:teuthology.orchestra.run.smithi123.stdout:427.out 2024-04-24T05:51:53.621 INFO:teuthology.orchestra.run.smithi123.stdout:428 2024-04-24T05:51:53.621 INFO:teuthology.orchestra.run.smithi123.stdout:428.out 2024-04-24T05:51:53.621 INFO:teuthology.orchestra.run.smithi123.stdout:429 2024-04-24T05:51:53.621 INFO:teuthology.orchestra.run.smithi123.stdout:429.out 2024-04-24T05:51:53.622 INFO:teuthology.orchestra.run.smithi123.stdout:430 2024-04-24T05:51:53.622 INFO:teuthology.orchestra.run.smithi123.stdout:430.out 2024-04-24T05:51:53.622 INFO:teuthology.orchestra.run.smithi123.stdout:431 2024-04-24T05:51:53.622 INFO:teuthology.orchestra.run.smithi123.stdout:431.out 2024-04-24T05:51:53.622 INFO:teuthology.orchestra.run.smithi123.stdout:432 2024-04-24T05:51:53.622 INFO:teuthology.orchestra.run.smithi123.stdout:432.out 2024-04-24T05:51:53.622 INFO:teuthology.orchestra.run.smithi123.stdout:433 2024-04-24T05:51:53.622 INFO:teuthology.orchestra.run.smithi123.stdout:433.out 2024-04-24T05:51:53.622 INFO:teuthology.orchestra.run.smithi123.stdout:434 2024-04-24T05:51:53.622 INFO:teuthology.orchestra.run.smithi123.stdout:434.out 2024-04-24T05:51:53.622 INFO:teuthology.orchestra.run.smithi123.stdout:435 2024-04-24T05:51:53.622 INFO:teuthology.orchestra.run.smithi123.stdout:435.out 2024-04-24T05:51:53.623 INFO:teuthology.orchestra.run.smithi123.stdout:436 2024-04-24T05:51:53.623 INFO:teuthology.orchestra.run.smithi123.stdout:436.out 2024-04-24T05:51:53.623 INFO:teuthology.orchestra.run.smithi123.stdout:437 2024-04-24T05:51:53.623 INFO:teuthology.orchestra.run.smithi123.stdout:437.out 2024-04-24T05:51:53.623 INFO:teuthology.orchestra.run.smithi123.stdout:438 2024-04-24T05:51:53.623 INFO:teuthology.orchestra.run.smithi123.stdout:438.out 2024-04-24T05:51:53.623 INFO:teuthology.orchestra.run.smithi123.stdout:439 2024-04-24T05:51:53.623 INFO:teuthology.orchestra.run.smithi123.stdout:439.out 2024-04-24T05:51:53.623 INFO:teuthology.orchestra.run.smithi123.stdout:440 2024-04-24T05:51:53.623 INFO:teuthology.orchestra.run.smithi123.stdout:440.out 2024-04-24T05:51:53.623 INFO:teuthology.orchestra.run.smithi123.stdout:441 2024-04-24T05:51:53.624 INFO:teuthology.orchestra.run.smithi123.stdout:441.out 2024-04-24T05:51:53.624 INFO:teuthology.orchestra.run.smithi123.stdout:442 2024-04-24T05:51:53.624 INFO:teuthology.orchestra.run.smithi123.stdout:442.out 2024-04-24T05:51:53.624 INFO:teuthology.orchestra.run.smithi123.stdout:443 2024-04-24T05:51:53.624 INFO:teuthology.orchestra.run.smithi123.stdout:443.out 2024-04-24T05:51:53.624 INFO:teuthology.orchestra.run.smithi123.stdout:444 2024-04-24T05:51:53.624 INFO:teuthology.orchestra.run.smithi123.stdout:444.out 2024-04-24T05:51:53.624 INFO:teuthology.orchestra.run.smithi123.stdout:445 2024-04-24T05:51:53.624 INFO:teuthology.orchestra.run.smithi123.stdout:445.out 2024-04-24T05:51:53.624 INFO:teuthology.orchestra.run.smithi123.stdout:446 2024-04-24T05:51:53.624 INFO:teuthology.orchestra.run.smithi123.stdout:446.out 2024-04-24T05:51:53.624 INFO:teuthology.orchestra.run.smithi123.stdout:447 2024-04-24T05:51:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:447.out 2024-04-24T05:51:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:448 2024-04-24T05:51:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:448.out 2024-04-24T05:51:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:449 2024-04-24T05:51:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:449.out 2024-04-24T05:51:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:450 2024-04-24T05:51:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:450.out 2024-04-24T05:51:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:451 2024-04-24T05:51:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:451.out 2024-04-24T05:51:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:452 2024-04-24T05:51:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:452.out 2024-04-24T05:51:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:453 2024-04-24T05:51:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:453.out 2024-04-24T05:51:53.626 INFO:teuthology.orchestra.run.smithi123.stdout:454 2024-04-24T05:51:53.626 INFO:teuthology.orchestra.run.smithi123.stdout:454.out 2024-04-24T05:51:53.626 INFO:teuthology.orchestra.run.smithi123.stdout:455 2024-04-24T05:51:53.626 INFO:teuthology.orchestra.run.smithi123.stdout:455.out 2024-04-24T05:51:53.626 INFO:teuthology.orchestra.run.smithi123.stdout:456 2024-04-24T05:51:53.626 INFO:teuthology.orchestra.run.smithi123.stdout:456.out 2024-04-24T05:51:53.626 INFO:teuthology.orchestra.run.smithi123.stdout:457 2024-04-24T05:51:53.626 INFO:teuthology.orchestra.run.smithi123.stdout:457.out 2024-04-24T05:51:53.626 INFO:teuthology.orchestra.run.smithi123.stdout:458 2024-04-24T05:51:53.626 INFO:teuthology.orchestra.run.smithi123.stdout:458.out 2024-04-24T05:51:53.626 INFO:teuthology.orchestra.run.smithi123.stdout:459 2024-04-24T05:51:53.626 INFO:teuthology.orchestra.run.smithi123.stdout:459.out 2024-04-24T05:51:53.627 INFO:teuthology.orchestra.run.smithi123.stdout:460 2024-04-24T05:51:53.627 INFO:teuthology.orchestra.run.smithi123.stdout:460.out 2024-04-24T05:51:53.627 INFO:teuthology.orchestra.run.smithi123.stdout:461 2024-04-24T05:51:53.627 INFO:teuthology.orchestra.run.smithi123.stdout:461.out 2024-04-24T05:51:53.627 INFO:teuthology.orchestra.run.smithi123.stdout:462 2024-04-24T05:51:53.627 INFO:teuthology.orchestra.run.smithi123.stdout:462.out 2024-04-24T05:51:53.627 INFO:teuthology.orchestra.run.smithi123.stdout:463 2024-04-24T05:51:53.627 INFO:teuthology.orchestra.run.smithi123.stdout:463.out 2024-04-24T05:51:53.627 INFO:teuthology.orchestra.run.smithi123.stdout:464 2024-04-24T05:51:53.627 INFO:teuthology.orchestra.run.smithi123.stdout:464.out 2024-04-24T05:51:53.627 INFO:teuthology.orchestra.run.smithi123.stdout:465 2024-04-24T05:51:53.627 INFO:teuthology.orchestra.run.smithi123.stdout:465.out 2024-04-24T05:51:53.628 INFO:teuthology.orchestra.run.smithi123.stdout:466 2024-04-24T05:51:53.628 INFO:teuthology.orchestra.run.smithi123.stdout:466.out 2024-04-24T05:51:53.628 INFO:teuthology.orchestra.run.smithi123.stdout:467 2024-04-24T05:51:53.628 INFO:teuthology.orchestra.run.smithi123.stdout:467.out 2024-04-24T05:51:53.628 INFO:teuthology.orchestra.run.smithi123.stdout:468 2024-04-24T05:51:53.628 INFO:teuthology.orchestra.run.smithi123.stdout:468.out 2024-04-24T05:51:53.628 INFO:teuthology.orchestra.run.smithi123.stdout:469 2024-04-24T05:51:53.628 INFO:teuthology.orchestra.run.smithi123.stdout:469.out 2024-04-24T05:51:53.628 INFO:teuthology.orchestra.run.smithi123.stdout:470 2024-04-24T05:51:53.628 INFO:teuthology.orchestra.run.smithi123.stdout:470.out 2024-04-24T05:51:53.628 INFO:teuthology.orchestra.run.smithi123.stdout:471 2024-04-24T05:51:53.629 INFO:teuthology.orchestra.run.smithi123.stdout:471.out 2024-04-24T05:51:53.629 INFO:teuthology.orchestra.run.smithi123.stdout:472 2024-04-24T05:51:53.629 INFO:teuthology.orchestra.run.smithi123.stdout:472.out 2024-04-24T05:51:53.629 INFO:teuthology.orchestra.run.smithi123.stdout:473 2024-04-24T05:51:53.629 INFO:teuthology.orchestra.run.smithi123.stdout:473.out 2024-04-24T05:51:53.629 INFO:teuthology.orchestra.run.smithi123.stdout:474 2024-04-24T05:51:53.629 INFO:teuthology.orchestra.run.smithi123.stdout:474.out 2024-04-24T05:51:53.629 INFO:teuthology.orchestra.run.smithi123.stdout:475 2024-04-24T05:51:53.629 INFO:teuthology.orchestra.run.smithi123.stdout:475.out 2024-04-24T05:51:53.629 INFO:teuthology.orchestra.run.smithi123.stdout:476 2024-04-24T05:51:53.629 INFO:teuthology.orchestra.run.smithi123.stdout:476.out 2024-04-24T05:51:53.630 INFO:teuthology.orchestra.run.smithi123.stdout:477 2024-04-24T05:51:53.630 INFO:teuthology.orchestra.run.smithi123.stdout:477.out 2024-04-24T05:51:53.630 INFO:teuthology.orchestra.run.smithi123.stdout:478 2024-04-24T05:51:53.630 INFO:teuthology.orchestra.run.smithi123.stdout:478.out 2024-04-24T05:51:53.630 INFO:teuthology.orchestra.run.smithi123.stdout:479 2024-04-24T05:51:53.630 INFO:teuthology.orchestra.run.smithi123.stdout:479.out 2024-04-24T05:51:53.630 INFO:teuthology.orchestra.run.smithi123.stdout:480 2024-04-24T05:51:53.630 INFO:teuthology.orchestra.run.smithi123.stdout:480.out 2024-04-24T05:51:53.630 INFO:teuthology.orchestra.run.smithi123.stdout:481 2024-04-24T05:51:53.630 INFO:teuthology.orchestra.run.smithi123.stdout:481.out 2024-04-24T05:51:53.630 INFO:teuthology.orchestra.run.smithi123.stdout:482 2024-04-24T05:51:53.631 INFO:teuthology.orchestra.run.smithi123.stdout:482.out 2024-04-24T05:51:53.631 INFO:teuthology.orchestra.run.smithi123.stdout:483 2024-04-24T05:51:53.631 INFO:teuthology.orchestra.run.smithi123.stdout:483.out 2024-04-24T05:51:53.631 INFO:teuthology.orchestra.run.smithi123.stdout:484 2024-04-24T05:51:53.631 INFO:teuthology.orchestra.run.smithi123.stdout:484.out 2024-04-24T05:51:53.631 INFO:teuthology.orchestra.run.smithi123.stdout:485 2024-04-24T05:51:53.631 INFO:teuthology.orchestra.run.smithi123.stdout:485.out 2024-04-24T05:51:53.631 INFO:teuthology.orchestra.run.smithi123.stdout:486 2024-04-24T05:51:53.631 INFO:teuthology.orchestra.run.smithi123.stdout:486.out 2024-04-24T05:51:53.631 INFO:teuthology.orchestra.run.smithi123.stdout:487 2024-04-24T05:51:53.631 INFO:teuthology.orchestra.run.smithi123.stdout:487.out 2024-04-24T05:51:53.631 INFO:teuthology.orchestra.run.smithi123.stdout:488 2024-04-24T05:51:53.631 INFO:teuthology.orchestra.run.smithi123.stdout:488.out 2024-04-24T05:51:53.632 INFO:teuthology.orchestra.run.smithi123.stdout:489 2024-04-24T05:51:53.632 INFO:teuthology.orchestra.run.smithi123.stdout:489.out 2024-04-24T05:51:53.632 INFO:teuthology.orchestra.run.smithi123.stdout:490 2024-04-24T05:51:53.632 INFO:teuthology.orchestra.run.smithi123.stdout:490.out 2024-04-24T05:51:53.632 INFO:teuthology.orchestra.run.smithi123.stdout:491 2024-04-24T05:51:53.632 INFO:teuthology.orchestra.run.smithi123.stdout:491.out 2024-04-24T05:51:53.632 INFO:teuthology.orchestra.run.smithi123.stdout:492 2024-04-24T05:51:53.632 INFO:teuthology.orchestra.run.smithi123.stdout:492.out 2024-04-24T05:51:53.632 INFO:teuthology.orchestra.run.smithi123.stdout:493 2024-04-24T05:51:53.632 INFO:teuthology.orchestra.run.smithi123.stdout:493.out 2024-04-24T05:51:53.632 INFO:teuthology.orchestra.run.smithi123.stdout:494 2024-04-24T05:51:53.633 INFO:teuthology.orchestra.run.smithi123.stdout:494.out 2024-04-24T05:51:53.633 INFO:teuthology.orchestra.run.smithi123.stdout:495 2024-04-24T05:51:53.633 INFO:teuthology.orchestra.run.smithi123.stdout:495.out 2024-04-24T05:51:53.633 INFO:teuthology.orchestra.run.smithi123.stdout:496 2024-04-24T05:51:53.633 INFO:teuthology.orchestra.run.smithi123.stdout:496.out 2024-04-24T05:51:53.633 INFO:teuthology.orchestra.run.smithi123.stdout:497 2024-04-24T05:51:53.633 INFO:teuthology.orchestra.run.smithi123.stdout:497.out 2024-04-24T05:51:53.633 INFO:teuthology.orchestra.run.smithi123.stdout:498 2024-04-24T05:51:53.633 INFO:teuthology.orchestra.run.smithi123.stdout:498.out 2024-04-24T05:51:53.633 INFO:teuthology.orchestra.run.smithi123.stdout:499 2024-04-24T05:51:53.633 INFO:teuthology.orchestra.run.smithi123.stdout:499.out 2024-04-24T05:51:53.633 INFO:teuthology.orchestra.run.smithi123.stdout:500 2024-04-24T05:51:53.633 INFO:teuthology.orchestra.run.smithi123.stdout:500.out 2024-04-24T05:51:53.634 INFO:teuthology.orchestra.run.smithi123.stdout:501 2024-04-24T05:51:53.634 INFO:teuthology.orchestra.run.smithi123.stdout:501.out 2024-04-24T05:51:53.634 INFO:teuthology.orchestra.run.smithi123.stdout:502 2024-04-24T05:51:53.634 INFO:teuthology.orchestra.run.smithi123.stdout:502.out 2024-04-24T05:51:53.634 INFO:teuthology.orchestra.run.smithi123.stdout:503 2024-04-24T05:51:53.634 INFO:teuthology.orchestra.run.smithi123.stdout:503.out 2024-04-24T05:51:53.634 INFO:teuthology.orchestra.run.smithi123.stdout:504 2024-04-24T05:51:53.634 INFO:teuthology.orchestra.run.smithi123.stdout:504.out 2024-04-24T05:51:53.634 INFO:teuthology.orchestra.run.smithi123.stdout:505 2024-04-24T05:51:53.634 INFO:teuthology.orchestra.run.smithi123.stdout:505.out 2024-04-24T05:51:53.634 INFO:teuthology.orchestra.run.smithi123.stdout:506 2024-04-24T05:51:53.634 INFO:teuthology.orchestra.run.smithi123.stdout:506.out 2024-04-24T05:51:53.635 INFO:teuthology.orchestra.run.smithi123.stdout:507 2024-04-24T05:51:53.635 INFO:teuthology.orchestra.run.smithi123.stdout:507.out 2024-04-24T05:51:53.635 INFO:teuthology.orchestra.run.smithi123.stdout:508 2024-04-24T05:51:53.635 INFO:teuthology.orchestra.run.smithi123.stdout:508.out 2024-04-24T05:51:53.635 INFO:teuthology.orchestra.run.smithi123.stdout:509 2024-04-24T05:51:53.635 INFO:teuthology.orchestra.run.smithi123.stdout:509.out 2024-04-24T05:51:53.635 INFO:teuthology.orchestra.run.smithi123.stdout:510 2024-04-24T05:51:53.635 INFO:teuthology.orchestra.run.smithi123.stdout:510.out 2024-04-24T05:51:53.635 INFO:teuthology.orchestra.run.smithi123.stdout:511 2024-04-24T05:51:53.635 INFO:teuthology.orchestra.run.smithi123.stdout:511.out 2024-04-24T05:51:53.635 INFO:teuthology.orchestra.run.smithi123.stdout:512 2024-04-24T05:51:53.635 INFO:teuthology.orchestra.run.smithi123.stdout:512.out 2024-04-24T05:51:53.635 INFO:teuthology.orchestra.run.smithi123.stdout:513 2024-04-24T05:51:53.636 INFO:teuthology.orchestra.run.smithi123.stdout:513.out 2024-04-24T05:51:53.636 INFO:teuthology.orchestra.run.smithi123.stdout:514 2024-04-24T05:51:53.636 INFO:teuthology.orchestra.run.smithi123.stdout:514.out 2024-04-24T05:51:53.636 INFO:teuthology.orchestra.run.smithi123.stdout:515 2024-04-24T05:51:53.636 INFO:teuthology.orchestra.run.smithi123.stdout:515.out 2024-04-24T05:51:53.636 INFO:teuthology.orchestra.run.smithi123.stdout:516 2024-04-24T05:51:53.636 INFO:teuthology.orchestra.run.smithi123.stdout:516.out 2024-04-24T05:51:53.636 INFO:teuthology.orchestra.run.smithi123.stdout:517 2024-04-24T05:51:53.636 INFO:teuthology.orchestra.run.smithi123.stdout:517.out 2024-04-24T05:51:53.636 INFO:teuthology.orchestra.run.smithi123.stdout:518 2024-04-24T05:51:53.636 INFO:teuthology.orchestra.run.smithi123.stdout:518.out 2024-04-24T05:51:53.636 INFO:teuthology.orchestra.run.smithi123.stdout:519 2024-04-24T05:51:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:519.out 2024-04-24T05:51:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:520 2024-04-24T05:51:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:520.out 2024-04-24T05:51:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:521 2024-04-24T05:51:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:521.out 2024-04-24T05:51:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:522 2024-04-24T05:51:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:522.out 2024-04-24T05:51:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:523 2024-04-24T05:51:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:523.out 2024-04-24T05:51:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:524 2024-04-24T05:51:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:524.out 2024-04-24T05:51:53.638 INFO:teuthology.orchestra.run.smithi123.stdout:525 2024-04-24T05:51:53.638 INFO:teuthology.orchestra.run.smithi123.stdout:525.out 2024-04-24T05:51:53.638 INFO:teuthology.orchestra.run.smithi123.stdout:526 2024-04-24T05:51:53.638 INFO:teuthology.orchestra.run.smithi123.stdout:526.out 2024-04-24T05:51:53.638 INFO:teuthology.orchestra.run.smithi123.stdout:527 2024-04-24T05:51:53.638 INFO:teuthology.orchestra.run.smithi123.stdout:527.out 2024-04-24T05:51:53.638 INFO:teuthology.orchestra.run.smithi123.stdout:528 2024-04-24T05:51:53.638 INFO:teuthology.orchestra.run.smithi123.stdout:528.out 2024-04-24T05:51:53.638 INFO:teuthology.orchestra.run.smithi123.stdout:529 2024-04-24T05:51:53.638 INFO:teuthology.orchestra.run.smithi123.stdout:529.out 2024-04-24T05:51:53.638 INFO:teuthology.orchestra.run.smithi123.stdout:530 2024-04-24T05:51:53.638 INFO:teuthology.orchestra.run.smithi123.stdout:530.out 2024-04-24T05:51:53.639 INFO:teuthology.orchestra.run.smithi123.stdout:531 2024-04-24T05:51:53.639 INFO:teuthology.orchestra.run.smithi123.stdout:531.out 2024-04-24T05:51:53.639 INFO:teuthology.orchestra.run.smithi123.stdout:532 2024-04-24T05:51:53.639 INFO:teuthology.orchestra.run.smithi123.stdout:532.out 2024-04-24T05:51:53.639 INFO:teuthology.orchestra.run.smithi123.stdout:533 2024-04-24T05:51:53.639 INFO:teuthology.orchestra.run.smithi123.stdout:533.out 2024-04-24T05:51:53.639 INFO:teuthology.orchestra.run.smithi123.stdout:534 2024-04-24T05:51:53.639 INFO:teuthology.orchestra.run.smithi123.stdout:534.out 2024-04-24T05:51:53.639 INFO:teuthology.orchestra.run.smithi123.stdout:535 2024-04-24T05:51:53.639 INFO:teuthology.orchestra.run.smithi123.stdout:535.out 2024-04-24T05:51:53.639 INFO:teuthology.orchestra.run.smithi123.stdout:536 2024-04-24T05:51:53.639 INFO:teuthology.orchestra.run.smithi123.stdout:536.out 2024-04-24T05:51:53.640 INFO:teuthology.orchestra.run.smithi123.stdout:537 2024-04-24T05:51:53.640 INFO:teuthology.orchestra.run.smithi123.stdout:537.out 2024-04-24T05:51:53.640 INFO:teuthology.orchestra.run.smithi123.stdout:538 2024-04-24T05:51:53.640 INFO:teuthology.orchestra.run.smithi123.stdout:538.out 2024-04-24T05:51:53.640 INFO:teuthology.orchestra.run.smithi123.stdout:539 2024-04-24T05:51:53.640 INFO:teuthology.orchestra.run.smithi123.stdout:539.out 2024-04-24T05:51:53.640 INFO:teuthology.orchestra.run.smithi123.stdout:540 2024-04-24T05:51:53.640 INFO:teuthology.orchestra.run.smithi123.stdout:540.out 2024-04-24T05:51:53.640 INFO:teuthology.orchestra.run.smithi123.stdout:541 2024-04-24T05:51:53.640 INFO:teuthology.orchestra.run.smithi123.stdout:541.out 2024-04-24T05:51:53.640 INFO:teuthology.orchestra.run.smithi123.stdout:542 2024-04-24T05:51:53.640 INFO:teuthology.orchestra.run.smithi123.stdout:542.out 2024-04-24T05:51:53.641 INFO:teuthology.orchestra.run.smithi123.stdout:543 2024-04-24T05:51:53.641 INFO:teuthology.orchestra.run.smithi123.stdout:543.out 2024-04-24T05:51:53.641 INFO:teuthology.orchestra.run.smithi123.stdout:544 2024-04-24T05:51:53.641 INFO:teuthology.orchestra.run.smithi123.stdout:544.out 2024-04-24T05:51:53.641 INFO:teuthology.orchestra.run.smithi123.stdout:545 2024-04-24T05:51:53.641 INFO:teuthology.orchestra.run.smithi123.stdout:545.out 2024-04-24T05:51:53.641 INFO:teuthology.orchestra.run.smithi123.stdout:546 2024-04-24T05:51:53.641 INFO:teuthology.orchestra.run.smithi123.stdout:546.out 2024-04-24T05:51:53.641 INFO:teuthology.orchestra.run.smithi123.stdout:547 2024-04-24T05:51:53.641 INFO:teuthology.orchestra.run.smithi123.stdout:547.out 2024-04-24T05:51:53.641 INFO:teuthology.orchestra.run.smithi123.stdout:548 2024-04-24T05:51:53.642 INFO:teuthology.orchestra.run.smithi123.stdout:548.out 2024-04-24T05:51:53.642 INFO:teuthology.orchestra.run.smithi123.stdout:549 2024-04-24T05:51:53.642 INFO:teuthology.orchestra.run.smithi123.stdout:549.out 2024-04-24T05:51:53.642 INFO:teuthology.orchestra.run.smithi123.stdout:550 2024-04-24T05:51:53.642 INFO:teuthology.orchestra.run.smithi123.stdout:550.out 2024-04-24T05:51:53.642 INFO:teuthology.orchestra.run.smithi123.stdout:551 2024-04-24T05:51:53.642 INFO:teuthology.orchestra.run.smithi123.stdout:551.out 2024-04-24T05:51:53.642 INFO:teuthology.orchestra.run.smithi123.stdout:552 2024-04-24T05:51:53.642 INFO:teuthology.orchestra.run.smithi123.stdout:552.out 2024-04-24T05:51:53.642 INFO:teuthology.orchestra.run.smithi123.stdout:553 2024-04-24T05:51:53.642 INFO:teuthology.orchestra.run.smithi123.stdout:553.out 2024-04-24T05:51:53.643 INFO:teuthology.orchestra.run.smithi123.stdout:554 2024-04-24T05:51:53.643 INFO:teuthology.orchestra.run.smithi123.stdout:554.out 2024-04-24T05:51:53.643 INFO:teuthology.orchestra.run.smithi123.stdout:555 2024-04-24T05:51:53.643 INFO:teuthology.orchestra.run.smithi123.stdout:555.out 2024-04-24T05:51:53.643 INFO:teuthology.orchestra.run.smithi123.stdout:556 2024-04-24T05:51:53.643 INFO:teuthology.orchestra.run.smithi123.stdout:556.out 2024-04-24T05:51:53.643 INFO:teuthology.orchestra.run.smithi123.stdout:557 2024-04-24T05:51:53.643 INFO:teuthology.orchestra.run.smithi123.stdout:557.out 2024-04-24T05:51:53.643 INFO:teuthology.orchestra.run.smithi123.stdout:558 2024-04-24T05:51:53.643 INFO:teuthology.orchestra.run.smithi123.stdout:558.out 2024-04-24T05:51:53.643 INFO:teuthology.orchestra.run.smithi123.stdout:559 2024-04-24T05:51:53.643 INFO:teuthology.orchestra.run.smithi123.stdout:559.out 2024-04-24T05:51:53.643 INFO:teuthology.orchestra.run.smithi123.stdout:560 2024-04-24T05:51:53.644 INFO:teuthology.orchestra.run.smithi123.stdout:560.out 2024-04-24T05:51:53.644 INFO:teuthology.orchestra.run.smithi123.stdout:561 2024-04-24T05:51:53.644 INFO:teuthology.orchestra.run.smithi123.stdout:561.out 2024-04-24T05:51:53.644 INFO:teuthology.orchestra.run.smithi123.stdout:562 2024-04-24T05:51:53.644 INFO:teuthology.orchestra.run.smithi123.stdout:562.out 2024-04-24T05:51:53.644 INFO:teuthology.orchestra.run.smithi123.stdout:563 2024-04-24T05:51:53.644 INFO:teuthology.orchestra.run.smithi123.stdout:563.out 2024-04-24T05:51:53.644 INFO:teuthology.orchestra.run.smithi123.stdout:564 2024-04-24T05:51:53.644 INFO:teuthology.orchestra.run.smithi123.stdout:564.out 2024-04-24T05:51:53.644 INFO:teuthology.orchestra.run.smithi123.stdout:565 2024-04-24T05:51:53.644 INFO:teuthology.orchestra.run.smithi123.stdout:565.out 2024-04-24T05:51:53.644 INFO:teuthology.orchestra.run.smithi123.stdout:566 2024-04-24T05:51:53.644 INFO:teuthology.orchestra.run.smithi123.stdout:566.out 2024-04-24T05:51:53.645 INFO:teuthology.orchestra.run.smithi123.stdout:567 2024-04-24T05:51:53.645 INFO:teuthology.orchestra.run.smithi123.stdout:567.out 2024-04-24T05:51:53.645 INFO:teuthology.orchestra.run.smithi123.stdout:568 2024-04-24T05:51:53.645 INFO:teuthology.orchestra.run.smithi123.stdout:568.out 2024-04-24T05:51:53.645 INFO:teuthology.orchestra.run.smithi123.stdout:569 2024-04-24T05:51:53.645 INFO:teuthology.orchestra.run.smithi123.stdout:569.out 2024-04-24T05:51:53.645 INFO:teuthology.orchestra.run.smithi123.stdout:570 2024-04-24T05:51:53.645 INFO:teuthology.orchestra.run.smithi123.stdout:570.out 2024-04-24T05:51:53.645 INFO:teuthology.orchestra.run.smithi123.stdout:571 2024-04-24T05:51:53.645 INFO:teuthology.orchestra.run.smithi123.stdout:571.out 2024-04-24T05:51:53.645 INFO:teuthology.orchestra.run.smithi123.stdout:572 2024-04-24T05:51:53.645 INFO:teuthology.orchestra.run.smithi123.stdout:572.out 2024-04-24T05:51:53.645 INFO:teuthology.orchestra.run.smithi123.stdout:573 2024-04-24T05:51:53.646 INFO:teuthology.orchestra.run.smithi123.stdout:573.out 2024-04-24T05:51:53.646 INFO:teuthology.orchestra.run.smithi123.stdout:574 2024-04-24T05:51:53.646 INFO:teuthology.orchestra.run.smithi123.stdout:574.out 2024-04-24T05:51:53.646 INFO:teuthology.orchestra.run.smithi123.stdout:575 2024-04-24T05:51:53.646 INFO:teuthology.orchestra.run.smithi123.stdout:575.out 2024-04-24T05:51:53.646 INFO:teuthology.orchestra.run.smithi123.stdout:576 2024-04-24T05:51:53.646 INFO:teuthology.orchestra.run.smithi123.stdout:576.out 2024-04-24T05:51:53.646 INFO:teuthology.orchestra.run.smithi123.stdout:577 2024-04-24T05:51:53.646 INFO:teuthology.orchestra.run.smithi123.stdout:577.out 2024-04-24T05:51:53.646 INFO:teuthology.orchestra.run.smithi123.stdout:578 2024-04-24T05:51:53.646 INFO:teuthology.orchestra.run.smithi123.stdout:578.out 2024-04-24T05:51:53.647 INFO:teuthology.orchestra.run.smithi123.stdout:579 2024-04-24T05:51:53.647 INFO:teuthology.orchestra.run.smithi123.stdout:579.out 2024-04-24T05:51:53.647 INFO:teuthology.orchestra.run.smithi123.stdout:580 2024-04-24T05:51:53.647 INFO:teuthology.orchestra.run.smithi123.stdout:580.out 2024-04-24T05:51:53.647 INFO:teuthology.orchestra.run.smithi123.stdout:581 2024-04-24T05:51:53.647 INFO:teuthology.orchestra.run.smithi123.stdout:581.out 2024-04-24T05:51:53.647 INFO:teuthology.orchestra.run.smithi123.stdout:582 2024-04-24T05:51:53.647 INFO:teuthology.orchestra.run.smithi123.stdout:582.out 2024-04-24T05:51:53.647 INFO:teuthology.orchestra.run.smithi123.stdout:583 2024-04-24T05:51:53.647 INFO:teuthology.orchestra.run.smithi123.stdout:583.out 2024-04-24T05:51:53.647 INFO:teuthology.orchestra.run.smithi123.stdout:584 2024-04-24T05:51:53.647 INFO:teuthology.orchestra.run.smithi123.stdout:584.out 2024-04-24T05:51:53.647 INFO:teuthology.orchestra.run.smithi123.stdout:585 2024-04-24T05:51:53.648 INFO:teuthology.orchestra.run.smithi123.stdout:585.out 2024-04-24T05:51:53.648 INFO:teuthology.orchestra.run.smithi123.stdout:586 2024-04-24T05:51:53.648 INFO:teuthology.orchestra.run.smithi123.stdout:586.out 2024-04-24T05:51:53.648 INFO:teuthology.orchestra.run.smithi123.stdout:587 2024-04-24T05:51:53.648 INFO:teuthology.orchestra.run.smithi123.stdout:587.out 2024-04-24T05:51:53.648 INFO:teuthology.orchestra.run.smithi123.stdout:588 2024-04-24T05:51:53.648 INFO:teuthology.orchestra.run.smithi123.stdout:588.out 2024-04-24T05:51:53.648 INFO:teuthology.orchestra.run.smithi123.stdout:589 2024-04-24T05:51:53.648 INFO:teuthology.orchestra.run.smithi123.stdout:589.out 2024-04-24T05:51:53.648 INFO:teuthology.orchestra.run.smithi123.stdout:590 2024-04-24T05:51:53.648 INFO:teuthology.orchestra.run.smithi123.stdout:590.out 2024-04-24T05:51:53.648 INFO:teuthology.orchestra.run.smithi123.stdout:591 2024-04-24T05:51:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:591.out 2024-04-24T05:51:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:592 2024-04-24T05:51:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:592.out 2024-04-24T05:51:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:593 2024-04-24T05:51:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:593.out 2024-04-24T05:51:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:594 2024-04-24T05:51:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:594.out 2024-04-24T05:51:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:595 2024-04-24T05:51:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:595.out 2024-04-24T05:51:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:596 2024-04-24T05:51:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:596.out 2024-04-24T05:51:53.650 INFO:teuthology.orchestra.run.smithi123.stdout:597 2024-04-24T05:51:53.650 INFO:teuthology.orchestra.run.smithi123.stdout:597.out 2024-04-24T05:51:53.650 INFO:teuthology.orchestra.run.smithi123.stdout:598 2024-04-24T05:51:53.650 INFO:teuthology.orchestra.run.smithi123.stdout:598.out 2024-04-24T05:51:53.650 INFO:teuthology.orchestra.run.smithi123.stdout:599 2024-04-24T05:51:53.650 INFO:teuthology.orchestra.run.smithi123.stdout:599.out 2024-04-24T05:51:53.650 INFO:teuthology.orchestra.run.smithi123.stdout:600 2024-04-24T05:51:53.650 INFO:teuthology.orchestra.run.smithi123.stdout:600.out 2024-04-24T05:51:53.650 INFO:teuthology.orchestra.run.smithi123.stdout:601 2024-04-24T05:51:53.650 INFO:teuthology.orchestra.run.smithi123.stdout:601.out 2024-04-24T05:51:53.650 INFO:teuthology.orchestra.run.smithi123.stdout:602 2024-04-24T05:51:53.650 INFO:teuthology.orchestra.run.smithi123.stdout:602.out 2024-04-24T05:51:53.651 INFO:teuthology.orchestra.run.smithi123.stdout:603 2024-04-24T05:51:53.651 INFO:teuthology.orchestra.run.smithi123.stdout:603.out 2024-04-24T05:51:53.651 INFO:teuthology.orchestra.run.smithi123.stdout:604 2024-04-24T05:51:53.651 INFO:teuthology.orchestra.run.smithi123.stdout:604.out 2024-04-24T05:51:53.651 INFO:teuthology.orchestra.run.smithi123.stdout:605 2024-04-24T05:51:53.651 INFO:teuthology.orchestra.run.smithi123.stdout:605.out 2024-04-24T05:51:53.651 INFO:teuthology.orchestra.run.smithi123.stdout:606 2024-04-24T05:51:53.651 INFO:teuthology.orchestra.run.smithi123.stdout:606.out 2024-04-24T05:51:53.651 INFO:teuthology.orchestra.run.smithi123.stdout:607 2024-04-24T05:51:53.651 INFO:teuthology.orchestra.run.smithi123.stdout:607.out 2024-04-24T05:51:53.652 INFO:teuthology.orchestra.run.smithi123.stdout:608 2024-04-24T05:51:53.652 INFO:teuthology.orchestra.run.smithi123.stdout:608.out 2024-04-24T05:51:53.652 INFO:teuthology.orchestra.run.smithi123.stdout:609 2024-04-24T05:51:53.652 INFO:teuthology.orchestra.run.smithi123.stdout:609.out 2024-04-24T05:51:53.652 INFO:teuthology.orchestra.run.smithi123.stdout:610 2024-04-24T05:51:53.652 INFO:teuthology.orchestra.run.smithi123.stdout:610.out 2024-04-24T05:51:53.652 INFO:teuthology.orchestra.run.smithi123.stdout:611 2024-04-24T05:51:53.652 INFO:teuthology.orchestra.run.smithi123.stdout:611.out 2024-04-24T05:51:53.652 INFO:teuthology.orchestra.run.smithi123.stdout:612 2024-04-24T05:51:53.652 INFO:teuthology.orchestra.run.smithi123.stdout:612.out 2024-04-24T05:51:53.652 INFO:teuthology.orchestra.run.smithi123.stdout:613 2024-04-24T05:51:53.652 INFO:teuthology.orchestra.run.smithi123.stdout:613.out 2024-04-24T05:51:53.653 INFO:teuthology.orchestra.run.smithi123.stdout:614 2024-04-24T05:51:53.653 INFO:teuthology.orchestra.run.smithi123.stdout:614.out 2024-04-24T05:51:53.653 INFO:teuthology.orchestra.run.smithi123.stdout:615 2024-04-24T05:51:53.653 INFO:teuthology.orchestra.run.smithi123.stdout:615.out 2024-04-24T05:51:53.653 INFO:teuthology.orchestra.run.smithi123.stdout:616 2024-04-24T05:51:53.653 INFO:teuthology.orchestra.run.smithi123.stdout:616.out 2024-04-24T05:51:53.653 INFO:teuthology.orchestra.run.smithi123.stdout:617 2024-04-24T05:51:53.653 INFO:teuthology.orchestra.run.smithi123.stdout:617.out 2024-04-24T05:51:53.653 INFO:teuthology.orchestra.run.smithi123.stdout:618 2024-04-24T05:51:53.653 INFO:teuthology.orchestra.run.smithi123.stdout:618.out 2024-04-24T05:51:53.653 INFO:teuthology.orchestra.run.smithi123.stdout:619 2024-04-24T05:51:53.653 INFO:teuthology.orchestra.run.smithi123.stdout:619.out 2024-04-24T05:51:53.654 INFO:teuthology.orchestra.run.smithi123.stdout:620 2024-04-24T05:51:53.654 INFO:teuthology.orchestra.run.smithi123.stdout:620.out 2024-04-24T05:51:53.654 INFO:teuthology.orchestra.run.smithi123.stdout:621 2024-04-24T05:51:53.654 INFO:teuthology.orchestra.run.smithi123.stdout:621.out 2024-04-24T05:51:53.654 INFO:teuthology.orchestra.run.smithi123.stdout:622 2024-04-24T05:51:53.654 INFO:teuthology.orchestra.run.smithi123.stdout:622.out 2024-04-24T05:51:53.654 INFO:teuthology.orchestra.run.smithi123.stdout:623 2024-04-24T05:51:53.654 INFO:teuthology.orchestra.run.smithi123.stdout:623.out 2024-04-24T05:51:53.654 INFO:teuthology.orchestra.run.smithi123.stdout:624 2024-04-24T05:51:53.654 INFO:teuthology.orchestra.run.smithi123.stdout:624.out 2024-04-24T05:51:53.654 INFO:teuthology.orchestra.run.smithi123.stdout:625 2024-04-24T05:51:53.654 INFO:teuthology.orchestra.run.smithi123.stdout:625.out 2024-04-24T05:51:53.655 INFO:teuthology.orchestra.run.smithi123.stdout:626 2024-04-24T05:51:53.655 INFO:teuthology.orchestra.run.smithi123.stdout:626.out 2024-04-24T05:51:53.655 INFO:teuthology.orchestra.run.smithi123.stdout:627 2024-04-24T05:51:53.655 INFO:teuthology.orchestra.run.smithi123.stdout:627.out 2024-04-24T05:51:53.655 INFO:teuthology.orchestra.run.smithi123.stdout:628 2024-04-24T05:51:53.655 INFO:teuthology.orchestra.run.smithi123.stdout:628.out 2024-04-24T05:51:53.655 INFO:teuthology.orchestra.run.smithi123.stdout:629 2024-04-24T05:51:53.655 INFO:teuthology.orchestra.run.smithi123.stdout:629.out 2024-04-24T05:51:53.655 INFO:teuthology.orchestra.run.smithi123.stdout:630 2024-04-24T05:51:53.655 INFO:teuthology.orchestra.run.smithi123.stdout:630.out 2024-04-24T05:51:53.655 INFO:teuthology.orchestra.run.smithi123.stdout:631 2024-04-24T05:51:53.655 INFO:teuthology.orchestra.run.smithi123.stdout:631.out 2024-04-24T05:51:53.655 INFO:teuthology.orchestra.run.smithi123.stdout:632 2024-04-24T05:51:53.656 INFO:teuthology.orchestra.run.smithi123.stdout:632.out 2024-04-24T05:51:53.656 INFO:teuthology.orchestra.run.smithi123.stdout:633 2024-04-24T05:51:53.656 INFO:teuthology.orchestra.run.smithi123.stdout:633.out 2024-04-24T05:51:53.656 INFO:teuthology.orchestra.run.smithi123.stdout:634 2024-04-24T05:51:53.656 INFO:teuthology.orchestra.run.smithi123.stdout:634.out 2024-04-24T05:51:53.656 INFO:teuthology.orchestra.run.smithi123.stdout:635 2024-04-24T05:51:53.656 INFO:teuthology.orchestra.run.smithi123.stdout:635.out 2024-04-24T05:51:53.656 INFO:teuthology.orchestra.run.smithi123.stdout:636 2024-04-24T05:51:53.656 INFO:teuthology.orchestra.run.smithi123.stdout:636.out 2024-04-24T05:51:53.656 INFO:teuthology.orchestra.run.smithi123.stdout:637 2024-04-24T05:51:53.656 INFO:teuthology.orchestra.run.smithi123.stdout:637.out 2024-04-24T05:51:53.656 INFO:teuthology.orchestra.run.smithi123.stdout:638 2024-04-24T05:51:53.656 INFO:teuthology.orchestra.run.smithi123.stdout:638.out 2024-04-24T05:51:53.657 INFO:teuthology.orchestra.run.smithi123.stdout:639 2024-04-24T05:51:53.657 INFO:teuthology.orchestra.run.smithi123.stdout:639.out 2024-04-24T05:51:53.657 INFO:teuthology.orchestra.run.smithi123.stdout:640 2024-04-24T05:51:53.657 INFO:teuthology.orchestra.run.smithi123.stdout:640.out 2024-04-24T05:51:53.657 INFO:teuthology.orchestra.run.smithi123.stdout:641 2024-04-24T05:51:53.657 INFO:teuthology.orchestra.run.smithi123.stdout:641.out 2024-04-24T05:51:53.657 INFO:teuthology.orchestra.run.smithi123.stdout:642 2024-04-24T05:51:53.657 INFO:teuthology.orchestra.run.smithi123.stdout:642.out 2024-04-24T05:51:53.657 INFO:teuthology.orchestra.run.smithi123.stdout:643 2024-04-24T05:51:53.657 INFO:teuthology.orchestra.run.smithi123.stdout:643.out 2024-04-24T05:51:53.657 INFO:teuthology.orchestra.run.smithi123.stdout:644 2024-04-24T05:51:53.657 INFO:teuthology.orchestra.run.smithi123.stdout:644.out 2024-04-24T05:51:53.657 INFO:teuthology.orchestra.run.smithi123.stdout:645 2024-04-24T05:51:53.657 INFO:teuthology.orchestra.run.smithi123.stdout:645.out 2024-04-24T05:51:53.658 INFO:teuthology.orchestra.run.smithi123.stdout:646 2024-04-24T05:51:53.658 INFO:teuthology.orchestra.run.smithi123.stdout:646.out 2024-04-24T05:51:53.658 INFO:teuthology.orchestra.run.smithi123.stdout:647 2024-04-24T05:51:53.658 INFO:teuthology.orchestra.run.smithi123.stdout:647.out 2024-04-24T05:51:53.658 INFO:teuthology.orchestra.run.smithi123.stdout:648 2024-04-24T05:51:53.658 INFO:teuthology.orchestra.run.smithi123.stdout:648.out 2024-04-24T05:51:53.658 INFO:teuthology.orchestra.run.smithi123.stdout:649 2024-04-24T05:51:53.658 INFO:teuthology.orchestra.run.smithi123.stdout:649.out 2024-04-24T05:51:53.658 INFO:teuthology.orchestra.run.smithi123.stdout:650 2024-04-24T05:51:53.658 INFO:teuthology.orchestra.run.smithi123.stdout:650.out 2024-04-24T05:51:53.658 INFO:teuthology.orchestra.run.smithi123.stdout:651 2024-04-24T05:51:53.659 INFO:teuthology.orchestra.run.smithi123.stdout:651.out 2024-04-24T05:51:53.659 INFO:teuthology.orchestra.run.smithi123.stdout:652 2024-04-24T05:51:53.659 INFO:teuthology.orchestra.run.smithi123.stdout:652.out 2024-04-24T05:51:53.659 INFO:teuthology.orchestra.run.smithi123.stdout:653 2024-04-24T05:51:53.659 INFO:teuthology.orchestra.run.smithi123.stdout:653.out 2024-04-24T05:51:53.659 INFO:teuthology.orchestra.run.smithi123.stdout:654 2024-04-24T05:51:53.659 INFO:teuthology.orchestra.run.smithi123.stdout:654.out 2024-04-24T05:51:53.659 INFO:teuthology.orchestra.run.smithi123.stdout:655 2024-04-24T05:51:53.659 INFO:teuthology.orchestra.run.smithi123.stdout:655.out 2024-04-24T05:51:53.659 INFO:teuthology.orchestra.run.smithi123.stdout:656 2024-04-24T05:51:53.659 INFO:teuthology.orchestra.run.smithi123.stdout:656.out 2024-04-24T05:51:53.659 INFO:teuthology.orchestra.run.smithi123.stdout:657 2024-04-24T05:51:53.659 INFO:teuthology.orchestra.run.smithi123.stdout:657.out 2024-04-24T05:51:53.659 INFO:teuthology.orchestra.run.smithi123.stdout:658 2024-04-24T05:51:53.660 INFO:teuthology.orchestra.run.smithi123.stdout:658.out 2024-04-24T05:51:53.660 INFO:teuthology.orchestra.run.smithi123.stdout:659 2024-04-24T05:51:53.660 INFO:teuthology.orchestra.run.smithi123.stdout:659.out 2024-04-24T05:51:53.660 INFO:teuthology.orchestra.run.smithi123.stdout:660 2024-04-24T05:51:53.660 INFO:teuthology.orchestra.run.smithi123.stdout:660.out 2024-04-24T05:51:53.660 INFO:teuthology.orchestra.run.smithi123.stdout:661 2024-04-24T05:51:53.660 INFO:teuthology.orchestra.run.smithi123.stdout:661.out 2024-04-24T05:51:53.660 INFO:teuthology.orchestra.run.smithi123.stdout:662 2024-04-24T05:51:53.660 INFO:teuthology.orchestra.run.smithi123.stdout:662.out 2024-04-24T05:51:53.660 INFO:teuthology.orchestra.run.smithi123.stdout:663 2024-04-24T05:51:53.660 INFO:teuthology.orchestra.run.smithi123.stdout:663.out 2024-04-24T05:51:53.660 INFO:teuthology.orchestra.run.smithi123.stdout:664 2024-04-24T05:51:53.660 INFO:teuthology.orchestra.run.smithi123.stdout:664.out 2024-04-24T05:51:53.661 INFO:teuthology.orchestra.run.smithi123.stdout:665 2024-04-24T05:51:53.661 INFO:teuthology.orchestra.run.smithi123.stdout:665.out 2024-04-24T05:51:53.661 INFO:teuthology.orchestra.run.smithi123.stdout:666 2024-04-24T05:51:53.661 INFO:teuthology.orchestra.run.smithi123.stdout:666.out 2024-04-24T05:51:53.661 INFO:teuthology.orchestra.run.smithi123.stdout:667 2024-04-24T05:51:53.661 INFO:teuthology.orchestra.run.smithi123.stdout:667.out 2024-04-24T05:51:53.661 INFO:teuthology.orchestra.run.smithi123.stdout:668 2024-04-24T05:51:53.661 INFO:teuthology.orchestra.run.smithi123.stdout:668.out 2024-04-24T05:51:53.661 INFO:teuthology.orchestra.run.smithi123.stdout:669 2024-04-24T05:51:53.661 INFO:teuthology.orchestra.run.smithi123.stdout:669.out 2024-04-24T05:51:53.661 INFO:teuthology.orchestra.run.smithi123.stdout:670 2024-04-24T05:51:53.661 INFO:teuthology.orchestra.run.smithi123.stdout:670.out 2024-04-24T05:51:53.661 INFO:teuthology.orchestra.run.smithi123.stdout:671 2024-04-24T05:51:53.662 INFO:teuthology.orchestra.run.smithi123.stdout:671.out 2024-04-24T05:51:53.662 INFO:teuthology.orchestra.run.smithi123.stdout:672 2024-04-24T05:51:53.662 INFO:teuthology.orchestra.run.smithi123.stdout:672.out 2024-04-24T05:51:53.662 INFO:teuthology.orchestra.run.smithi123.stdout:673 2024-04-24T05:51:53.662 INFO:teuthology.orchestra.run.smithi123.stdout:673.out 2024-04-24T05:51:53.662 INFO:teuthology.orchestra.run.smithi123.stdout:674 2024-04-24T05:51:53.662 INFO:teuthology.orchestra.run.smithi123.stdout:674.out 2024-04-24T05:51:53.662 INFO:teuthology.orchestra.run.smithi123.stdout:675 2024-04-24T05:51:53.662 INFO:teuthology.orchestra.run.smithi123.stdout:675.out 2024-04-24T05:51:53.662 INFO:teuthology.orchestra.run.smithi123.stdout:676 2024-04-24T05:51:53.662 INFO:teuthology.orchestra.run.smithi123.stdout:676.out 2024-04-24T05:51:53.663 INFO:teuthology.orchestra.run.smithi123.stdout:677 2024-04-24T05:51:53.663 INFO:teuthology.orchestra.run.smithi123.stdout:677.out 2024-04-24T05:51:53.663 INFO:teuthology.orchestra.run.smithi123.stdout:678 2024-04-24T05:51:53.663 INFO:teuthology.orchestra.run.smithi123.stdout:678.out 2024-04-24T05:51:53.663 INFO:teuthology.orchestra.run.smithi123.stdout:679 2024-04-24T05:51:53.663 INFO:teuthology.orchestra.run.smithi123.stdout:679.out 2024-04-24T05:51:53.663 INFO:teuthology.orchestra.run.smithi123.stdout:680 2024-04-24T05:51:53.663 INFO:teuthology.orchestra.run.smithi123.stdout:680.out 2024-04-24T05:51:53.663 INFO:teuthology.orchestra.run.smithi123.stdout:681 2024-04-24T05:51:53.663 INFO:teuthology.orchestra.run.smithi123.stdout:681.out 2024-04-24T05:51:53.663 INFO:teuthology.orchestra.run.smithi123.stdout:682 2024-04-24T05:51:53.663 INFO:teuthology.orchestra.run.smithi123.stdout:682.out 2024-04-24T05:51:53.664 INFO:teuthology.orchestra.run.smithi123.stdout:683 2024-04-24T05:51:53.664 INFO:teuthology.orchestra.run.smithi123.stdout:683.out 2024-04-24T05:51:53.664 INFO:teuthology.orchestra.run.smithi123.stdout:684 2024-04-24T05:51:53.664 INFO:teuthology.orchestra.run.smithi123.stdout:684.out 2024-04-24T05:51:53.664 INFO:teuthology.orchestra.run.smithi123.stdout:685 2024-04-24T05:51:53.664 INFO:teuthology.orchestra.run.smithi123.stdout:685.out 2024-04-24T05:51:53.664 INFO:teuthology.orchestra.run.smithi123.stdout:686 2024-04-24T05:51:53.664 INFO:teuthology.orchestra.run.smithi123.stdout:686.out 2024-04-24T05:51:53.664 INFO:teuthology.orchestra.run.smithi123.stdout:687 2024-04-24T05:51:53.664 INFO:teuthology.orchestra.run.smithi123.stdout:687.out 2024-04-24T05:51:53.664 INFO:teuthology.orchestra.run.smithi123.stdout:688 2024-04-24T05:51:53.665 INFO:teuthology.orchestra.run.smithi123.stdout:688.out 2024-04-24T05:51:53.665 INFO:teuthology.orchestra.run.smithi123.stdout:689 2024-04-24T05:51:53.665 INFO:teuthology.orchestra.run.smithi123.stdout:689.out 2024-04-24T05:51:53.665 INFO:teuthology.orchestra.run.smithi123.stdout:690 2024-04-24T05:51:53.665 INFO:teuthology.orchestra.run.smithi123.stdout:690.out 2024-04-24T05:51:53.665 INFO:teuthology.orchestra.run.smithi123.stdout:691 2024-04-24T05:51:53.665 INFO:teuthology.orchestra.run.smithi123.stdout:691.out 2024-04-24T05:51:53.665 INFO:teuthology.orchestra.run.smithi123.stdout:692 2024-04-24T05:51:53.665 INFO:teuthology.orchestra.run.smithi123.stdout:692.out 2024-04-24T05:51:53.665 INFO:teuthology.orchestra.run.smithi123.stdout:693 2024-04-24T05:51:53.665 INFO:teuthology.orchestra.run.smithi123.stdout:693.out 2024-04-24T05:51:53.666 INFO:teuthology.orchestra.run.smithi123.stdout:694 2024-04-24T05:51:53.666 INFO:teuthology.orchestra.run.smithi123.stdout:694.out 2024-04-24T05:51:53.666 INFO:teuthology.orchestra.run.smithi123.stdout:695 2024-04-24T05:51:53.666 INFO:teuthology.orchestra.run.smithi123.stdout:695.out 2024-04-24T05:51:53.666 INFO:teuthology.orchestra.run.smithi123.stdout:696 2024-04-24T05:51:53.666 INFO:teuthology.orchestra.run.smithi123.stdout:696.out 2024-04-24T05:51:53.666 INFO:teuthology.orchestra.run.smithi123.stdout:697 2024-04-24T05:51:53.666 INFO:teuthology.orchestra.run.smithi123.stdout:697.out 2024-04-24T05:51:53.666 INFO:teuthology.orchestra.run.smithi123.stdout:698 2024-04-24T05:51:53.666 INFO:teuthology.orchestra.run.smithi123.stdout:698.out 2024-04-24T05:51:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:699 2024-04-24T05:51:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:699.out 2024-04-24T05:51:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:700 2024-04-24T05:51:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:700.out 2024-04-24T05:51:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:701 2024-04-24T05:51:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:701.out 2024-04-24T05:51:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:702 2024-04-24T05:51:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:702.out 2024-04-24T05:51:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:703 2024-04-24T05:51:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:703.out 2024-04-24T05:51:53.667 INFO:teuthology.orchestra.run.smithi123.stdout:704 2024-04-24T05:51:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:704.out 2024-04-24T05:51:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:705 2024-04-24T05:51:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:705.out 2024-04-24T05:51:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:706 2024-04-24T05:51:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:706.out 2024-04-24T05:51:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:707 2024-04-24T05:51:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:707.out 2024-04-24T05:51:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:708 2024-04-24T05:51:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:708.out 2024-04-24T05:51:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:709 2024-04-24T05:51:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:709.out 2024-04-24T05:51:53.668 INFO:teuthology.orchestra.run.smithi123.stdout:710 2024-04-24T05:51:53.669 INFO:teuthology.orchestra.run.smithi123.stdout:710.out 2024-04-24T05:51:53.669 INFO:teuthology.orchestra.run.smithi123.stdout:711 2024-04-24T05:51:53.669 INFO:teuthology.orchestra.run.smithi123.stdout:711.out 2024-04-24T05:51:53.669 INFO:teuthology.orchestra.run.smithi123.stdout:712 2024-04-24T05:51:53.669 INFO:teuthology.orchestra.run.smithi123.stdout:712.out 2024-04-24T05:51:53.669 INFO:teuthology.orchestra.run.smithi123.stdout:713 2024-04-24T05:51:53.669 INFO:teuthology.orchestra.run.smithi123.stdout:713.out 2024-04-24T05:51:53.669 INFO:teuthology.orchestra.run.smithi123.stdout:714 2024-04-24T05:51:53.669 INFO:teuthology.orchestra.run.smithi123.stdout:714.out 2024-04-24T05:51:53.669 INFO:teuthology.orchestra.run.smithi123.stdout:715 2024-04-24T05:51:53.669 INFO:teuthology.orchestra.run.smithi123.stdout:715.out 2024-04-24T05:51:53.669 INFO:teuthology.orchestra.run.smithi123.stdout:716 2024-04-24T05:51:53.669 INFO:teuthology.orchestra.run.smithi123.stdout:716.out 2024-04-24T05:51:53.670 INFO:teuthology.orchestra.run.smithi123.stdout:717 2024-04-24T05:51:53.670 INFO:teuthology.orchestra.run.smithi123.stdout:717.out 2024-04-24T05:51:53.670 INFO:teuthology.orchestra.run.smithi123.stdout:718 2024-04-24T05:51:53.670 INFO:teuthology.orchestra.run.smithi123.stdout:718.out 2024-04-24T05:51:53.670 INFO:teuthology.orchestra.run.smithi123.stdout:719 2024-04-24T05:51:53.670 INFO:teuthology.orchestra.run.smithi123.stdout:719.out 2024-04-24T05:51:53.670 INFO:teuthology.orchestra.run.smithi123.stdout:720 2024-04-24T05:51:53.670 INFO:teuthology.orchestra.run.smithi123.stdout:720.out 2024-04-24T05:51:53.670 INFO:teuthology.orchestra.run.smithi123.stdout:721 2024-04-24T05:51:53.670 INFO:teuthology.orchestra.run.smithi123.stdout:721.out 2024-04-24T05:51:53.670 INFO:teuthology.orchestra.run.smithi123.stdout:722 2024-04-24T05:51:53.671 INFO:teuthology.orchestra.run.smithi123.stdout:722.out 2024-04-24T05:51:53.671 INFO:teuthology.orchestra.run.smithi123.stdout:723 2024-04-24T05:51:53.671 INFO:teuthology.orchestra.run.smithi123.stdout:723.out 2024-04-24T05:51:53.671 INFO:teuthology.orchestra.run.smithi123.stdout:724 2024-04-24T05:51:53.671 INFO:teuthology.orchestra.run.smithi123.stdout:724.out 2024-04-24T05:51:53.671 INFO:teuthology.orchestra.run.smithi123.stdout:725 2024-04-24T05:51:53.671 INFO:teuthology.orchestra.run.smithi123.stdout:725.out 2024-04-24T05:51:53.671 INFO:teuthology.orchestra.run.smithi123.stdout:726 2024-04-24T05:51:53.671 INFO:teuthology.orchestra.run.smithi123.stdout:726.out 2024-04-24T05:51:53.671 INFO:teuthology.orchestra.run.smithi123.stdout:727 2024-04-24T05:51:53.671 INFO:teuthology.orchestra.run.smithi123.stdout:727.out 2024-04-24T05:51:53.671 INFO:teuthology.orchestra.run.smithi123.stdout:728 2024-04-24T05:51:53.672 INFO:teuthology.orchestra.run.smithi123.stdout:728.out 2024-04-24T05:51:53.672 INFO:teuthology.orchestra.run.smithi123.stdout:729 2024-04-24T05:51:53.672 INFO:teuthology.orchestra.run.smithi123.stdout:729.out 2024-04-24T05:51:53.672 INFO:teuthology.orchestra.run.smithi123.stdout:730 2024-04-24T05:51:53.672 INFO:teuthology.orchestra.run.smithi123.stdout:730.out 2024-04-24T05:51:53.672 INFO:teuthology.orchestra.run.smithi123.stdout:731 2024-04-24T05:51:53.672 INFO:teuthology.orchestra.run.smithi123.stdout:731.out 2024-04-24T05:51:53.672 INFO:teuthology.orchestra.run.smithi123.stdout:732 2024-04-24T05:51:53.672 INFO:teuthology.orchestra.run.smithi123.stdout:732.out 2024-04-24T05:51:53.672 INFO:teuthology.orchestra.run.smithi123.stdout:733 2024-04-24T05:51:53.672 INFO:teuthology.orchestra.run.smithi123.stdout:733.out 2024-04-24T05:51:53.672 INFO:teuthology.orchestra.run.smithi123.stdout:734 2024-04-24T05:51:53.673 INFO:teuthology.orchestra.run.smithi123.stdout:734.out 2024-04-24T05:51:53.673 INFO:teuthology.orchestra.run.smithi123.stdout:735 2024-04-24T05:51:53.673 INFO:teuthology.orchestra.run.smithi123.stdout:735.out 2024-04-24T05:51:53.673 INFO:teuthology.orchestra.run.smithi123.stdout:736 2024-04-24T05:51:53.673 INFO:teuthology.orchestra.run.smithi123.stdout:736.out 2024-04-24T05:51:53.673 INFO:teuthology.orchestra.run.smithi123.stdout:737 2024-04-24T05:51:53.673 INFO:teuthology.orchestra.run.smithi123.stdout:737.out 2024-04-24T05:51:53.673 INFO:teuthology.orchestra.run.smithi123.stdout:738 2024-04-24T05:51:53.673 INFO:teuthology.orchestra.run.smithi123.stdout:738.out 2024-04-24T05:51:53.673 INFO:teuthology.orchestra.run.smithi123.stdout:739 2024-04-24T05:51:53.673 INFO:teuthology.orchestra.run.smithi123.stdout:739.out 2024-04-24T05:51:53.674 INFO:teuthology.orchestra.run.smithi123.stdout:741 2024-04-24T05:51:53.674 INFO:teuthology.orchestra.run.smithi123.stdout:741.out 2024-04-24T05:51:53.674 INFO:teuthology.orchestra.run.smithi123.stdout:742 2024-04-24T05:51:53.674 INFO:teuthology.orchestra.run.smithi123.stdout:742.out 2024-04-24T05:51:53.674 INFO:teuthology.orchestra.run.smithi123.stdout:743 2024-04-24T05:51:53.674 INFO:teuthology.orchestra.run.smithi123.stdout:743.out 2024-04-24T05:51:53.674 INFO:teuthology.orchestra.run.smithi123.stdout:group.list 2024-04-24T05:51:53.674 INFO:teuthology.orchestra.run.smithi123.stdout:Makefile 2024-04-24T05:51:53.676 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2024-04-24T05:51:54.180 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T05:51:54.192 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T05:51:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:51:55.025 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:51:55.031 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:52:02.325 INFO:teuthology.orchestra.run.smithi123.stdout:generic/001 8s 2024-04-24T05:52:02.341 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/001 2024-04-24T05:52:02.341 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T05:52:02.349 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:52:02.353 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:ceph-fuse[50115]: fuse finished with error 0 and tester_r 0 2024-04-24T05:52:02.381 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T05:52:02.381 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2024-04-24T05:52:02.910 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T05:52:02.923 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T05:52:02.923 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:52:04.024 INFO:teuthology.orchestra.run.smithi123.stdout:generic/002 2s 2024-04-24T05:52:04.041 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/002 2024-04-24T05:52:04.041 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T05:52:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:52:04.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T05:52:04.089 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2024-04-24T05:52:04.561 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T05:52:04.574 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T05:52:04.574 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:52:04.984 INFO:teuthology.orchestra.run.smithi123.stdout:generic/003 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T05:52:04.996 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/003 2024-04-24T05:52:04.997 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/003 2024-04-24T05:52:04.997 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T05:52:05.008 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:52:05.038 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T05:52:05.038 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2024-04-24T05:52:05.563 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T05:52:05.576 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T05:52:05.576 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:52:06.033 INFO:teuthology.orchestra.run.smithi123.stdout:generic/004 [not run] O_TMPFILE is not supported 2024-04-24T05:52:06.043 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/004 2024-04-24T05:52:06.043 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/004 2024-04-24T05:52:06.043 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T05:52:06.052 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:52:06.078 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T05:52:06.078 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2024-04-24T05:52:06.583 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T05:52:06.596 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T05:52:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:52:08.863 INFO:teuthology.orchestra.run.smithi123.stdout:generic/005 2s 2024-04-24T05:52:08.882 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/005 2024-04-24T05:52:08.882 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T05:52:08.888 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:52:08.920 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T05:52:08.920 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2024-04-24T05:52:09.402 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T05:52:09.414 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T05:52:09.414 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:52:25.172 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:52:25.182 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:52:55.310 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:52:55.319 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:53:25.448 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:53:25.454 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:53:34.378 INFO:teuthology.orchestra.run.smithi123.stdout:generic/006 85s 2024-04-24T05:53:34.392 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/006 2024-04-24T05:53:34.392 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T05:53:34.399 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:53:34.520 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T05:53:34.520 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2024-04-24T05:53:35.092 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T05:53:35.103 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T05:53:35.104 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:53:55.580 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:53:55.586 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:54:25.710 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:54:25.715 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:54:56.788 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:54:56.794 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:55:26.903 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:55:26.912 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:55:57.067 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:55:57.076 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:56:27.197 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:56:27.206 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:56:57.336 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:56:57.345 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:57:00.856 INFO:teuthology.orchestra.run.smithi123.stdout:generic/007 205s 2024-04-24T05:57:00.872 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/007 2024-04-24T05:57:00.872 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T05:57:00.880 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:57:00.918 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T05:57:00.918 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2024-04-24T05:57:01.419 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T05:57:01.431 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T05:57:01.431 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:57:03.619 INFO:teuthology.orchestra.run.smithi123.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-24T05:57:03.630 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/008 2024-04-24T05:57:03.631 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/008 2024-04-24T05:57:03.631 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T05:57:03.639 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:57:03.674 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T05:57:03.674 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2024-04-24T05:57:04.200 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T05:57:04.212 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T05:57:04.213 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:57:07.913 INFO:teuthology.orchestra.run.smithi123.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-24T05:57:07.927 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/009 2024-04-24T05:57:07.927 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/009 2024-04-24T05:57:07.927 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T05:57:07.939 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:57:07.972 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T05:57:07.972 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2024-04-24T05:57:08.482 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T05:57:08.497 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T05:57:08.497 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:57:10.315 INFO:teuthology.orchestra.run.smithi123.stdout:generic/010 2s 2024-04-24T05:57:11.850 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/010 2024-04-24T05:57:11.851 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T05:57:11.863 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:57:11.909 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T05:57:11.909 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2024-04-24T05:57:12.385 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T05:57:12.398 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T05:57:12.398 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:57:27.483 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:57:27.492 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:57:57.647 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:57:57.653 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:58:26.378 INFO:teuthology.orchestra.run.smithi123.stdout:generic/011 74s 2024-04-24T05:58:26.392 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/011 2024-04-24T05:58:26.392 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T05:58:26.400 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:58:26.444 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T05:58:26.444 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2024-04-24T05:58:26.923 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T05:58:26.937 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T05:58:26.937 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:58:29.150 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:58:29.160 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:58:30.950 INFO:teuthology.orchestra.run.smithi123.stdout:generic/012 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-24T05:58:30.963 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/012 2024-04-24T05:58:30.963 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/012 2024-04-24T05:58:30.963 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T05:58:30.972 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:58:31.007 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T05:58:31.008 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2024-04-24T05:58:31.480 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T05:58:31.493 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T05:58:31.493 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T05:58:59.325 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:58:59.332 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:59:30.164 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T05:59:30.170 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:00:01.055 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:00:01.065 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:00:01.446 INFO:tasks.ceph.mds.a.smithi123.stderr:2024-04-24T06:00:01.444+0000 7f7cdcc00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 116104) UID: 0 2024-04-24T06:00:01.446 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-24T06:00:01.444+0000 7f20c3400640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 116104) UID: 0 2024-04-24T06:00:52.287 INFO:teuthology.orchestra.run.smithi123.stdout:generic/013 141s 2024-04-24T06:00:52.310 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/013 2024-04-24T06:00:52.310 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:00:52.322 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:00:52.358 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:00:52.359 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2024-04-24T06:00:53.014 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:00:53.031 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:00:53.031 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:00:53.592 INFO:teuthology.orchestra.run.smithi123.stdout:generic/014 [not run] Sparse files are not supported or do not work as expected 2024-04-24T06:00:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/014 2024-04-24T06:00:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/014 2024-04-24T06:00:53.609 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:00:53.619 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:00:53.657 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:00:53.657 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2024-04-24T06:00:54.173 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:00:54.188 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:00:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:00:54.646 INFO:teuthology.orchestra.run.smithi123.stdout:generic/015 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:00:54.659 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/015 2024-04-24T06:00:54.659 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/015 2024-04-24T06:00:54.660 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:00:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:00:54.704 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:00:54.704 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2024-04-24T06:00:55.347 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:00:55.363 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:00:55.363 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:00:58.434 INFO:teuthology.orchestra.run.smithi123.stdout:generic/016 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-24T06:00:58.447 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/016 2024-04-24T06:00:58.447 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/016 2024-04-24T06:00:58.447 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:00:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:00:58.508 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:00:58.508 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2024-04-24T06:00:59.247 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:00:59.263 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:00:59.263 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:00:59.702 INFO:teuthology.orchestra.run.smithi123.stdout:generic/017 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:00:59.717 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/017 2024-04-24T06:00:59.717 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/017 2024-04-24T06:00:59.717 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:00:59.729 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:00:59.764 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:00:59.764 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2024-04-24T06:01:00.426 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:00.442 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:00.442 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:00.861 INFO:teuthology.orchestra.run.smithi123.stdout:generic/018 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:01:00.873 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/018 2024-04-24T06:01:00.877 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/018 2024-04-24T06:01:00.877 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:00.885 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:00.918 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:00.918 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2024-04-24T06:01:01.582 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:01.598 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:02.051 INFO:teuthology.orchestra.run.smithi123.stdout:generic/019 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:01:02.064 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/019 2024-04-24T06:01:02.064 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/019 2024-04-24T06:01:02.065 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:02.075 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:02.117 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:02.117 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2024-04-24T06:01:02.758 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:02.775 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:02.776 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:25.101 INFO:teuthology.orchestra.run.smithi123.stdout:generic/020 - output mismatch (see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/020.out.bad) 2024-04-24T06:01:25.108 INFO:teuthology.orchestra.run.smithi123.stdout: --- tests/generic/020.out 2024-04-24 05:47:53.224733205 +0000 2024-04-24T06:01:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: +++ /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/020.out.bad 2024-04-24 06:01:25.096624920 +0000 2024-04-24T06:01:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -47,9 +47,13 @@ 2024-04-24T06:01:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: user.snrub="fish2\012" 2024-04-24T06:01:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: *** really long value 2024-04-24T06:01:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: -0000000 00 2024-04-24T06:01:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: -* 2024-04-24T06:01:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: -ATTRSIZE 2024-04-24T06:01:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: +attr_set: No space left on device 2024-04-24T06:01:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: +Could not set "long_attr" for 2024-04-24T06:01:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: +attr_get: No data available 2024-04-24T06:01:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: +Could not get "long_attr" for 2024-04-24T06:01:25.109 INFO:teuthology.orchestra.run.smithi123.stdout: +0000000 2024-04-24T06:01:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: +attr_remove: No data available 2024-04-24T06:01:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: +Could not remove "long_attr" for 2024-04-24T06:01:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: *** set/get/remove really long names (expect failure) 2024-04-24T06:01:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: attr_set: Invalid argument 2024-04-24T06:01:25.110 INFO:teuthology.orchestra.run.smithi123.stdout: Could not set "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" for 2024-04-24T06:01:25.119 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/020 2024-04-24T06:01:25.119 INFO:teuthology.orchestra.run.smithi123.stdout:Failures: generic/020 2024-04-24T06:01:25.119 INFO:teuthology.orchestra.run.smithi123.stdout:Failed 1 of 1 tests 2024-04-24T06:01:25.128 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:25.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:01:25.176 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-24T06:01:25.176 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-24T06:01:25.176 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2024-04-24T06:01:25.965 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:25.982 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:25.982 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:generic/021 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-24T06:01:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/021 2024-04-24T06:01:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/021 2024-04-24T06:01:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:29.152 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:29.203 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:29.203 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2024-04-24T06:01:29.951 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:29.970 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:29.970 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:33.012 INFO:teuthology.orchestra.run.smithi123.stdout:generic/022 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-24T06:01:33.025 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/022 2024-04-24T06:01:33.025 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/022 2024-04-24T06:01:33.025 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:33.034 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:33.090 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:33.090 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2024-04-24T06:01:33.792 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:33.809 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:33.809 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:37.548 INFO:teuthology.orchestra.run.smithi123.stdout:generic/023 4s 2024-04-24T06:01:37.570 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/023 2024-04-24T06:01:37.570 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:37.581 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:37.621 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:37.621 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2024-04-24T06:01:38.372 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:38.389 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:38.921 INFO:teuthology.orchestra.run.smithi123.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2024-04-24T06:01:38.935 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/024 2024-04-24T06:01:38.935 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/024 2024-04-24T06:01:38.936 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:38.947 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:38.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:38.995 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2024-04-24T06:01:39.724 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:39.741 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:39.741 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:40.267 INFO:teuthology.orchestra.run.smithi123.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2024-04-24T06:01:40.280 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/025 2024-04-24T06:01:40.280 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/025 2024-04-24T06:01:40.280 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:40.290 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:40.335 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:40.335 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2024-04-24T06:01:41.083 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:41.103 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:41.104 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:41.669 INFO:teuthology.orchestra.run.smithi123.stdout:generic/026 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-24T06:01:41.683 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/026 2024-04-24T06:01:41.683 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/026 2024-04-24T06:01:41.683 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:41.694 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:41.731 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:41.731 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2024-04-24T06:01:42.418 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:42.434 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:42.434 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:42.890 INFO:teuthology.orchestra.run.smithi123.stdout:generic/027 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:01:42.905 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/027 2024-04-24T06:01:42.906 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/027 2024-04-24T06:01:42.906 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:42.915 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:42.950 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:42.950 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2024-04-24T06:01:43.609 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:43.625 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:43.625 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:49.130 INFO:teuthology.orchestra.run.smithi123.stdout:generic/028 6s 2024-04-24T06:01:49.149 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/028 2024-04-24T06:01:49.149 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:49.162 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:49.193 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:49.193 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2024-04-24T06:01:49.833 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:49.847 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:49.847 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:50.313 INFO:teuthology.orchestra.run.smithi123.stdout:generic/029 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:01:50.328 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/029 2024-04-24T06:01:50.328 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/029 2024-04-24T06:01:50.328 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:50.381 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:50.381 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2024-04-24T06:01:51.034 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:51.053 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:51.053 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:51.503 INFO:teuthology.orchestra.run.smithi123.stdout:generic/030 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:01:51.516 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/030 2024-04-24T06:01:51.516 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/030 2024-04-24T06:01:51.516 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:51.525 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:51.563 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:51.563 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2024-04-24T06:01:52.211 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:52.230 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:52.230 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:52.904 INFO:teuthology.orchestra.run.smithi123.stdout:generic/031 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:01:52.918 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/031 2024-04-24T06:01:52.918 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/031 2024-04-24T06:01:52.919 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:52.928 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:52.959 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:52.959 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2024-04-24T06:01:53.615 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:53.638 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:53.638 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:54.087 INFO:teuthology.orchestra.run.smithi123.stdout:generic/032 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:01:54.103 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/032 2024-04-24T06:01:54.103 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/032 2024-04-24T06:01:54.103 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:54.151 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:54.151 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2024-04-24T06:01:54.712 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:54.728 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:55.140 INFO:teuthology.orchestra.run.smithi123.stdout:generic/033 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:01:55.159 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/033 2024-04-24T06:01:55.159 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/033 2024-04-24T06:01:55.159 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:55.170 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:55.206 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:55.207 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2024-04-24T06:01:55.888 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:55.905 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:55.905 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:56.381 INFO:teuthology.orchestra.run.smithi123.stdout:generic/034 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:01:56.393 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/034 2024-04-24T06:01:56.393 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/034 2024-04-24T06:01:56.393 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:56.404 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:56.441 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:56.441 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2024-04-24T06:01:56.977 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:56.993 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:56.993 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:57.629 INFO:teuthology.orchestra.run.smithi123.stdout:generic/035 0s 2024-04-24T06:01:57.770 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/035 2024-04-24T06:01:57.770 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:01:57.780 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:01:57.841 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:01:57.841 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2024-04-24T06:01:58.567 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:01:58.589 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:01:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:09.268 INFO:teuthology.orchestra.run.smithi123.stdout:generic/036 11s 2024-04-24T06:02:09.283 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/036 2024-04-24T06:02:09.284 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:09.295 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:09.328 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:09.328 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2024-04-24T06:02:10.060 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:10.080 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:10.080 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:10.516 INFO:teuthology.orchestra.run.smithi123.stdout:generic/037 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:10.530 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/037 2024-04-24T06:02:10.530 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/037 2024-04-24T06:02:10.530 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:10.540 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:10.579 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:10.579 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2024-04-24T06:02:11.226 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:11.244 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:11.244 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:11.676 INFO:teuthology.orchestra.run.smithi123.stdout:generic/038 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:11.692 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/038 2024-04-24T06:02:11.692 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/038 2024-04-24T06:02:11.692 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:11.704 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:11.747 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:11.747 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2024-04-24T06:02:12.396 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:12.410 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:12.410 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:12.879 INFO:teuthology.orchestra.run.smithi123.stdout:generic/039 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:12.892 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/039 2024-04-24T06:02:12.892 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/039 2024-04-24T06:02:12.893 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:12.902 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:12.933 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:12.934 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2024-04-24T06:02:13.585 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:13.605 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:13.605 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:14.089 INFO:teuthology.orchestra.run.smithi123.stdout:generic/040 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:14.102 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/040 2024-04-24T06:02:14.103 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/040 2024-04-24T06:02:14.103 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:14.113 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:14.149 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:14.149 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2024-04-24T06:02:14.764 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:14.781 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:14.781 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:15.233 INFO:teuthology.orchestra.run.smithi123.stdout:generic/041 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:15.246 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/041 2024-04-24T06:02:15.247 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/041 2024-04-24T06:02:15.247 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:15.258 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:15.289 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:15.289 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2024-04-24T06:02:15.884 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:15.902 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:15.902 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:16.338 INFO:teuthology.orchestra.run.smithi123.stdout:generic/042 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:16.351 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/042 2024-04-24T06:02:16.351 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/042 2024-04-24T06:02:16.351 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:16.361 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:16.390 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:16.390 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2024-04-24T06:02:17.058 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:17.075 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:17.497 INFO:teuthology.orchestra.run.smithi123.stdout:generic/043 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:17.507 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/043 2024-04-24T06:02:17.507 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/043 2024-04-24T06:02:17.508 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:17.517 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:17.546 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:17.547 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2024-04-24T06:02:18.182 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:18.200 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:18.200 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:18.616 INFO:teuthology.orchestra.run.smithi123.stdout:generic/044 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:18.627 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/044 2024-04-24T06:02:18.627 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/044 2024-04-24T06:02:18.627 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:18.635 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:18.668 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:18.668 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2024-04-24T06:02:19.276 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:19.291 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:19.292 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:19.734 INFO:teuthology.orchestra.run.smithi123.stdout:generic/045 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:19.749 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/045 2024-04-24T06:02:19.749 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/045 2024-04-24T06:02:19.749 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:19.759 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:19.788 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:19.788 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2024-04-24T06:02:20.433 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:20.448 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:20.448 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:20.898 INFO:teuthology.orchestra.run.smithi123.stdout:generic/046 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:20.908 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/046 2024-04-24T06:02:20.908 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/046 2024-04-24T06:02:20.908 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:20.917 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:20.949 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:20.949 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2024-04-24T06:02:21.591 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:21.608 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:21.608 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:22.039 INFO:teuthology.orchestra.run.smithi123.stdout:generic/047 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:22.053 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/047 2024-04-24T06:02:22.053 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/047 2024-04-24T06:02:22.053 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:22.061 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:22.093 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:22.093 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2024-04-24T06:02:22.734 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:22.751 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:22.751 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:23.195 INFO:teuthology.orchestra.run.smithi123.stdout:generic/048 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:23.209 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/048 2024-04-24T06:02:23.209 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/048 2024-04-24T06:02:23.209 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:23.219 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:23.250 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:23.250 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2024-04-24T06:02:23.911 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:23.932 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:23.932 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:24.367 INFO:teuthology.orchestra.run.smithi123.stdout:generic/049 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:24.382 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/049 2024-04-24T06:02:24.382 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/049 2024-04-24T06:02:24.382 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:24.391 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:24.428 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:24.429 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2024-04-24T06:02:25.092 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:25.108 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:25.108 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:26.329 INFO:teuthology.orchestra.run.smithi123.stdout:generic/050 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:26.341 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/050 2024-04-24T06:02:26.341 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/050 2024-04-24T06:02:26.342 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:26.351 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:26.385 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:26.385 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2024-04-24T06:02:27.068 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:27.087 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:27.502 INFO:teuthology.orchestra.run.smithi123.stdout:generic/051 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:27.516 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/051 2024-04-24T06:02:27.516 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/051 2024-04-24T06:02:27.516 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:27.526 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:27.557 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:27.558 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2024-04-24T06:02:28.198 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:28.213 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:28.213 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:28.676 INFO:teuthology.orchestra.run.smithi123.stdout:generic/052 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:28.689 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/052 2024-04-24T06:02:28.689 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/052 2024-04-24T06:02:28.689 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:28.697 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:28.737 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:28.738 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2024-04-24T06:02:29.356 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:29.373 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:29.373 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:29.791 INFO:teuthology.orchestra.run.smithi123.stdout:generic/053 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:29.804 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/053 2024-04-24T06:02:29.804 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/053 2024-04-24T06:02:29.804 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:29.813 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:29.852 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:29.852 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2024-04-24T06:02:30.482 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:30.496 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:30.496 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:30.930 INFO:teuthology.orchestra.run.smithi123.stdout:generic/054 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:30.942 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/054 2024-04-24T06:02:30.942 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/054 2024-04-24T06:02:30.942 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:30.953 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:30.987 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:30.987 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2024-04-24T06:02:31.627 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:31.641 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:31.641 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:32.077 INFO:teuthology.orchestra.run.smithi123.stdout:generic/055 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:32.090 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/055 2024-04-24T06:02:32.090 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/055 2024-04-24T06:02:32.090 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:32.102 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:32.132 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:32.133 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2024-04-24T06:02:32.741 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:32.755 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:32.755 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:33.208 INFO:teuthology.orchestra.run.smithi123.stdout:generic/056 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:33.220 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/056 2024-04-24T06:02:33.221 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/056 2024-04-24T06:02:33.221 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:33.231 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:33.272 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:33.272 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2024-04-24T06:02:33.888 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:33.904 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:33.904 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:34.375 INFO:teuthology.orchestra.run.smithi123.stdout:generic/057 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:34.385 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/057 2024-04-24T06:02:34.385 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/057 2024-04-24T06:02:34.386 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:34.395 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:34.427 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:34.427 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2024-04-24T06:02:35.041 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:35.059 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:35.059 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:38.000 INFO:teuthology.orchestra.run.smithi123.stdout:generic/058 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-24T06:02:38.014 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/058 2024-04-24T06:02:38.015 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/058 2024-04-24T06:02:38.015 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:38.024 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:38.077 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:38.078 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2024-04-24T06:02:38.837 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:38.852 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:38.852 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:39.337 INFO:teuthology.orchestra.run.smithi123.stdout:generic/059 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:39.351 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/059 2024-04-24T06:02:39.352 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/059 2024-04-24T06:02:39.352 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:39.365 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:39.398 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:39.398 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2024-04-24T06:02:40.032 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:40.050 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:40.050 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:42.937 INFO:teuthology.orchestra.run.smithi123.stdout:generic/060 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-24T06:02:42.952 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/060 2024-04-24T06:02:42.952 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/060 2024-04-24T06:02:42.952 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:42.963 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:43.034 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:43.034 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2024-04-24T06:02:43.684 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:43.699 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:43.699 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:46.412 INFO:teuthology.orchestra.run.smithi123.stdout:generic/061 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-24T06:02:46.424 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/061 2024-04-24T06:02:46.424 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/061 2024-04-24T06:02:46.425 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:46.433 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:46.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:46.463 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2024-04-24T06:02:47.169 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:47.184 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:47.184 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:47.587 INFO:teuthology.orchestra.run.smithi123.stdout:generic/062 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:47.600 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/062 2024-04-24T06:02:47.601 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/062 2024-04-24T06:02:47.601 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:47.643 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:47.643 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2024-04-24T06:02:48.244 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:48.260 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:48.260 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:50.984 INFO:teuthology.orchestra.run.smithi123.stdout:generic/063 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-24T06:02:50.994 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/063 2024-04-24T06:02:50.994 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/063 2024-04-24T06:02:50.994 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:51.003 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:51.048 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:51.048 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2024-04-24T06:02:51.765 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:51.783 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:51.783 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:52.185 INFO:teuthology.orchestra.run.smithi123.stdout:generic/064 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:52.197 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/064 2024-04-24T06:02:52.197 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/064 2024-04-24T06:02:52.197 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:52.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:52.233 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2024-04-24T06:02:52.805 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:52.818 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:52.818 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:53.255 INFO:teuthology.orchestra.run.smithi123.stdout:generic/065 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:53.265 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/065 2024-04-24T06:02:53.266 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/065 2024-04-24T06:02:53.266 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:53.275 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:53.308 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:53.308 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2024-04-24T06:02:53.885 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:53.900 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:54.333 INFO:teuthology.orchestra.run.smithi123.stdout:generic/066 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:54.344 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/066 2024-04-24T06:02:54.344 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/066 2024-04-24T06:02:54.344 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:54.390 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:54.390 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2024-04-24T06:02:54.970 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:54.987 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:55.425 INFO:teuthology.orchestra.run.smithi123.stdout:generic/067 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:55.437 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/067 2024-04-24T06:02:55.437 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/067 2024-04-24T06:02:55.437 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:55.447 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:55.480 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:55.481 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2024-04-24T06:02:56.054 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:56.072 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:56.073 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:56.482 INFO:teuthology.orchestra.run.smithi123.stdout:generic/068 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:56.495 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/068 2024-04-24T06:02:56.495 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/068 2024-04-24T06:02:56.495 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:56.506 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:56.538 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:56.538 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2024-04-24T06:02:57.057 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:57.068 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:57.068 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:57.460 INFO:teuthology.orchestra.run.smithi123.stdout:generic/069 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:02:57.470 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/069 2024-04-24T06:02:57.470 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/069 2024-04-24T06:02:57.471 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:02:57.479 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:02:57.511 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:02:57.511 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2024-04-24T06:02:58.079 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:02:58.094 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:02:58.094 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:03:10.824 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:03:10.831 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:03:41.337 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:03:41.342 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:04:11.470 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:04:11.475 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:04:41.621 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:04:41.628 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:05:10.403 INFO:teuthology.orchestra.run.smithi123.stdout:generic/070 132s 2024-04-24T06:05:10.422 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/070 2024-04-24T06:05:10.423 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:05:10.431 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:05:10.477 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:05:10.478 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2024-04-24T06:05:11.214 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:05:11.230 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:05:11.230 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:05:11.614 INFO:teuthology.orchestra.run.smithi123.stdout:generic/071 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:05:11.626 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/071 2024-04-24T06:05:11.626 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/071 2024-04-24T06:05:11.627 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:05:11.638 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:05:11.668 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:05:11.669 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2024-04-24T06:05:11.774 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:05:11.780 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:05:12.242 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:05:12.253 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:05:12.254 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:05:14.737 INFO:teuthology.orchestra.run.smithi123.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-04-24T06:05:14.749 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/072 2024-04-24T06:05:14.749 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/072 2024-04-24T06:05:14.749 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:05:14.759 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:05:14.790 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:05:14.793 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2024-04-24T06:05:15.350 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:05:15.366 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:05:15.366 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:05:15.783 INFO:teuthology.orchestra.run.smithi123.stdout:generic/073 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:05:15.797 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/073 2024-04-24T06:05:15.797 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/073 2024-04-24T06:05:15.797 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:05:15.805 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:05:15.833 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:05:15.834 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2024-04-24T06:05:16.392 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:05:16.405 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:05:16.406 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:05:41.887 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:05:41.893 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:06:12.061 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:06:12.069 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:06:42.243 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:06:42.250 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:07:12.457 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:07:12.467 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:07:42.752 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:07:42.761 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:08:12.909 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:08:12.914 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:08:43.074 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:08:43.091 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:09:13.346 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:09:13.354 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:09:44.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:09:44.046 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:10:14.304 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:10:14.310 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:10:44.470 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:10:44.476 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:11:14.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:11:14.697 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:11:45.962 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:11:45.969 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:12:16.129 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:12:16.137 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:12:46.299 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:12:46.309 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:13:16.453 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:13:16.461 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:13:47.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:13:47.139 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:14:20.163 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:14:20.169 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:14:21.211 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T06:14:21.210+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 41146) UID: 0 2024-04-24T06:14:21.212 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T06:14:21.210+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 41146) UID: 0 2024-04-24T06:14:21.212 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T06:14:21.210+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 41146) UID: 0 2024-04-24T06:14:21.213 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T06:14:21.211+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 41146) UID: 0 2024-04-24T06:18:54.261 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:18:54.267 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:18:54.553 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T06:18:54.550+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 41174) UID: 0 2024-04-24T06:18:54.554 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T06:18:54.550+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 41174) UID: 0 2024-04-24T06:18:54.556 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T06:18:54.554+0000 7f92cc400640 -1 Fail to open '/proc/41174/cmdline' error = (2) No such file or directory 2024-04-24T06:18:54.556 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T06:18:54.554+0000 7f92cc400640 -1 received signal: Hangup from (PID: 41174) UID: 0 2024-04-24T06:18:54.557 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T06:18:54.552+0000 7ff1d3600640 -1 received signal: Hangup from (PID: 41174) UID: 0 2024-04-24T06:18:58.126 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T06:18:58.124+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 147144) UID: 0 2024-04-24T06:18:58.127 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T06:18:58.125+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 147144) UID: 0 2024-04-24T06:18:58.127 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T06:18:58.125+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 147144) UID: 0 2024-04-24T06:18:58.127 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T06:18:58.125+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 147144) UID: 0 2024-04-24T06:24:19.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:24:19.360 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:24:19.587 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T06:24:19.585+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 41238) UID: 0 2024-04-24T06:24:19.587 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T06:24:19.585+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 41238) UID: 0 2024-04-24T06:24:19.587 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T06:24:19.585+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 41238) UID: 0 2024-04-24T06:24:19.588 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T06:24:19.585+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 41238) UID: 0 2024-04-24T06:24:25.232 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T06:24:25.229+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 147271) UID: 0 2024-04-24T06:24:25.234 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T06:24:25.229+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 147271) UID: 0 2024-04-24T06:24:25.235 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T06:24:25.229+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 147271) UID: 0 2024-04-24T06:24:25.241 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T06:24:25.232+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 147271) UID: 0 2024-04-24T06:35:46.721 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:35:46.732 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:35:50.485 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T06:35:50.483+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 41589) UID: 0 2024-04-24T06:35:50.486 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T06:35:50.483+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 41589) UID: 0 2024-04-24T06:35:50.486 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T06:35:50.483+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 41589) UID: 0 2024-04-24T06:35:50.486 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T06:35:50.483+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 41589) UID: 0 2024-04-24T06:35:51.553 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T06:35:51.552+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 148755) UID: 0 2024-04-24T06:35:51.553 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T06:35:51.552+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 148755) UID: 0 2024-04-24T06:35:51.554 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T06:35:51.552+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 148755) UID: 0 2024-04-24T06:35:51.554 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T06:35:51.552+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 148755) UID: 0 2024-04-24T06:36:39.732 INFO:teuthology.orchestra.run.smithi123.stdout:generic/074 1883s 2024-04-24T06:36:39.770 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/074 2024-04-24T06:36:39.770 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:36:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:36:39.860 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:36:39.860 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2024-04-24T06:36:41.091 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:36:41.104 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:36:41.104 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:15.704 INFO:teuthology.orchestra.run.smithi123.stdout:generic/075 94s 2024-04-24T06:38:15.719 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/075 2024-04-24T06:38:15.719 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:38:15.728 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:15.762 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:38:15.763 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2024-04-24T06:38:16.504 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:38:16.517 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:38:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:16.911 INFO:teuthology.orchestra.run.smithi123.stdout:generic/076 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:38:16.923 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/076 2024-04-24T06:38:16.924 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/076 2024-04-24T06:38:16.924 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:38:16.934 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:16.962 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:38:16.962 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2024-04-24T06:38:17.547 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:38:17.559 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:38:17.559 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:20.172 INFO:teuthology.orchestra.run.smithi123.stdout:generic/077 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:38:20.184 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/077 2024-04-24T06:38:20.184 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/077 2024-04-24T06:38:20.184 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:38:20.194 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:20.221 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:38:20.221 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2024-04-24T06:38:20.704 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:38:20.719 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:38:20.719 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:21.188 INFO:teuthology.orchestra.run.smithi123.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2024-04-24T06:38:21.201 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/078 2024-04-24T06:38:21.201 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/078 2024-04-24T06:38:21.202 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:38:21.212 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:21.248 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:38:21.248 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2024-04-24T06:38:21.942 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:38:21.954 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:38:21.954 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:22.470 INFO:teuthology.orchestra.run.smithi123.stdout:generic/079 [not run] file system doesn't support chattr +ia 2024-04-24T06:38:22.481 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/079 2024-04-24T06:38:22.481 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/079 2024-04-24T06:38:22.481 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:38:22.492 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:22.522 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:38:22.522 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2024-04-24T06:38:22.990 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:38:23.001 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:38:23.001 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:25.526 INFO:teuthology.orchestra.run.smithi123.stdout:generic/080 2s 2024-04-24T06:38:25.540 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/080 2024-04-24T06:38:25.541 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:38:25.548 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:25.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:38:25.573 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2024-04-24T06:38:26.028 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:38:26.041 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:38:26.041 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:26.414 INFO:teuthology.orchestra.run.smithi123.stdout:generic/081 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:38:26.425 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/081 2024-04-24T06:38:26.425 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/081 2024-04-24T06:38:26.425 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:38:26.434 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:26.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:38:26.462 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2024-04-24T06:38:26.953 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:38:26.964 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:38:26.964 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:27.369 INFO:teuthology.orchestra.run.smithi123.stdout:generic/082 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:38:27.379 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/082 2024-04-24T06:38:27.379 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/082 2024-04-24T06:38:27.379 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:38:27.388 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:27.412 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:38:27.412 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2024-04-24T06:38:27.877 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:38:27.889 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:38:27.889 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:28.273 INFO:teuthology.orchestra.run.smithi123.stdout:generic/083 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:38:28.285 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/083 2024-04-24T06:38:28.285 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/083 2024-04-24T06:38:28.285 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:38:28.295 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:28.323 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:38:28.324 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2024-04-24T06:38:28.772 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:38:28.784 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:38:28.784 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:generic/084 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:38:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/084 2024-04-24T06:38:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/084 2024-04-24T06:38:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:38:29.145 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:29.166 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:38:29.166 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2024-04-24T06:38:29.607 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:38:29.619 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:38:29.619 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:30.030 INFO:teuthology.orchestra.run.smithi123.stdout:generic/085 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T06:38:30.043 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/085 2024-04-24T06:38:30.043 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/085 2024-04-24T06:38:30.043 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:38:30.054 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:30.082 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:38:30.082 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2024-04-24T06:38:30.544 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:38:30.555 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:38:30.555 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:51.511 INFO:teuthology.orchestra.run.smithi123.stdout:generic/086 21s 2024-04-24T06:38:51.660 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/086 2024-04-24T06:38:51.660 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:38:51.673 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:51.910 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:38:51.911 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2024-04-24T06:38:54.754 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:38:54.765 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:38:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:56.188 INFO:teuthology.orchestra.run.smithi123.stdout:generic/087 2s 2024-04-24T06:38:56.204 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/087 2024-04-24T06:38:56.204 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:38:56.213 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:56.290 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:38:56.290 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2024-04-24T06:38:56.957 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:38:56.967 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:38:56.967 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:57.502 INFO:teuthology.orchestra.run.smithi123.stdout:generic/088 1s 2024-04-24T06:38:57.518 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/088 2024-04-24T06:38:57.518 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T06:38:57.527 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:38:57.566 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T06:38:57.566 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2024-04-24T06:38:58.214 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T06:38:58.225 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T06:38:58.225 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T06:42:26.100 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:42:26.110 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:42:56.234 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:42:56.243 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:43:26.354 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:43:26.363 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:43:57.436 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:43:57.446 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:44:27.567 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:44:27.575 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:44:57.700 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:44:57.709 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:44:57.839 INFO:tasks.ceph.mds.a.smithi123.stderr:2024-04-24T06:44:57.838+0000 7f7cdcc00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 155745) UID: 0 2024-04-24T06:44:57.840 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-24T06:44:57.838+0000 7f20c3400640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 155745) UID: 0 2024-04-24T06:48:11.970 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:48:11.980 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:48:42.169 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:48:42.179 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:49:12.302 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:49:12.314 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:49:42.853 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:49:42.863 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:50:12.975 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:50:12.984 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:50:44.117 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:50:44.124 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:51:14.258 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:51:14.267 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:51:44.490 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:51:44.499 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:52:14.619 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:52:14.627 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:52:44.741 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:52:44.749 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:53:15.054 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:53:15.064 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:53:15.151 INFO:tasks.ceph.mds.a.smithi123.stderr:2024-04-24T06:53:15.150+0000 7f7cdcc00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 156166) UID: 0 2024-04-24T06:53:15.151 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-24T06:53:15.150+0000 7f20c3400640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 156166) UID: 0 2024-04-24T06:56:06.382 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:56:06.392 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:56:36.517 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:56:36.525 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:57:06.664 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:57:06.673 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:57:36.791 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:57:36.801 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:58:06.934 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:58:06.946 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:58:37.108 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:58:37.119 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:59:07.864 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:59:07.876 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:59:38.183 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T06:59:38.194 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:00:08.307 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:00:08.316 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:00:38.472 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:00:38.482 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:01:08.644 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:01:08.653 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:01:39.558 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:01:39.568 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:01:39.656 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T07:01:39.656+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 158419) UID: 0 2024-04-24T07:01:39.657 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T07:01:39.656+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 158419) UID: 0 2024-04-24T07:01:39.657 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T07:01:39.656+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 158419) UID: 0 2024-04-24T07:01:39.657 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T07:01:39.656+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 158419) UID: 0 2024-04-24T07:05:33.614 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:05:33.625 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:05:34.133 INFO:tasks.ceph.mds.a.smithi123.stderr:2024-04-24T07:05:34.132+0000 7f7cdcc00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 158444) UID: 0 2024-04-24T07:05:34.133 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-24T07:05:34.132+0000 7f20c3400640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 158444) UID: 0 2024-04-24T07:09:23.748 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:09:23.787 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:09:53.907 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:09:53.917 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:10:24.124 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:10:24.133 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:10:54.248 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:10:54.256 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:11:25.255 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:11:25.264 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:11:55.376 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:11:55.385 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:12:25.848 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:12:25.861 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:12:55.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:12:56.005 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:13:26.148 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:13:26.156 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:13:40.247 INFO:teuthology.orchestra.run.smithi123.stdout:generic/089 2082s 2024-04-24T07:13:40.292 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/089 2024-04-24T07:13:40.292 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:13:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:13:40.342 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:13:40.342 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/090) 2024-04-24T07:13:41.226 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:13:41.238 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:13:41.238 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:13:42.524 INFO:teuthology.orchestra.run.smithi123.stdout:generic/090 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:13:42.534 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/090 2024-04-24T07:13:42.539 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/090 2024-04-24T07:13:42.539 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:13:42.563 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:13:42.581 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:13:42.581 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/091) 2024-04-24T07:13:43.146 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:13:43.159 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:13:43.159 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:13:56.269 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:13:56.279 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:14:26.392 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:14:26.401 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:14:56.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:14:56.520 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:15:26.635 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:15:26.643 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:15:56.785 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:15:56.793 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:16:26.912 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:16:26.920 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:16:57.035 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:16:57.045 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:17:27.184 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:17:27.192 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:17:57.314 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:17:57.322 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:18:27.438 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:18:27.452 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:18:57.592 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:18:57.601 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:19:27.806 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:19:27.814 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:19:36.794 INFO:teuthology.orchestra.run.smithi123.stdout:generic/091 353s 2024-04-24T07:19:36.811 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/091 2024-04-24T07:19:36.811 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:19:36.819 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:36.851 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:19:36.851 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/092) 2024-04-24T07:19:37.559 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:19:37.571 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:19:37.572 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:38.147 INFO:teuthology.orchestra.run.smithi123.stdout:generic/092 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-24T07:19:38.160 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/092 2024-04-24T07:19:38.160 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/092 2024-04-24T07:19:38.161 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:19:38.172 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:38.203 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:19:38.204 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/093) 2024-04-24T07:19:38.901 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:19:38.913 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:19:38.913 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:40.462 INFO:teuthology.orchestra.run.smithi123.stdout:generic/093 2s 2024-04-24T07:19:40.480 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/093 2024-04-24T07:19:40.480 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:19:40.491 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:40.538 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:19:40.538 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/094) 2024-04-24T07:19:41.225 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:19:41.236 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:19:41.236 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:41.677 INFO:teuthology.orchestra.run.smithi123.stdout:generic/094 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:19:41.687 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/094 2024-04-24T07:19:41.688 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/094 2024-04-24T07:19:41.688 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:19:41.696 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:41.725 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:19:41.725 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/095) 2024-04-24T07:19:42.510 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:19:42.524 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:19:42.524 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:42.938 INFO:teuthology.orchestra.run.smithi123.stdout:generic/095 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:19:42.948 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/095 2024-04-24T07:19:42.949 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/095 2024-04-24T07:19:42.949 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:19:42.956 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:42.984 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:19:42.985 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/096) 2024-04-24T07:19:43.526 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:19:43.539 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:19:43.539 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:43.958 INFO:teuthology.orchestra.run.smithi123.stdout:generic/096 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:19:43.967 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/096 2024-04-24T07:19:43.967 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/096 2024-04-24T07:19:43.967 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:19:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:44.004 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:19:44.004 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/097) 2024-04-24T07:19:44.551 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:19:44.566 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:19:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:45.717 INFO:teuthology.orchestra.run.smithi123.stdout:generic/097 1s 2024-04-24T07:19:45.729 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/097 2024-04-24T07:19:45.729 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:19:45.735 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:45.755 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:19:45.756 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/098) 2024-04-24T07:19:46.213 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:19:46.224 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:19:46.224 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:46.614 INFO:teuthology.orchestra.run.smithi123.stdout:generic/098 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:19:46.623 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/098 2024-04-24T07:19:46.623 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/098 2024-04-24T07:19:46.623 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:19:46.631 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:46.657 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:19:46.657 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-04-24T07:19:47.190 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:19:47.203 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:19:47.203 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:47.783 INFO:teuthology.orchestra.run.smithi123.stdout:generic/099 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-24T07:19:47.796 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/099 2024-04-24T07:19:47.796 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/099 2024-04-24T07:19:47.796 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:19:47.806 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:47.840 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:19:47.840 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/100) 2024-04-24T07:19:48.396 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:19:48.409 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:19:48.409 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:19:57.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:19:57.999 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:20:28.161 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:20:28.172 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:20:58.307 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:20:58.316 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:21:28.487 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:21:28.496 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:21:40.158 INFO:teuthology.orchestra.run.smithi123.stdout:generic/100 112s 2024-04-24T07:21:40.175 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/100 2024-04-24T07:21:40.175 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:21:40.185 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:40.242 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:21:40.242 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/101) 2024-04-24T07:21:40.977 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:21:40.987 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:21:40.987 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:41.418 INFO:teuthology.orchestra.run.smithi123.stdout:generic/101 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:21:41.430 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/101 2024-04-24T07:21:41.430 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/101 2024-04-24T07:21:41.430 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:21:41.440 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:41.469 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:21:41.469 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/102) 2024-04-24T07:21:41.968 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:21:41.981 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:21:41.982 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:42.409 INFO:teuthology.orchestra.run.smithi123.stdout:generic/102 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:21:42.420 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/102 2024-04-24T07:21:42.420 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/102 2024-04-24T07:21:42.421 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:21:42.429 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:42.456 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:21:42.456 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/103) 2024-04-24T07:21:42.980 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:21:42.993 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:21:42.994 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:43.411 INFO:teuthology.orchestra.run.smithi123.stdout:generic/103 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:21:43.422 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/103 2024-04-24T07:21:43.422 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/103 2024-04-24T07:21:43.422 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:21:43.431 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:43.461 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:21:43.461 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/104) 2024-04-24T07:21:43.991 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:21:44.003 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:21:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:44.455 INFO:teuthology.orchestra.run.smithi123.stdout:generic/104 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:21:44.468 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/104 2024-04-24T07:21:44.468 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/104 2024-04-24T07:21:44.468 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:21:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:44.508 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:21:44.509 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/105) 2024-04-24T07:21:45.051 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:21:45.065 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:21:45.065 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:45.500 INFO:teuthology.orchestra.run.smithi123.stdout:generic/105 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:21:45.514 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/105 2024-04-24T07:21:45.514 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/105 2024-04-24T07:21:45.515 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:21:45.526 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:45.556 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:21:45.557 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/106) 2024-04-24T07:21:46.082 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:21:46.096 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:21:46.096 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:46.586 INFO:teuthology.orchestra.run.smithi123.stdout:generic/106 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:21:46.600 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/106 2024-04-24T07:21:46.600 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/106 2024-04-24T07:21:46.600 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:21:46.613 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:46.645 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:21:46.646 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/107) 2024-04-24T07:21:47.178 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:21:47.190 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:21:47.190 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:47.641 INFO:teuthology.orchestra.run.smithi123.stdout:generic/107 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:21:47.653 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/107 2024-04-24T07:21:47.653 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/107 2024-04-24T07:21:47.653 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:21:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:47.694 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:21:47.694 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/108) 2024-04-24T07:21:48.214 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:21:48.227 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:21:48.227 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:48.914 INFO:teuthology.orchestra.run.smithi123.stdout:generic/108 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:21:48.928 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/108 2024-04-24T07:21:48.928 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/108 2024-04-24T07:21:48.929 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:21:48.940 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:48.971 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:21:48.971 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/109) 2024-04-24T07:21:49.501 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:21:49.513 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:21:49.513 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:49.981 INFO:teuthology.orchestra.run.smithi123.stdout:generic/109 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:21:49.992 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/109 2024-04-24T07:21:49.992 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/109 2024-04-24T07:21:49.992 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:21:50.001 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:50.028 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:21:50.028 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/110) 2024-04-24T07:21:50.570 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:21:50.582 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:21:50.582 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:51.152 INFO:teuthology.orchestra.run.smithi123.stdout:generic/110 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T07:21:51.167 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/110 2024-04-24T07:21:51.167 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/110 2024-04-24T07:21:51.167 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:21:51.178 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:51.227 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:21:51.227 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/111) 2024-04-24T07:21:51.907 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:21:51.918 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:21:51.918 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:52.456 INFO:teuthology.orchestra.run.smithi123.stdout:generic/111 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T07:21:52.471 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/111 2024-04-24T07:21:52.471 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/111 2024-04-24T07:21:52.471 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:21:52.483 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:52.535 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:21:52.535 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/112) 2024-04-24T07:21:53.217 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:21:53.231 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:21:53.231 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:21:58.670 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:21:59.169 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:22:29.291 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:22:29.368 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:22:59.490 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:22:59.499 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:23:29.613 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:23:29.623 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:23:59.751 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:23:59.760 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:24:01.377 INFO:teuthology.orchestra.run.smithi123.stdout:generic/112 128s 2024-04-24T07:24:01.395 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/112 2024-04-24T07:24:01.395 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:24:01.404 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:01.439 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:24:01.440 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/113) 2024-04-24T07:24:02.168 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:24:02.182 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:24:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:29.874 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:24:29.883 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:24:51.919 INFO:teuthology.orchestra.run.smithi123.stdout:generic/113 49s 2024-04-24T07:24:51.937 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/113 2024-04-24T07:24:51.937 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:24:51.946 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:51.986 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:24:51.986 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/114) 2024-04-24T07:24:52.726 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:24:52.737 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:24:52.737 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:53.230 INFO:teuthology.orchestra.run.smithi123.stdout:generic/114 [not run] Sparse files are not supported or do not work as expected 2024-04-24T07:24:53.245 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/114 2024-04-24T07:24:53.245 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/114 2024-04-24T07:24:53.245 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:24:53.256 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:53.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:24:53.294 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/115) 2024-04-24T07:24:53.844 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:24:53.856 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:24:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:54.404 INFO:teuthology.orchestra.run.smithi123.stdout:generic/115 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T07:24:54.417 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/115 2024-04-24T07:24:54.417 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/115 2024-04-24T07:24:54.417 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:24:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:54.476 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:24:54.476 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/116) 2024-04-24T07:24:55.156 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:24:55.168 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:24:55.168 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:55.688 INFO:teuthology.orchestra.run.smithi123.stdout:generic/116 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T07:24:55.701 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/116 2024-04-24T07:24:55.701 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/116 2024-04-24T07:24:55.702 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:24:55.714 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:55.748 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:24:55.749 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/117) 2024-04-24T07:24:56.455 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:24:56.466 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:24:56.466 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:56.862 INFO:teuthology.orchestra.run.smithi123.stdout:generic/117 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:24:56.873 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/117 2024-04-24T07:24:56.873 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/117 2024-04-24T07:24:56.873 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:24:56.882 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:56.913 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:24:56.913 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/118) 2024-04-24T07:24:57.420 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:24:57.433 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:24:57.433 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:58.003 INFO:teuthology.orchestra.run.smithi123.stdout:generic/118 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T07:24:58.017 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/118 2024-04-24T07:24:58.018 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/118 2024-04-24T07:24:58.018 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:24:58.029 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:58.082 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:24:58.082 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/119) 2024-04-24T07:24:58.778 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:24:58.790 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:24:58.790 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:59.332 INFO:teuthology.orchestra.run.smithi123.stdout:generic/119 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T07:24:59.344 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/119 2024-04-24T07:24:59.344 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/119 2024-04-24T07:24:59.344 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:24:59.353 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:24:59.390 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:24:59.390 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/120) 2024-04-24T07:25:00.008 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:25:00.014 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:25:00.075 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:25:00.086 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:25:00.087 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:25:00.514 INFO:teuthology.orchestra.run.smithi123.stdout:generic/120 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:25:00.525 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/120 2024-04-24T07:25:00.526 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/120 2024-04-24T07:25:00.526 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:25:00.534 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:25:00.562 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:25:00.562 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/121) 2024-04-24T07:25:01.088 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:25:01.100 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:25:01.101 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:25:01.656 INFO:teuthology.orchestra.run.smithi123.stdout:generic/121 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-24T07:25:01.669 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/121 2024-04-24T07:25:01.669 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/121 2024-04-24T07:25:01.669 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:25:01.679 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:25:01.712 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:25:01.712 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/122) 2024-04-24T07:25:02.384 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:25:02.394 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:25:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:25:03.177 INFO:teuthology.orchestra.run.smithi123.stdout:generic/122 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-24T07:25:03.187 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/122 2024-04-24T07:25:03.187 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/122 2024-04-24T07:25:03.187 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:25:03.194 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:25:03.218 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:25:03.218 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/123) 2024-04-24T07:25:03.917 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:25:03.963 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:25:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:25:05.134 INFO:teuthology.orchestra.run.smithi123.stdout:generic/123 2s 2024-04-24T07:25:05.152 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/123 2024-04-24T07:25:05.152 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:25:05.160 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:25:05.192 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:25:05.193 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/124) 2024-04-24T07:25:05.886 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:25:05.897 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:25:05.897 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:25:06.306 INFO:teuthology.orchestra.run.smithi123.stdout:generic/124 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T07:25:06.317 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/124 2024-04-24T07:25:06.317 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/124 2024-04-24T07:25:06.317 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:25:06.326 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:25:06.356 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:25:06.357 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/125) 2024-04-24T07:25:06.888 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:25:06.901 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:25:06.902 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:25:30.130 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:25:30.139 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:26:00.346 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:26:00.354 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:26:08.958 INFO:teuthology.orchestra.run.smithi123.stdout:generic/125 62s 2024-04-24T07:26:08.976 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/125 2024-04-24T07:26:08.976 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T07:26:08.985 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:26:09.028 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T07:26:09.028 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/126) 2024-04-24T07:26:09.696 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:26:09.710 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:26:09.711 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:26:11.400 INFO:teuthology.orchestra.run.smithi123.stdout:generic/126 - output mismatch (see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/126.out.bad) 2024-04-24T07:26:11.407 INFO:teuthology.orchestra.run.smithi123.stdout: --- tests/generic/126.out 2024-04-24 05:47:53.240732828 +0000 2024-04-24T07:26:11.407 INFO:teuthology.orchestra.run.smithi123.stdout: +++ /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/126.out.bad 2024-04-24 07:26:11.392718318 +0000 2024-04-24T07:26:11.407 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -1,7 +1,7 @@ 2024-04-24T07:26:11.408 INFO:teuthology.orchestra.run.smithi123.stdout: QA output created by 126 2024-04-24T07:26:11.408 INFO:teuthology.orchestra.run.smithi123.stdout: -x a 001 file owned by (99/99) as user/group(12/100) PASS 2024-04-24T07:26:11.408 INFO:teuthology.orchestra.run.smithi123.stdout: -x a 010 file owned by (99/99) as user/group(200/99) PASS 2024-04-24T07:26:11.408 INFO:teuthology.orchestra.run.smithi123.stdout: -x a 100 file owned by (99/99) as user/group(99/500) PASS 2024-04-24T07:26:11.408 INFO:teuthology.orchestra.run.smithi123.stdout: +x a 001 file owned by (99/99) as user/group(12/100) FAIL 2024-04-24T07:26:11.408 INFO:teuthology.orchestra.run.smithi123.stdout: +x a 010 file owned by (99/99) as user/group(200/99) FAIL 2024-04-24T07:26:11.408 INFO:teuthology.orchestra.run.smithi123.stdout: +x a 100 file owned by (99/99) as user/group(99/500) FAIL 2024-04-24T07:26:11.408 INFO:teuthology.orchestra.run.smithi123.stdout: w a 002 file owned by (99/99) as user/group(12/100) PASS 2024-04-24T07:26:11.408 INFO:teuthology.orchestra.run.smithi123.stdout: w a 020 file owned by (99/99) as user/group(200/99) PASS 2024-04-24T07:26:11.408 INFO:teuthology.orchestra.run.smithi123.stdout: w a 200 file owned by (99/99) as user/group(99/500) PASS 2024-04-24T07:26:11.415 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/126 2024-04-24T07:26:11.416 INFO:teuthology.orchestra.run.smithi123.stdout:Failures: generic/126 2024-04-24T07:26:11.416 INFO:teuthology.orchestra.run.smithi123.stdout:Failed 1 of 1 tests 2024-04-24T07:26:11.424 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:26:11.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T07:26:11.452 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-24T07:26:11.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-24T07:26:11.453 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/127) 2024-04-24T07:26:12.155 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T07:26:12.169 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T07:26:12.169 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T07:26:30.476 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:26:30.485 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:27:00.632 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:27:00.641 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:27:30.846 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:27:30.855 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:28:01.060 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:28:01.072 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:28:31.191 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:28:31.203 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:29:01.397 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:29:01.407 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:29:31.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:29:31.822 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:30:01.940 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:30:01.949 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:30:32.080 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:30:32.094 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:31:02.226 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:31:02.234 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:31:32.357 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:31:32.366 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:32:02.479 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:32:02.488 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:32:32.610 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:32:32.620 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:33:02.740 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:33:02.749 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:33:32.879 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:33:32.889 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:34:03.009 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:34:03.018 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:34:33.140 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:34:33.149 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:35:03.270 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:35:03.279 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:35:33.417 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:35:33.426 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:36:03.539 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:36:03.547 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:36:03.643 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T07:36:03.643+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 43241) UID: 0 2024-04-24T07:36:03.644 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T07:36:03.644+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 43241) UID: 0 2024-04-24T07:36:03.644 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T07:36:03.644+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 43241) UID: 0 2024-04-24T07:36:03.644 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T07:36:03.644+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 43241) UID: 0 2024-04-24T07:38:51.529 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:38:51.538 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:39:21.726 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:39:21.735 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:39:51.871 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:39:51.880 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:40:22.018 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:40:22.029 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:40:22.124 INFO:tasks.ceph.mds.a.smithi123.stderr:2024-04-24T07:40:22.123+0000 7f7cdcc00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 190967) UID: 0 2024-04-24T07:40:22.124 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-24T07:40:22.123+0000 7f20c3400640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 190967) UID: 0 2024-04-24T07:43:03.844 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:43:03.853 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:43:34.040 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:43:34.050 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:44:04.161 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:44:04.171 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:44:34.292 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:44:34.301 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:45:04.440 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:45:04.449 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:45:34.571 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:45:34.579 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:46:04.701 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:46:04.709 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:46:34.831 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:46:34.841 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:47:04.954 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:47:04.963 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:47:35.076 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:47:35.086 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:48:05.216 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:48:05.225 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:48:35.346 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:48:35.355 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:49:05.468 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:49:05.477 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:49:35.599 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:49:35.608 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:50:05.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:50:05.741 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:50:35.851 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:50:35.860 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:51:05.965 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:51:05.975 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:51:36.163 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:51:36.171 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:52:06.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:52:06.339 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:52:36.510 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:52:36.518 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:53:06.657 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:53:06.668 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:53:36.812 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:53:36.821 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:54:06.950 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:54:06.959 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:54:37.107 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:54:37.117 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:55:07.287 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:55:07.295 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:55:37.467 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:55:37.475 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:56:07.601 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:56:07.610 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:56:37.904 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:56:37.917 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:57:08.051 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:57:08.060 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:57:38.260 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:57:38.268 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:58:08.386 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:58:08.395 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:58:38.541 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:58:38.550 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:59:08.681 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:59:08.692 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:59:38.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T07:59:38.828 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:00:08.941 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:00:08.952 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:00:39.089 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:00:39.099 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:01:09.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:01:09.227 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:01:39.355 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:01:39.364 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:02:09.486 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:02:09.496 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:02:39.626 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:02:39.635 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:03:09.749 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:03:09.758 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:03:39.879 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:03:39.888 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:04:10.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:04:10.013 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:04:40.132 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:04:40.140 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:05:10.288 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:05:10.300 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:05:40.553 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:05:40.562 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:06:10.682 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:06:10.694 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:06:17.201 INFO:teuthology.orchestra.run.smithi123.stdout:generic/127 2405s 2024-04-24T08:06:17.219 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/127 2024-04-24T08:06:17.219 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:06:17.228 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:06:17.290 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:06:17.290 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/128) 2024-04-24T08:06:17.996 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:06:18.007 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:06:18.007 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:06:18.476 INFO:teuthology.orchestra.run.smithi123.stdout:generic/128 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:06:18.490 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/128 2024-04-24T08:06:18.490 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/128 2024-04-24T08:06:18.490 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:06:18.503 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:06:18.534 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:06:18.534 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/129) 2024-04-24T08:06:19.081 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:06:19.094 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:06:19.095 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:06:19.535 INFO:teuthology.orchestra.run.smithi123.stdout:generic/129 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:06:19.547 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/129 2024-04-24T08:06:19.547 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/129 2024-04-24T08:06:19.547 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:06:19.557 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:06:19.586 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:06:19.586 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/130) 2024-04-24T08:06:20.144 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:06:20.156 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:06:20.156 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:06:20.561 INFO:teuthology.orchestra.run.smithi123.stdout:generic/130 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:06:20.574 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/130 2024-04-24T08:06:20.575 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/130 2024-04-24T08:06:20.575 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:06:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:06:20.617 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:06:20.617 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/131) 2024-04-24T08:06:21.177 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:06:21.191 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:06:21.191 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:06:23.150 INFO:teuthology.orchestra.run.smithi123.stdout:generic/131 2s 2024-04-24T08:06:23.167 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/131 2024-04-24T08:06:23.167 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:06:23.177 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:06:23.206 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:06:23.207 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/132) 2024-04-24T08:06:23.898 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:06:23.909 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:06:23.909 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:06:24.301 INFO:teuthology.orchestra.run.smithi123.stdout:generic/132 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:06:24.314 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/132 2024-04-24T08:06:24.314 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/132 2024-04-24T08:06:24.315 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:06:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:06:24.356 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:06:24.356 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/133) 2024-04-24T08:06:24.883 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:06:24.895 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:06:24.895 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:06:40.803 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:06:40.811 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:07:11.071 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:07:11.082 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:07:41.197 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:07:41.206 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:08:11.322 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:08:11.329 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:08:41.501 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:08:41.511 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:09:11.638 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:09:11.646 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:09:41.759 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:09:41.768 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:10:11.900 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:10:11.909 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:10:42.026 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:10:42.034 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:11:12.155 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:11:12.164 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:11:42.278 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:11:42.287 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:12:12.434 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:12:12.443 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:12:42.568 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:12:42.576 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:13:12.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:13:12.714 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:13:42.871 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:13:42.879 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:14:12.992 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:14:13.001 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:14:43.128 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:14:43.137 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:15:13.361 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:15:13.370 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:15:43.491 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:15:43.500 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:16:13.621 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:16:13.632 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:16:43.752 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:16:43.761 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:17:13.872 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:17:13.881 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:17:17.220 INFO:teuthology.orchestra.run.smithi123.stdout:generic/133 653s 2024-04-24T08:17:17.239 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/133 2024-04-24T08:17:17.239 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:17.249 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:17.282 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:17.283 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/134) 2024-04-24T08:17:18.075 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:18.086 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:18.087 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:18.626 INFO:teuthology.orchestra.run.smithi123.stdout:generic/134 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:18.638 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/134 2024-04-24T08:17:18.638 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/134 2024-04-24T08:17:18.639 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:18.649 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:18.692 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:18.692 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/135) 2024-04-24T08:17:19.361 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:19.374 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:19.374 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:19.896 INFO:teuthology.orchestra.run.smithi123.stdout:generic/135 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:17:19.910 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/135 2024-04-24T08:17:19.910 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/135 2024-04-24T08:17:19.911 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:19.923 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:19.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:19.960 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/136) 2024-04-24T08:17:20.484 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:20.497 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:20.498 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:21.031 INFO:teuthology.orchestra.run.smithi123.stdout:generic/136 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:21.044 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/136 2024-04-24T08:17:21.044 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/136 2024-04-24T08:17:21.044 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:21.054 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:21.104 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:21.104 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/137) 2024-04-24T08:17:21.766 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:21.777 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:21.777 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:22.321 INFO:teuthology.orchestra.run.smithi123.stdout:generic/137 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:22.335 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/137 2024-04-24T08:17:22.335 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/137 2024-04-24T08:17:22.335 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:22.346 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:22.394 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:22.394 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/138) 2024-04-24T08:17:23.074 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:23.087 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:23.087 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:23.628 INFO:teuthology.orchestra.run.smithi123.stdout:generic/138 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:23.641 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/138 2024-04-24T08:17:23.642 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/138 2024-04-24T08:17:23.642 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:23.652 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:23.708 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:23.708 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/139) 2024-04-24T08:17:24.361 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:24.371 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:24.371 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:24.854 INFO:teuthology.orchestra.run.smithi123.stdout:generic/139 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:24.865 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/139 2024-04-24T08:17:24.865 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/139 2024-04-24T08:17:24.865 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:24.873 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:24.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:24.903 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/140) 2024-04-24T08:17:25.561 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:25.573 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:25.573 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:26.131 INFO:teuthology.orchestra.run.smithi123.stdout:generic/140 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:26.145 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/140 2024-04-24T08:17:26.145 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/140 2024-04-24T08:17:26.145 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:26.156 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:26.188 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:26.189 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/141) 2024-04-24T08:17:26.840 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:26.851 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:26.851 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:27.232 INFO:teuthology.orchestra.run.smithi123.stdout:generic/141 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:17:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/141 2024-04-24T08:17:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/141 2024-04-24T08:17:27.243 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:27.282 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:27.282 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/142) 2024-04-24T08:17:27.804 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:27.817 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:27.817 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:28.412 INFO:teuthology.orchestra.run.smithi123.stdout:generic/142 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:28.424 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/142 2024-04-24T08:17:28.424 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/142 2024-04-24T08:17:28.425 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:28.434 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:28.466 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:28.466 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/143) 2024-04-24T08:17:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:29.183 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:29.733 INFO:teuthology.orchestra.run.smithi123.stdout:generic/143 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/143 2024-04-24T08:17:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/143 2024-04-24T08:17:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:29.758 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:29.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:29.809 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/144) 2024-04-24T08:17:30.510 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:30.522 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:30.523 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:31.082 INFO:teuthology.orchestra.run.smithi123.stdout:generic/144 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/144 2024-04-24T08:17:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/144 2024-04-24T08:17:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:31.107 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:31.140 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:31.140 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/145) 2024-04-24T08:17:31.827 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:31.838 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:31.838 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:32.415 INFO:teuthology.orchestra.run.smithi123.stdout:generic/145 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:32.427 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/145 2024-04-24T08:17:32.427 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/145 2024-04-24T08:17:32.427 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:32.437 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:32.467 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:32.468 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/146) 2024-04-24T08:17:33.157 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:33.168 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:33.168 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:33.705 INFO:teuthology.orchestra.run.smithi123.stdout:generic/146 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:33.716 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/146 2024-04-24T08:17:33.717 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/146 2024-04-24T08:17:33.717 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:33.726 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:33.769 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:33.770 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/147) 2024-04-24T08:17:34.440 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:34.452 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:34.452 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:34.954 INFO:teuthology.orchestra.run.smithi123.stdout:generic/147 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:34.965 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/147 2024-04-24T08:17:34.965 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/147 2024-04-24T08:17:34.965 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:34.977 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:35.019 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:35.019 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/148) 2024-04-24T08:17:35.670 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:35.681 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:35.681 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:36.327 INFO:teuthology.orchestra.run.smithi123.stdout:generic/148 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:36.338 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/148 2024-04-24T08:17:36.338 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/148 2024-04-24T08:17:36.338 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:36.347 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:36.378 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:36.378 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/149) 2024-04-24T08:17:37.080 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:37.091 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:37.092 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:37.630 INFO:teuthology.orchestra.run.smithi123.stdout:generic/149 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:37.641 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/149 2024-04-24T08:17:37.641 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/149 2024-04-24T08:17:37.641 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:37.650 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:37.701 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:37.701 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/150) 2024-04-24T08:17:38.370 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:38.381 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:38.381 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:38.896 INFO:teuthology.orchestra.run.smithi123.stdout:generic/150 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:38.907 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/150 2024-04-24T08:17:38.908 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/150 2024-04-24T08:17:38.908 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:38.916 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:38.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:38.942 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/151) 2024-04-24T08:17:39.604 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:39.617 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:39.617 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:40.526 INFO:teuthology.orchestra.run.smithi123.stdout:generic/151 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:40.539 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/151 2024-04-24T08:17:40.540 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/151 2024-04-24T08:17:40.540 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:40.551 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:40.603 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:40.603 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/152) 2024-04-24T08:17:41.274 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:41.286 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:41.287 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:41.841 INFO:teuthology.orchestra.run.smithi123.stdout:generic/152 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:41.853 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/152 2024-04-24T08:17:41.853 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/152 2024-04-24T08:17:41.853 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:41.862 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:41.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:41.903 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/153) 2024-04-24T08:17:42.580 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:42.591 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:42.591 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:43.078 INFO:teuthology.orchestra.run.smithi123.stdout:generic/153 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:43.091 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/153 2024-04-24T08:17:43.091 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/153 2024-04-24T08:17:43.091 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:43.101 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:43.154 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:43.154 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/154) 2024-04-24T08:17:43.814 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:43.827 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:43.827 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:44.020 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:17:44.026 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:17:44.345 INFO:teuthology.orchestra.run.smithi123.stdout:generic/154 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:44.355 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/154 2024-04-24T08:17:44.356 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/154 2024-04-24T08:17:44.356 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:44.364 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:44.399 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:44.399 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/155) 2024-04-24T08:17:45.079 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:45.090 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:45.091 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:45.598 INFO:teuthology.orchestra.run.smithi123.stdout:generic/155 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:45.610 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/155 2024-04-24T08:17:45.610 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/155 2024-04-24T08:17:45.610 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:45.619 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:45.658 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:45.658 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/156) 2024-04-24T08:17:46.386 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:46.400 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:46.400 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:46.964 INFO:teuthology.orchestra.run.smithi123.stdout:generic/156 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:46.976 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/156 2024-04-24T08:17:46.976 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/156 2024-04-24T08:17:46.976 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:46.987 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:47.016 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:47.017 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/157) 2024-04-24T08:17:47.703 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:47.714 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:47.714 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:48.194 INFO:teuthology.orchestra.run.smithi123.stdout:generic/157 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:48.208 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/157 2024-04-24T08:17:48.208 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/157 2024-04-24T08:17:48.208 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:48.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:48.253 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/158) 2024-04-24T08:17:48.916 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:48.947 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:48.947 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:49.506 INFO:teuthology.orchestra.run.smithi123.stdout:generic/158 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-24T08:17:49.516 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/158 2024-04-24T08:17:49.516 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/158 2024-04-24T08:17:49.516 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:49.526 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:49.563 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:49.563 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/159) 2024-04-24T08:17:50.243 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:50.253 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:50.253 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:50.787 INFO:teuthology.orchestra.run.smithi123.stdout:generic/159 [not run] file system doesn't support chattr +i 2024-04-24T08:17:50.800 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/159 2024-04-24T08:17:50.801 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/159 2024-04-24T08:17:50.801 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:50.814 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:50.843 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:50.843 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/160) 2024-04-24T08:17:51.385 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:51.396 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:51.396 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:51.883 INFO:teuthology.orchestra.run.smithi123.stdout:generic/160 [not run] file system doesn't support chattr +i 2024-04-24T08:17:51.897 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/160 2024-04-24T08:17:51.897 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/160 2024-04-24T08:17:51.898 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:51.907 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:51.941 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:51.942 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/161) 2024-04-24T08:17:52.545 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:52.558 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:52.558 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:53.010 INFO:teuthology.orchestra.run.smithi123.stdout:generic/161 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:17:53.026 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/161 2024-04-24T08:17:53.026 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/161 2024-04-24T08:17:53.026 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:53.038 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:53.071 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:53.072 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/162) 2024-04-24T08:17:53.625 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:53.641 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:53.641 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:54.071 INFO:teuthology.orchestra.run.smithi123.stdout:generic/162 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:17:54.083 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/162 2024-04-24T08:17:54.083 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/162 2024-04-24T08:17:54.084 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:54.094 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:54.127 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:54.128 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/163) 2024-04-24T08:17:54.653 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:54.666 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:55.076 INFO:teuthology.orchestra.run.smithi123.stdout:generic/163 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:17:55.088 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/163 2024-04-24T08:17:55.088 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/163 2024-04-24T08:17:55.089 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:55.129 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:55.129 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/164) 2024-04-24T08:17:55.658 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:55.670 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:55.670 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:56.132 INFO:teuthology.orchestra.run.smithi123.stdout:generic/164 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:17:56.149 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/164 2024-04-24T08:17:56.149 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/164 2024-04-24T08:17:56.149 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:56.161 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:56.195 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:56.195 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/165) 2024-04-24T08:17:56.736 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:56.750 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:56.750 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:57.166 INFO:teuthology.orchestra.run.smithi123.stdout:generic/165 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:17:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/165 2024-04-24T08:17:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/165 2024-04-24T08:17:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:57.185 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:57.215 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:57.215 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/166) 2024-04-24T08:17:57.732 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:57.746 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:57.746 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:58.188 INFO:teuthology.orchestra.run.smithi123.stdout:generic/166 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:17:58.200 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/166 2024-04-24T08:17:58.200 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/166 2024-04-24T08:17:58.200 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:58.236 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:58.236 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/167) 2024-04-24T08:17:58.765 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:58.777 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:58.777 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:59.226 INFO:teuthology.orchestra.run.smithi123.stdout:generic/167 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:17:59.240 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/167 2024-04-24T08:17:59.240 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/167 2024-04-24T08:17:59.241 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:17:59.250 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:17:59.280 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:17:59.280 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/168) 2024-04-24T08:17:59.845 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:17:59.858 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:17:59.859 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:00.273 INFO:teuthology.orchestra.run.smithi123.stdout:generic/168 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:00.288 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/168 2024-04-24T08:18:00.288 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/168 2024-04-24T08:18:00.288 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:00.300 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:00.338 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:00.339 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/169) 2024-04-24T08:18:00.899 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:00.914 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:00.914 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:01.323 INFO:teuthology.orchestra.run.smithi123.stdout:generic/169 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:01.334 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/169 2024-04-24T08:18:01.335 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/169 2024-04-24T08:18:01.335 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:01.345 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:01.375 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:01.375 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/170) 2024-04-24T08:18:01.920 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:01.934 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:01.934 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:02.398 INFO:teuthology.orchestra.run.smithi123.stdout:generic/170 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:02.412 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/170 2024-04-24T08:18:02.412 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/170 2024-04-24T08:18:02.412 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:02.424 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:02.455 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:02.455 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/171) 2024-04-24T08:18:03.020 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:03.034 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:03.034 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:03.482 INFO:teuthology.orchestra.run.smithi123.stdout:generic/171 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:03.494 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/171 2024-04-24T08:18:03.494 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/171 2024-04-24T08:18:03.494 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:03.503 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:03.536 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:03.536 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/172) 2024-04-24T08:18:04.060 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:04.071 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:04.071 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:04.513 INFO:teuthology.orchestra.run.smithi123.stdout:generic/172 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:04.527 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/172 2024-04-24T08:18:04.527 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/172 2024-04-24T08:18:04.527 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:04.538 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:04.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:04.575 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/173) 2024-04-24T08:18:05.114 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:05.128 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:05.128 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:05.572 INFO:teuthology.orchestra.run.smithi123.stdout:generic/173 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:05.586 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/173 2024-04-24T08:18:05.586 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/173 2024-04-24T08:18:05.586 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:05.597 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:05.628 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:05.629 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/174) 2024-04-24T08:18:06.181 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:06.193 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:06.193 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:06.625 INFO:teuthology.orchestra.run.smithi123.stdout:generic/174 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:06.638 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/174 2024-04-24T08:18:06.638 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/174 2024-04-24T08:18:06.638 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:06.649 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:06.679 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:06.679 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/175) 2024-04-24T08:18:07.205 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:07.217 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:07.218 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:07.635 INFO:teuthology.orchestra.run.smithi123.stdout:generic/175 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:07.650 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/175 2024-04-24T08:18:07.650 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/175 2024-04-24T08:18:07.650 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:07.659 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:07.692 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:07.693 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/176) 2024-04-24T08:18:08.248 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:08.262 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:08.262 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:08.671 INFO:teuthology.orchestra.run.smithi123.stdout:generic/176 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:08.684 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/176 2024-04-24T08:18:08.684 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/176 2024-04-24T08:18:08.684 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:08.695 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:08.725 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:08.726 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/177) 2024-04-24T08:18:09.267 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:09.279 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:09.279 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:09.766 INFO:teuthology.orchestra.run.smithi123.stdout:generic/177 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:09.780 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/177 2024-04-24T08:18:09.780 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/177 2024-04-24T08:18:09.780 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:09.792 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:09.822 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:09.822 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/178) 2024-04-24T08:18:10.397 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:10.411 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:10.411 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:10.990 INFO:teuthology.orchestra.run.smithi123.stdout:generic/178 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:18:11.005 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/178 2024-04-24T08:18:11.005 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/178 2024-04-24T08:18:11.005 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:11.017 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:11.073 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:11.073 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/179) 2024-04-24T08:18:11.761 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:11.773 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:11.773 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:12.323 INFO:teuthology.orchestra.run.smithi123.stdout:generic/179 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:18:12.334 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/179 2024-04-24T08:18:12.334 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/179 2024-04-24T08:18:12.334 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:12.342 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:12.396 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:12.396 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/180) 2024-04-24T08:18:13.065 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:13.078 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:13.634 INFO:teuthology.orchestra.run.smithi123.stdout:generic/180 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:18:13.648 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/180 2024-04-24T08:18:13.648 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/180 2024-04-24T08:18:13.648 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:13.660 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:13.707 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:13.708 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/181) 2024-04-24T08:18:14.393 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:14.404 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:14.405 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:14.948 INFO:teuthology.orchestra.run.smithi123.stdout:generic/181 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:18:14.959 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/181 2024-04-24T08:18:14.959 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/181 2024-04-24T08:18:14.959 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:14.969 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:15.009 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:15.009 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/182) 2024-04-24T08:18:15.100 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:18:15.113 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:18:15.792 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:15.804 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:15.804 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:16.387 INFO:teuthology.orchestra.run.smithi123.stdout:generic/182 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-24T08:18:16.397 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/182 2024-04-24T08:18:16.397 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/182 2024-04-24T08:18:16.397 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:16.405 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:16.432 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:16.432 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/183) 2024-04-24T08:18:17.163 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:17.174 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:17.174 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:17.535 INFO:teuthology.orchestra.run.smithi123.stdout:generic/183 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:17.545 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/183 2024-04-24T08:18:17.545 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/183 2024-04-24T08:18:17.545 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:17.553 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:17.577 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:17.577 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/184) 2024-04-24T08:18:18.055 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:18.067 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:18.067 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:18.606 INFO:teuthology.orchestra.run.smithi123.stdout:generic/184 [failed, exit status 1]- output mismatch (see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/184.out.bad) 2024-04-24T08:18:18.613 INFO:teuthology.orchestra.run.smithi123.stdout: --- tests/generic/184.out 2024-04-24 05:47:53.250732592 +0000 2024-04-24T08:18:18.613 INFO:teuthology.orchestra.run.smithi123.stdout: +++ /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/184.out.bad 2024-04-24 08:18:18.601277841 +0000 2024-04-24T08:18:18.613 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -1 +1,2 @@ 2024-04-24T08:18:18.613 INFO:teuthology.orchestra.run.smithi123.stdout: QA output created by 184 2024-04-24T08:18:18.613 INFO:teuthology.orchestra.run.smithi123.stdout: +/tmp/tmp.ejcLBa8fOwxfstests-dev/tests/generic/184: line 23: /tmp/tmp.CHueoLyoRxtest/null: Permission denied 2024-04-24T08:18:18.621 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/184 2024-04-24T08:18:18.621 INFO:teuthology.orchestra.run.smithi123.stdout:Failures: generic/184 2024-04-24T08:18:18.621 INFO:teuthology.orchestra.run.smithi123.stdout:Failed 1 of 1 tests 2024-04-24T08:18:18.630 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:18.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:18:18.662 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-24T08:18:18.662 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-24T08:18:18.662 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/185) 2024-04-24T08:18:19.186 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:19.199 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:19.199 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:19.627 INFO:teuthology.orchestra.run.smithi123.stdout:generic/185 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:19.705 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/185 2024-04-24T08:18:19.705 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/185 2024-04-24T08:18:19.705 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:19.705 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:19.705 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:19.706 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/186) 2024-04-24T08:18:20.238 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:20.252 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:20.252 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:20.674 INFO:teuthology.orchestra.run.smithi123.stdout:generic/186 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:20.686 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/186 2024-04-24T08:18:20.686 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/186 2024-04-24T08:18:20.686 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:20.695 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:20.724 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:20.724 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/187) 2024-04-24T08:18:21.254 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:21.267 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:21.267 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:21.677 INFO:teuthology.orchestra.run.smithi123.stdout:generic/187 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:21.688 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/187 2024-04-24T08:18:21.688 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/187 2024-04-24T08:18:21.688 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:21.697 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:21.728 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:21.728 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/188) 2024-04-24T08:18:22.289 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:22.302 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:22.302 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:22.699 INFO:teuthology.orchestra.run.smithi123.stdout:generic/188 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:22.710 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/188 2024-04-24T08:18:22.710 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/188 2024-04-24T08:18:22.710 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:22.719 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:22.746 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:22.747 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/189) 2024-04-24T08:18:23.258 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:23.269 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:23.269 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:23.674 INFO:teuthology.orchestra.run.smithi123.stdout:generic/189 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:23.685 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/189 2024-04-24T08:18:23.685 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/189 2024-04-24T08:18:23.685 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:23.694 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:23.721 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:23.721 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/190) 2024-04-24T08:18:24.250 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:24.264 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:24.702 INFO:teuthology.orchestra.run.smithi123.stdout:generic/190 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:24.714 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/190 2024-04-24T08:18:24.714 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/190 2024-04-24T08:18:24.714 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:24.723 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:24.759 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:24.759 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/191) 2024-04-24T08:18:25.278 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:25.290 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:25.291 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:25.697 INFO:teuthology.orchestra.run.smithi123.stdout:generic/191 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:25.709 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/191 2024-04-24T08:18:25.709 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/191 2024-04-24T08:18:25.709 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:25.719 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:25.754 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:25.754 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/192) 2024-04-24T08:18:26.312 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:26.325 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:26.326 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:26.777 INFO:teuthology.orchestra.run.smithi123.stdout:generic/192 [not run] atime not maintained by ceph-fuse 2024-04-24T08:18:26.791 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/192 2024-04-24T08:18:26.791 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/192 2024-04-24T08:18:26.792 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:26.803 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:26.833 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:26.833 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/193) 2024-04-24T08:18:27.353 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:27.366 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:27.366 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:30.016 INFO:teuthology.orchestra.run.smithi123.stdout:generic/193 - output mismatch (see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/193.out.bad) 2024-04-24T08:18:30.027 INFO:teuthology.orchestra.run.smithi123.stdout: --- tests/generic/193.out 2024-04-24 05:47:53.251732569 +0000 2024-04-24T08:18:30.027 INFO:teuthology.orchestra.run.smithi123.stdout: +++ /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/193.out.bad 2024-04-24 08:18:30.012004601 +0000 2024-04-24T08:18:30.027 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -24,7 +24,6 @@ 2024-04-24T08:18:30.027 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:30.027 INFO:teuthology.orchestra.run.smithi123.stdout: user: chmod a+r on qa_user owned file (should succeed) 2024-04-24T08:18:30.027 INFO:teuthology.orchestra.run.smithi123.stdout: user: chmod a+r on root owned file (should fail) 2024-04-24T08:18:30.027 INFO:teuthology.orchestra.run.smithi123.stdout: -chmod: changing permissions of 'test.root': Operation not permitted 2024-04-24T08:18:30.027 INFO:teuthology.orchestra.run.smithi123.stdout: check that the sgid bit is cleared 2024-04-24T08:18:30.027 INFO:teuthology.orchestra.run.smithi123.stdout: -rw-rw-rw- 2024-04-24T08:18:30.027 INFO:teuthology.orchestra.run.smithi123.stdout: check that suid bit is not cleared 2024-04-24T08:18:30.036 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/193 2024-04-24T08:18:30.036 INFO:teuthology.orchestra.run.smithi123.stdout:Failures: generic/193 2024-04-24T08:18:30.036 INFO:teuthology.orchestra.run.smithi123.stdout:Failed 1 of 1 tests 2024-04-24T08:18:30.047 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:30.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:18:30.094 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-24T08:18:30.094 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-24T08:18:30.095 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/194) 2024-04-24T08:18:30.775 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:30.787 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:30.788 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:generic/194 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/194 2024-04-24T08:18:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/194 2024-04-24T08:18:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:31.182 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:31.212 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:31.212 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/195) 2024-04-24T08:18:31.728 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:31.742 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:31.742 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:32.210 INFO:teuthology.orchestra.run.smithi123.stdout:generic/195 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:32.224 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/195 2024-04-24T08:18:32.225 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/195 2024-04-24T08:18:32.225 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:32.236 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:32.268 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:32.269 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/196) 2024-04-24T08:18:32.810 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:32.823 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:33.261 INFO:teuthology.orchestra.run.smithi123.stdout:generic/196 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:33.273 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/196 2024-04-24T08:18:33.273 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/196 2024-04-24T08:18:33.273 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:33.282 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:33.314 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:33.314 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/197) 2024-04-24T08:18:33.849 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:33.863 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:33.863 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:34.267 INFO:teuthology.orchestra.run.smithi123.stdout:generic/197 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:34.277 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/197 2024-04-24T08:18:34.277 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/197 2024-04-24T08:18:34.277 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:34.285 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:34.313 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:34.313 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/198) 2024-04-24T08:18:34.841 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:34.855 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:36.929 INFO:teuthology.orchestra.run.smithi123.stdout:generic/198 2s 2024-04-24T08:18:36.945 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/198 2024-04-24T08:18:36.945 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:36.954 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:36.987 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:36.987 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/199) 2024-04-24T08:18:37.698 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:37.711 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:37.711 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:38.088 INFO:teuthology.orchestra.run.smithi123.stdout:generic/199 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:38.101 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/199 2024-04-24T08:18:38.101 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/199 2024-04-24T08:18:38.101 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:38.111 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:38.138 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:38.138 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/200) 2024-04-24T08:18:38.645 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:38.658 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:39.071 INFO:teuthology.orchestra.run.smithi123.stdout:generic/200 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:39.086 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/200 2024-04-24T08:18:39.086 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/200 2024-04-24T08:18:39.086 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:39.098 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:39.134 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:39.135 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/201) 2024-04-24T08:18:39.668 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:39.682 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:39.682 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:40.120 INFO:teuthology.orchestra.run.smithi123.stdout:generic/201 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:40.130 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/201 2024-04-24T08:18:40.130 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/201 2024-04-24T08:18:40.130 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:40.138 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:40.167 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:40.167 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/202) 2024-04-24T08:18:40.711 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:40.724 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:40.724 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:41.143 INFO:teuthology.orchestra.run.smithi123.stdout:generic/202 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:41.155 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/202 2024-04-24T08:18:41.156 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/202 2024-04-24T08:18:41.156 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:41.166 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:41.197 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:41.197 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/203) 2024-04-24T08:18:41.745 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:41.756 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:41.756 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:42.207 INFO:teuthology.orchestra.run.smithi123.stdout:generic/203 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:42.221 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/203 2024-04-24T08:18:42.221 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/203 2024-04-24T08:18:42.221 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:42.233 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:42.265 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:42.265 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/204) 2024-04-24T08:18:42.820 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:42.833 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:42.833 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:43.576 INFO:teuthology.orchestra.run.smithi123.stdout:generic/204 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:43.588 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/204 2024-04-24T08:18:43.588 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/204 2024-04-24T08:18:43.588 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:43.623 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:43.624 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/205) 2024-04-24T08:18:44.174 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:44.189 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:44.189 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:44.602 INFO:teuthology.orchestra.run.smithi123.stdout:generic/205 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:44.612 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/205 2024-04-24T08:18:44.612 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/205 2024-04-24T08:18:44.612 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:44.619 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:44.645 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:44.645 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/206) 2024-04-24T08:18:45.154 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:45.241 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:45.241 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:45.316 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:18:45.321 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:18:45.497 INFO:teuthology.orchestra.run.smithi123.stdout:generic/206 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:18:45.506 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/206 2024-04-24T08:18:45.506 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/206 2024-04-24T08:18:45.506 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:18:45.514 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:18:45.537 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:18:45.537 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/207) 2024-04-24T08:18:45.997 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:18:46.009 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:18:46.009 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:19:15.445 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:19:15.454 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:19:32.913 INFO:teuthology.orchestra.run.smithi123.stdout:generic/207 46s 2024-04-24T08:19:32.935 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/207 2024-04-24T08:19:32.935 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:19:32.947 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:19:33.000 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:19:33.000 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/208) 2024-04-24T08:19:33.683 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:19:33.695 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:19:33.695 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:19:45.571 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:19:45.580 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:20:15.697 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:20:15.706 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:20:45.895 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:20:45.904 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:21:16.042 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:21:16.052 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:21:46.191 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:21:46.203 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:22:16.344 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:22:16.353 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:22:46.491 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:22:46.499 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:22:54.453 INFO:teuthology.orchestra.run.smithi123.stdout:generic/208 201s 2024-04-24T08:22:54.476 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/208 2024-04-24T08:22:54.477 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:22:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:22:54.521 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:22:54.522 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/209) 2024-04-24T08:22:55.230 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:22:55.242 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:22:55.242 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:16.604 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:23:16.613 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:23:22.348 INFO:teuthology.orchestra.run.smithi123.stdout:generic/209 [failed, exit status 1]- output mismatch (see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/209.out.bad) 2024-04-24T08:23:22.364 INFO:teuthology.orchestra.run.smithi123.stdout: --- tests/generic/209.out 2024-04-24 05:47:53.254732498 +0000 2024-04-24T08:23:22.364 INFO:teuthology.orchestra.run.smithi123.stdout: +++ /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/209.out.bad 2024-04-24 08:23:22.336990674 +0000 2024-04-24T08:23:22.364 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -1,2 +1,2 @@ 2024-04-24T08:23:22.364 INFO:teuthology.orchestra.run.smithi123.stdout: QA output created by 209 2024-04-24T08:23:22.364 INFO:teuthology.orchestra.run.smithi123.stdout: -test ran for 30 seconds without error 2024-04-24T08:23:22.364 INFO:teuthology.orchestra.run.smithi123.stdout: +reader found old byte at pos 4096 2024-04-24T08:23:22.364 INFO:teuthology.orchestra.run.smithi123.stdout: \ No newline at end of file 2024-04-24T08:23:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/209 2024-04-24T08:23:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:Failures: generic/209 2024-04-24T08:23:22.384 INFO:teuthology.orchestra.run.smithi123.stdout:Failed 1 of 1 tests 2024-04-24T08:23:22.395 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:22.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:23:22.443 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-24T08:23:22.443 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-24T08:23:22.444 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/210) 2024-04-24T08:23:23.026 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:23.040 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:23.040 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:23.691 INFO:teuthology.orchestra.run.smithi123.stdout:generic/210 0s 2024-04-24T08:23:23.709 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/210 2024-04-24T08:23:23.709 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:23.717 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:23.759 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:23.759 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/211) 2024-04-24T08:23:24.412 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:24.423 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:24.423 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:25.413 INFO:teuthology.orchestra.run.smithi123.stdout:generic/211 1s 2024-04-24T08:23:25.429 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/211 2024-04-24T08:23:25.429 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:25.440 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:25.475 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:25.476 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/212) 2024-04-24T08:23:26.000 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:26.014 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:26.014 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:26.638 INFO:teuthology.orchestra.run.smithi123.stdout:generic/212 0s 2024-04-24T08:23:26.656 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/212 2024-04-24T08:23:26.656 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:26.665 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:26.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:26.693 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/213) 2024-04-24T08:23:27.211 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:27.224 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:27.224 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:28.062 INFO:teuthology.orchestra.run.smithi123.stdout:generic/213 - output mismatch (see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/213.out.bad) 2024-04-24T08:23:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: --- tests/generic/213.out 2024-04-24 05:47:53.255732474 +0000 2024-04-24T08:23:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: +++ /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/213.out.bad 2024-04-24 08:23:28.057853987 +0000 2024-04-24T08:23:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -1,4 +1,3 @@ 2024-04-24T08:23:28.067 INFO:teuthology.orchestra.run.smithi123.stdout: QA output created by 213 2024-04-24T08:23:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: We should get: fallocate: No space left on device 2024-04-24T08:23:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: Strangely, xfs_io sometimes says "Success" when something went wrong, FYI 2024-04-24T08:23:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: -fallocate: No space left on device 2024-04-24T08:23:28.073 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/213 2024-04-24T08:23:28.074 INFO:teuthology.orchestra.run.smithi123.stdout:Failures: generic/213 2024-04-24T08:23:28.074 INFO:teuthology.orchestra.run.smithi123.stdout:Failed 1 of 1 tests 2024-04-24T08:23:28.080 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:28.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:23:28.117 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-24T08:23:28.117 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-24T08:23:28.117 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/214) 2024-04-24T08:23:28.872 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:28.886 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:28.886 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:29.858 INFO:teuthology.orchestra.run.smithi123.stdout:generic/214 1s 2024-04-24T08:23:29.874 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/214 2024-04-24T08:23:29.874 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:29.886 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:29.923 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:29.923 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/215) 2024-04-24T08:23:30.709 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:30.723 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:30.723 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:34.740 INFO:teuthology.orchestra.run.smithi123.stdout:generic/215 4s 2024-04-24T08:23:34.758 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/215 2024-04-24T08:23:34.758 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:34.765 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:34.796 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:34.796 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/216) 2024-04-24T08:23:35.351 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:35.365 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:35.365 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:35.763 INFO:teuthology.orchestra.run.smithi123.stdout:generic/216 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:35.775 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/216 2024-04-24T08:23:35.775 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/216 2024-04-24T08:23:35.775 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:35.784 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:35.815 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:35.815 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/217) 2024-04-24T08:23:36.361 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:36.381 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:36.381 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:36.766 INFO:teuthology.orchestra.run.smithi123.stdout:generic/217 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:36.778 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/217 2024-04-24T08:23:36.778 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/217 2024-04-24T08:23:36.778 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:36.789 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:36.822 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:36.822 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/218) 2024-04-24T08:23:37.382 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:37.397 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:37.398 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:37.794 INFO:teuthology.orchestra.run.smithi123.stdout:generic/218 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:37.807 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/218 2024-04-24T08:23:37.807 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/218 2024-04-24T08:23:37.807 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:37.817 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:37.847 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:37.848 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/219) 2024-04-24T08:23:38.409 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:38.422 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:38.862 INFO:teuthology.orchestra.run.smithi123.stdout:generic/219 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/219 2024-04-24T08:23:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/219 2024-04-24T08:23:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:38.885 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:38.921 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:38.921 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/220) 2024-04-24T08:23:39.477 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:39.490 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:39.490 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:39.883 INFO:teuthology.orchestra.run.smithi123.stdout:generic/220 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:39.894 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/220 2024-04-24T08:23:39.894 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/220 2024-04-24T08:23:39.894 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:39.903 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:39.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:39.933 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/221) 2024-04-24T08:23:40.501 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:40.515 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:40.515 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:42.005 INFO:teuthology.orchestra.run.smithi123.stdout:generic/221 1s 2024-04-24T08:23:42.022 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/221 2024-04-24T08:23:42.022 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:42.032 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:42.060 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:42.060 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/222) 2024-04-24T08:23:42.616 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:42.630 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:42.630 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:43.005 INFO:teuthology.orchestra.run.smithi123.stdout:generic/222 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:43.015 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/222 2024-04-24T08:23:43.015 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/222 2024-04-24T08:23:43.016 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:43.025 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:43.056 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:43.056 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/223) 2024-04-24T08:23:43.655 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:43.669 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:43.669 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:45.230 INFO:teuthology.orchestra.run.smithi123.stdout:generic/223 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:45.240 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/223 2024-04-24T08:23:45.240 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/223 2024-04-24T08:23:45.240 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:45.249 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:45.280 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:45.281 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/224) 2024-04-24T08:23:45.977 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:45.991 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:45.992 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:46.462 INFO:teuthology.orchestra.run.smithi123.stdout:generic/224 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:46.475 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/224 2024-04-24T08:23:46.475 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/224 2024-04-24T08:23:46.475 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:46.484 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:46.512 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:46.512 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/225) 2024-04-24T08:23:47.054 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:23:47.060 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:23:47.146 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:47.161 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:47.161 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:47.537 INFO:teuthology.orchestra.run.smithi123.stdout:generic/225 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:47.549 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/225 2024-04-24T08:23:47.549 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/225 2024-04-24T08:23:47.549 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:47.593 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:47.594 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/226) 2024-04-24T08:23:48.375 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:48.390 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:48.391 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:48.780 INFO:teuthology.orchestra.run.smithi123.stdout:generic/226 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:48.790 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/226 2024-04-24T08:23:48.790 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/226 2024-04-24T08:23:48.791 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:48.828 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:48.828 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/227) 2024-04-24T08:23:49.402 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:49.416 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:49.416 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:49.881 INFO:teuthology.orchestra.run.smithi123.stdout:generic/227 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:49.891 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/227 2024-04-24T08:23:49.891 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/227 2024-04-24T08:23:49.891 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:49.900 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:49.931 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:49.931 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/228) 2024-04-24T08:23:50.557 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:50.570 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:50.571 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:51.203 INFO:teuthology.orchestra.run.smithi123.stdout:generic/228 1s 2024-04-24T08:23:51.222 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/228 2024-04-24T08:23:51.222 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:51.230 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:51.285 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:51.286 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/229) 2024-04-24T08:23:52.062 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:52.075 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:52.075 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:52.458 INFO:teuthology.orchestra.run.smithi123.stdout:generic/229 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:52.469 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/229 2024-04-24T08:23:52.470 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/229 2024-04-24T08:23:52.470 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:52.481 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:52.510 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:52.511 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/230) 2024-04-24T08:23:53.072 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:53.085 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:53.085 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:53.478 INFO:teuthology.orchestra.run.smithi123.stdout:generic/230 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:53.491 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/230 2024-04-24T08:23:53.491 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/230 2024-04-24T08:23:53.491 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:53.500 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:53.536 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:53.537 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/231) 2024-04-24T08:23:54.216 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:54.229 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:54.716 INFO:teuthology.orchestra.run.smithi123.stdout:generic/231 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:54.733 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/231 2024-04-24T08:23:54.733 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/231 2024-04-24T08:23:54.733 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:54.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:54.774 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/232) 2024-04-24T08:23:55.337 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:55.352 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:55.352 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:55.738 INFO:teuthology.orchestra.run.smithi123.stdout:generic/232 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:55.753 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/232 2024-04-24T08:23:55.753 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/232 2024-04-24T08:23:55.753 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:55.761 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:55.795 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:55.795 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/233) 2024-04-24T08:23:56.358 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:56.370 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:56.370 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:56.753 INFO:teuthology.orchestra.run.smithi123.stdout:generic/233 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:56.766 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/233 2024-04-24T08:23:56.766 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/233 2024-04-24T08:23:56.767 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:56.777 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:56.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:56.813 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/234) 2024-04-24T08:23:57.379 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:57.391 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:57.391 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:57.790 INFO:teuthology.orchestra.run.smithi123.stdout:generic/234 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:57.801 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/234 2024-04-24T08:23:57.801 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/234 2024-04-24T08:23:57.801 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:57.811 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:57.843 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:57.843 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/235) 2024-04-24T08:23:58.393 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:58.408 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:58.408 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:58.809 INFO:teuthology.orchestra.run.smithi123.stdout:generic/235 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:23:58.821 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/235 2024-04-24T08:23:58.821 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/235 2024-04-24T08:23:58.821 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:23:58.832 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:23:58.864 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:23:58.865 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/236) 2024-04-24T08:23:59.406 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:23:59.422 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:23:59.422 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:24:01.128 INFO:teuthology.orchestra.run.smithi123.stdout:generic/236 2s 2024-04-24T08:24:01.219 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/236 2024-04-24T08:24:01.220 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:24:01.229 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:24:01.278 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:24:01.278 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/237) 2024-04-24T08:24:02.070 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:24:02.084 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:24:02.084 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:24:02.571 INFO:teuthology.orchestra.run.smithi123.stdout:generic/237 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-24T08:24:02.580 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/237 2024-04-24T08:24:02.580 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/237 2024-04-24T08:24:02.580 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:24:02.589 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:24:02.620 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:24:02.620 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/238) 2024-04-24T08:24:03.180 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:24:03.194 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:24:03.194 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:24:03.584 INFO:teuthology.orchestra.run.smithi123.stdout:generic/238 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:24:03.595 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/238 2024-04-24T08:24:03.595 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/238 2024-04-24T08:24:03.595 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:24:03.603 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:24:03.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:24:03.635 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/239) 2024-04-24T08:24:04.207 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:24:04.220 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:24:04.220 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:24:04.689 INFO:teuthology.orchestra.run.smithi123.stdout:generic/239 [not run] Sparse files are not supported or do not work as expected 2024-04-24T08:24:04.701 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/239 2024-04-24T08:24:04.701 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/239 2024-04-24T08:24:04.701 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:24:04.710 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:24:04.743 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:24:04.743 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/240) 2024-04-24T08:24:05.301 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:24:05.315 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:24:05.315 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:24:05.766 INFO:teuthology.orchestra.run.smithi123.stdout:generic/240 [not run] Sparse files are not supported or do not work as expected 2024-04-24T08:24:05.776 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/240 2024-04-24T08:24:05.776 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/240 2024-04-24T08:24:05.777 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:24:05.786 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:24:05.818 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:24:05.818 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/241) 2024-04-24T08:24:06.394 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:24:06.412 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:24:06.413 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:24:17.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:24:17.203 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:24:47.366 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:24:47.371 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:25:18.847 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:25:18.856 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:25:18.949 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T08:25:18.948+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 44925) UID: 0 2024-04-24T08:25:18.949 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T08:25:18.948+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 44925) UID: 0 2024-04-24T08:25:18.949 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T08:25:18.949+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 44925) UID: 0 2024-04-24T08:25:18.950 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T08:25:18.949+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 44925) UID: 0 2024-04-24T08:25:22.810 INFO:teuthology.orchestra.run.smithi123.stdout:generic/241 76s 2024-04-24T08:25:22.851 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/241 2024-04-24T08:25:22.851 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:25:22.859 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:25:22.892 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:25:22.892 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/242) 2024-04-24T08:25:23.926 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:25:23.945 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:25:23.946 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:25:24.369 INFO:teuthology.orchestra.run.smithi123.stdout:generic/242 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:25:24.384 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/242 2024-04-24T08:25:24.384 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/242 2024-04-24T08:25:24.384 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:25:24.393 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:25:24.426 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:25:24.427 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/243) 2024-04-24T08:25:24.955 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:25:24.967 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:25:24.967 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:25:25.320 INFO:teuthology.orchestra.run.smithi123.stdout:generic/243 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:25:25.329 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/243 2024-04-24T08:25:25.329 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/243 2024-04-24T08:25:25.329 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:25:25.337 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:25:25.367 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:25:25.367 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/244) 2024-04-24T08:25:25.895 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:25:25.906 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:25:25.907 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:25:26.267 INFO:teuthology.orchestra.run.smithi123.stdout:generic/244 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-24T08:25:26.279 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/244 2024-04-24T08:25:26.279 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/244 2024-04-24T08:25:26.279 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:25:26.288 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:25:26.324 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:25:26.324 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/245) 2024-04-24T08:25:26.915 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:25:26.930 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:25:26.930 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:25:27.512 INFO:teuthology.orchestra.run.smithi123.stdout:generic/245 1s 2024-04-24T08:25:27.531 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/245 2024-04-24T08:25:27.532 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:25:27.543 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:25:27.593 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:25:27.594 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/246) 2024-04-24T08:25:28.316 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:25:28.331 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:25:28.331 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:25:29.930 INFO:teuthology.orchestra.run.smithi123.stdout:generic/246 1s 2024-04-24T08:25:29.957 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/246 2024-04-24T08:25:29.957 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:25:29.962 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:25:30.011 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:25:30.011 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/247) 2024-04-24T08:25:30.711 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:25:30.727 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:25:30.727 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:24.946 INFO:teuthology.orchestra.run.smithi123.stdout:generic/247 54s 2024-04-24T08:26:24.965 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/247 2024-04-24T08:26:24.965 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:24.975 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:25.003 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:25.004 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/248) 2024-04-24T08:26:25.543 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:25.558 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:25.558 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:26.069 INFO:teuthology.orchestra.run.smithi123.stdout:generic/248 1s 2024-04-24T08:26:26.086 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/248 2024-04-24T08:26:26.086 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:26.096 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:26.128 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:26.128 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/249) 2024-04-24T08:26:26.676 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:26.689 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:26.689 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:30.711 INFO:teuthology.orchestra.run.smithi123.stdout:generic/249 4s 2024-04-24T08:26:30.727 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/249 2024-04-24T08:26:30.728 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:30.739 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:30.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:30.774 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/250) 2024-04-24T08:26:31.532 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:31.545 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:31.545 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:32.020 INFO:teuthology.orchestra.run.smithi123.stdout:generic/250 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:26:32.032 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/250 2024-04-24T08:26:32.032 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/250 2024-04-24T08:26:32.032 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:32.040 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:32.070 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:32.070 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/251) 2024-04-24T08:26:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:32.625 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:33.048 INFO:teuthology.orchestra.run.smithi123.stdout:generic/251 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:26:33.059 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/251 2024-04-24T08:26:33.059 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/251 2024-04-24T08:26:33.059 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:33.066 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:33.095 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:33.096 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/252) 2024-04-24T08:26:33.662 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:33.677 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:33.677 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:34.091 INFO:teuthology.orchestra.run.smithi123.stdout:generic/252 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:26:34.101 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/252 2024-04-24T08:26:34.101 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/252 2024-04-24T08:26:34.101 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:34.112 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:34.145 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:34.146 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/253) 2024-04-24T08:26:34.679 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:34.693 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:34.693 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:35.066 INFO:teuthology.orchestra.run.smithi123.stdout:generic/253 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:26:35.078 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/253 2024-04-24T08:26:35.078 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/253 2024-04-24T08:26:35.078 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:35.086 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:35.126 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:35.126 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/254) 2024-04-24T08:26:35.665 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:35.678 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:35.679 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:36.055 INFO:teuthology.orchestra.run.smithi123.stdout:generic/254 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:26:36.064 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/254 2024-04-24T08:26:36.064 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/254 2024-04-24T08:26:36.064 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:36.073 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:36.101 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:36.101 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/255) 2024-04-24T08:26:36.649 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:36.665 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:36.665 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:39.083 INFO:teuthology.orchestra.run.smithi123.stdout:generic/255 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-24T08:26:39.094 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/255 2024-04-24T08:26:39.094 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/255 2024-04-24T08:26:39.094 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:39.105 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:39.134 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:39.134 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/256) 2024-04-24T08:26:39.875 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:39.890 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:39.890 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:42.266 INFO:teuthology.orchestra.run.smithi123.stdout:generic/256 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:26:42.276 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/256 2024-04-24T08:26:42.276 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/256 2024-04-24T08:26:42.276 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:42.283 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:42.311 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:42.311 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/257) 2024-04-24T08:26:42.859 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:42.874 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:42.874 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:45.274 INFO:teuthology.orchestra.run.smithi123.stdout:generic/257 3s 2024-04-24T08:26:46.407 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/257 2024-04-24T08:26:46.407 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:46.420 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:46.446 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:46.446 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/258) 2024-04-24T08:26:47.238 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:47.250 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:47.250 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:48.239 INFO:teuthology.orchestra.run.smithi123.stdout:generic/258 [failed, exit status 1]- output mismatch (see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/258.out.bad) 2024-04-24T08:26:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: --- tests/generic/258.out 2024-04-24 05:47:53.262732309 +0000 2024-04-24T08:26:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: +++ /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/258.out.bad 2024-04-24 08:26:48.199091723 +0000 2024-04-24T08:26:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -1,5 +1,6 @@ 2024-04-24T08:26:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: QA output created by 258 2024-04-24T08:26:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: Creating file with timestamp of Jan 1, 1960 2024-04-24T08:26:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: Testing for negative seconds since epoch 2024-04-24T08:26:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: -Remounting to flush cache 2024-04-24T08:26:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: -Testing for negative seconds since epoch 2024-04-24T08:26:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: +Timestamp wrapped: 3979351756 2024-04-24T08:26:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: +Timestamp wrapped 2024-04-24T08:26:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: +(see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/258.full for details) 2024-04-24T08:26:48.254 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/258 2024-04-24T08:26:48.254 INFO:teuthology.orchestra.run.smithi123.stdout:Failures: generic/258 2024-04-24T08:26:48.255 INFO:teuthology.orchestra.run.smithi123.stdout:Failed 1 of 1 tests 2024-04-24T08:26:48.262 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:48.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:26:48.287 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-24T08:26:48.287 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-24T08:26:48.287 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/259) 2024-04-24T08:26:48.939 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:48.952 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:48.953 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:49.320 INFO:teuthology.orchestra.run.smithi123.stdout:generic/259 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:26:49.330 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/259 2024-04-24T08:26:49.330 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/259 2024-04-24T08:26:49.330 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:49.339 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:49.370 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:49.370 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/260) 2024-04-24T08:26:49.938 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:49.952 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:49.952 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:50.323 INFO:teuthology.orchestra.run.smithi123.stdout:generic/260 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:26:50.335 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/260 2024-04-24T08:26:50.335 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/260 2024-04-24T08:26:50.335 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:50.375 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:50.376 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/261) 2024-04-24T08:26:50.914 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:50.930 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:50.930 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:51.335 INFO:teuthology.orchestra.run.smithi123.stdout:generic/261 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:26:51.348 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/261 2024-04-24T08:26:51.349 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/261 2024-04-24T08:26:51.349 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:51.391 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:51.391 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/262) 2024-04-24T08:26:51.970 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:51.985 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:51.985 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:52.379 INFO:teuthology.orchestra.run.smithi123.stdout:generic/262 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:26:52.391 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/262 2024-04-24T08:26:52.391 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/262 2024-04-24T08:26:52.392 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:26:52.403 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:26:52.436 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:26:52.436 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/263) 2024-04-24T08:26:52.984 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:26:52.998 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:26:52.998 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:51.190 INFO:teuthology.orchestra.run.smithi123.stdout:generic/263 118s 2024-04-24T08:28:51.212 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/263 2024-04-24T08:28:51.212 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:28:51.224 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:51.276 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:28:51.276 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/264) 2024-04-24T08:28:51.957 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:28:51.968 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:28:51.968 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:52.405 INFO:teuthology.orchestra.run.smithi123.stdout:generic/264 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:28:52.415 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/264 2024-04-24T08:28:52.415 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/264 2024-04-24T08:28:52.415 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:28:52.425 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:52.457 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:28:52.457 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/265) 2024-04-24T08:28:52.992 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:28:53.006 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:28:53.006 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:53.499 INFO:teuthology.orchestra.run.smithi123.stdout:generic/265 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:28:53.512 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/265 2024-04-24T08:28:53.512 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/265 2024-04-24T08:28:53.512 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:28:53.521 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:53.553 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:28:53.554 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/266) 2024-04-24T08:28:54.114 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:28:54.126 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:28:54.126 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:54.616 INFO:teuthology.orchestra.run.smithi123.stdout:generic/266 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:28:54.627 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/266 2024-04-24T08:28:54.628 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/266 2024-04-24T08:28:54.628 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:28:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:54.681 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:28:54.681 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/267) 2024-04-24T08:28:55.220 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:28:55.235 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:28:55.235 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:55.673 INFO:teuthology.orchestra.run.smithi123.stdout:generic/267 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:28:55.688 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/267 2024-04-24T08:28:55.688 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/267 2024-04-24T08:28:55.688 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:28:55.700 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:55.732 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:28:55.733 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/268) 2024-04-24T08:28:56.268 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:28:56.283 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:28:56.283 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:56.745 INFO:teuthology.orchestra.run.smithi123.stdout:generic/268 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:28:56.760 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/268 2024-04-24T08:28:56.760 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/268 2024-04-24T08:28:56.760 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:28:56.772 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:56.803 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:28:56.803 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/269) 2024-04-24T08:28:57.374 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:28:57.388 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:28:57.388 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:57.804 INFO:teuthology.orchestra.run.smithi123.stdout:generic/269 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:28:57.816 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/269 2024-04-24T08:28:57.816 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/269 2024-04-24T08:28:57.817 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:28:57.827 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:57.860 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:28:57.860 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/270) 2024-04-24T08:28:58.389 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:28:58.401 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:28:58.401 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:58.845 INFO:teuthology.orchestra.run.smithi123.stdout:generic/270 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-24T08:28:58.858 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/270 2024-04-24T08:28:58.858 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/270 2024-04-24T08:28:58.858 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:28:58.870 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:58.900 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:28:58.900 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/271) 2024-04-24T08:28:59.426 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:28:59.439 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:28:59.439 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:59.922 INFO:teuthology.orchestra.run.smithi123.stdout:generic/271 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:28:59.936 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/271 2024-04-24T08:28:59.936 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/271 2024-04-24T08:28:59.937 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:28:59.948 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:28:59.984 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:28:59.984 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/272) 2024-04-24T08:29:00.528 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:00.541 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:00.541 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:00.975 INFO:teuthology.orchestra.run.smithi123.stdout:generic/272 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:29:00.987 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/272 2024-04-24T08:29:00.987 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/272 2024-04-24T08:29:00.988 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:29:00.999 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:01.029 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:29:01.029 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/273) 2024-04-24T08:29:01.571 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:01.584 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:01.997 INFO:teuthology.orchestra.run.smithi123.stdout:generic/273 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:29:02.010 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/273 2024-04-24T08:29:02.011 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/273 2024-04-24T08:29:02.011 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:29:02.023 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:02.055 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:29:02.055 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/274) 2024-04-24T08:29:02.590 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:02.606 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:02.920 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:29:02.926 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:29:03.024 INFO:teuthology.orchestra.run.smithi123.stdout:generic/274 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:29:03.035 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/274 2024-04-24T08:29:03.035 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/274 2024-04-24T08:29:03.035 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:29:03.043 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:03.074 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:29:03.075 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/275) 2024-04-24T08:29:03.661 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:03.674 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:03.674 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:04.101 INFO:teuthology.orchestra.run.smithi123.stdout:generic/275 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:29:04.112 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/275 2024-04-24T08:29:04.112 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/275 2024-04-24T08:29:04.112 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:29:04.120 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:04.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:29:04.150 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/276) 2024-04-24T08:29:04.702 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:04.717 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:04.717 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:05.196 INFO:teuthology.orchestra.run.smithi123.stdout:generic/276 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:29:05.208 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/276 2024-04-24T08:29:05.208 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/276 2024-04-24T08:29:05.208 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:29:05.219 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:05.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:29:05.253 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/277) 2024-04-24T08:29:05.791 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:05.803 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:06.229 INFO:teuthology.orchestra.run.smithi123.stdout:generic/277 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:29:06.241 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/277 2024-04-24T08:29:06.241 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/277 2024-04-24T08:29:06.241 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:29:06.251 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:06.281 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:29:06.282 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/278) 2024-04-24T08:29:06.826 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:06.841 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:07.319 INFO:teuthology.orchestra.run.smithi123.stdout:generic/278 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:29:07.335 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/278 2024-04-24T08:29:07.336 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/278 2024-04-24T08:29:07.336 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:29:07.346 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:07.378 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:29:07.378 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/279) 2024-04-24T08:29:07.916 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:07.928 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:07.928 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:08.391 INFO:teuthology.orchestra.run.smithi123.stdout:generic/279 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:29:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/279 2024-04-24T08:29:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/279 2024-04-24T08:29:08.403 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:29:08.413 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:08.441 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:29:08.441 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/280) 2024-04-24T08:29:08.998 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:09.012 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:09.012 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:09.495 INFO:teuthology.orchestra.run.smithi123.stdout:generic/280 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:29:09.508 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/280 2024-04-24T08:29:09.508 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/280 2024-04-24T08:29:09.508 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:29:09.518 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:09.549 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:29:09.549 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/281) 2024-04-24T08:29:10.074 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:10.087 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:10.087 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:10.567 INFO:teuthology.orchestra.run.smithi123.stdout:generic/281 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:29:10.581 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/281 2024-04-24T08:29:10.581 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/281 2024-04-24T08:29:10.581 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:29:10.592 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:10.623 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:29:10.623 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/282) 2024-04-24T08:29:11.193 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:11.206 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:11.207 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:11.658 INFO:teuthology.orchestra.run.smithi123.stdout:generic/282 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:29:11.670 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/282 2024-04-24T08:29:11.670 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/282 2024-04-24T08:29:11.671 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:29:11.682 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:11.708 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:29:11.709 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/283) 2024-04-24T08:29:12.253 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:12.264 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:12.265 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:12.724 INFO:teuthology.orchestra.run.smithi123.stdout:generic/283 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:29:12.737 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/283 2024-04-24T08:29:12.737 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/283 2024-04-24T08:29:12.738 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:29:12.749 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:12.780 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:29:12.780 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/284) 2024-04-24T08:29:13.302 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:13.314 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:13.314 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:13.740 INFO:teuthology.orchestra.run.smithi123.stdout:generic/284 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:29:13.754 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/284 2024-04-24T08:29:13.754 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/284 2024-04-24T08:29:13.754 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:29:13.765 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:13.800 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:29:13.800 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/285) 2024-04-24T08:29:14.329 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:14.342 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:14.342 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:20.230 INFO:teuthology.orchestra.run.smithi123.stdout:generic/285 6s 2024-04-24T08:29:20.244 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/285 2024-04-24T08:29:20.245 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:29:20.253 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:20.282 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:29:20.283 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/286) 2024-04-24T08:29:21.037 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:29:21.050 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:29:21.050 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:29:33.961 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:29:33.971 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:30:04.094 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:30:04.103 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:30:34.566 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:30:34.577 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:31:04.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:31:04.714 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:31:34.886 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:31:34.896 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:32:05.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:32:05.084 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:32:21.253 INFO:teuthology.orchestra.run.smithi123.stdout:generic/286 180s 2024-04-24T08:32:21.270 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/286 2024-04-24T08:32:21.271 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:21.281 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:21.312 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:21.312 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/287) 2024-04-24T08:32:21.987 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:21.999 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:21.999 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:22.452 INFO:teuthology.orchestra.run.smithi123.stdout:generic/287 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:22.464 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/287 2024-04-24T08:32:22.464 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/287 2024-04-24T08:32:22.464 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:22.475 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:22.503 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:22.503 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/288) 2024-04-24T08:32:22.940 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:22.952 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:22.952 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:23.386 INFO:teuthology.orchestra.run.smithi123.stdout:generic/288 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:23.397 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/288 2024-04-24T08:32:23.398 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/288 2024-04-24T08:32:23.398 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:23.406 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:23.441 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:23.442 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/289) 2024-04-24T08:32:24.025 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:24.039 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:24.039 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:24.492 INFO:teuthology.orchestra.run.smithi123.stdout:generic/289 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:24.502 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/289 2024-04-24T08:32:24.502 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/289 2024-04-24T08:32:24.502 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:24.511 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:24.539 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:24.539 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/290) 2024-04-24T08:32:25.052 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:25.065 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:25.065 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:25.459 INFO:teuthology.orchestra.run.smithi123.stdout:generic/290 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:25.469 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/290 2024-04-24T08:32:25.469 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/290 2024-04-24T08:32:25.469 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:25.478 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:25.506 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:25.506 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/291) 2024-04-24T08:32:26.044 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:26.057 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:26.058 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:26.478 INFO:teuthology.orchestra.run.smithi123.stdout:generic/291 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:26.490 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/291 2024-04-24T08:32:26.490 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/291 2024-04-24T08:32:26.490 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:26.502 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:26.535 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:26.535 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/292) 2024-04-24T08:32:27.077 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:27.090 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:27.541 INFO:teuthology.orchestra.run.smithi123.stdout:generic/292 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:27.553 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/292 2024-04-24T08:32:27.553 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/292 2024-04-24T08:32:27.553 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:27.563 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:27.594 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:27.594 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/293) 2024-04-24T08:32:28.129 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:28.143 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:28.144 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:28.566 INFO:teuthology.orchestra.run.smithi123.stdout:generic/293 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:28.576 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/293 2024-04-24T08:32:28.576 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/293 2024-04-24T08:32:28.576 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:28.585 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:28.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:28.615 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/294) 2024-04-24T08:32:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:29.156 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:29.567 INFO:teuthology.orchestra.run.smithi123.stdout:generic/294 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:29.577 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/294 2024-04-24T08:32:29.577 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/294 2024-04-24T08:32:29.577 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:29.585 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:29.614 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:29.615 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/295) 2024-04-24T08:32:30.135 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:30.148 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:30.148 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:30.580 INFO:teuthology.orchestra.run.smithi123.stdout:generic/295 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:30.590 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/295 2024-04-24T08:32:30.591 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/295 2024-04-24T08:32:30.591 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:30.599 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:30.628 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:30.628 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/296) 2024-04-24T08:32:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:31.163 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:31.581 INFO:teuthology.orchestra.run.smithi123.stdout:generic/296 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:31.594 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/296 2024-04-24T08:32:31.594 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/296 2024-04-24T08:32:31.594 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:31.604 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:31.637 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:31.637 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/297) 2024-04-24T08:32:32.201 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:32.211 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:32.211 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:32.692 INFO:teuthology.orchestra.run.smithi123.stdout:generic/297 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:32.705 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/297 2024-04-24T08:32:32.705 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/297 2024-04-24T08:32:32.705 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:32.715 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:32.745 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:32.746 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/298) 2024-04-24T08:32:33.284 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:33.298 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:33.701 INFO:teuthology.orchestra.run.smithi123.stdout:generic/298 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:33.710 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/298 2024-04-24T08:32:33.711 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/298 2024-04-24T08:32:33.711 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:33.718 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:33.745 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:33.746 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/299) 2024-04-24T08:32:34.278 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:34.291 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:34.291 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:34.768 INFO:teuthology.orchestra.run.smithi123.stdout:generic/299 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:34.782 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/299 2024-04-24T08:32:34.783 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/299 2024-04-24T08:32:34.783 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:34.794 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:34.831 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:34.831 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/300) 2024-04-24T08:32:35.291 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:32:35.296 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:32:35.386 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:35.398 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:35.398 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:35.773 INFO:teuthology.orchestra.run.smithi123.stdout:generic/300 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:35.785 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/300 2024-04-24T08:32:35.786 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/300 2024-04-24T08:32:35.786 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:35.798 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:35.834 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:35.834 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/301) 2024-04-24T08:32:36.416 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:36.430 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:36.430 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:36.836 INFO:teuthology.orchestra.run.smithi123.stdout:generic/301 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:36.849 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/301 2024-04-24T08:32:36.850 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/301 2024-04-24T08:32:36.850 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:36.861 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:36.894 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:36.894 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/302) 2024-04-24T08:32:37.408 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:37.421 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:37.421 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:37.851 INFO:teuthology.orchestra.run.smithi123.stdout:generic/302 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:37.861 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/302 2024-04-24T08:32:37.862 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/302 2024-04-24T08:32:37.862 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:37.872 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:37.901 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:37.901 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/303) 2024-04-24T08:32:38.438 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:38.452 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:38.994 INFO:teuthology.orchestra.run.smithi123.stdout:generic/303 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:32:39.006 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/303 2024-04-24T08:32:39.006 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/303 2024-04-24T08:32:39.006 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:39.015 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:39.065 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:39.065 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/304) 2024-04-24T08:32:39.742 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:39.755 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:40.273 INFO:teuthology.orchestra.run.smithi123.stdout:generic/304 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-24T08:32:40.284 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/304 2024-04-24T08:32:40.284 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/304 2024-04-24T08:32:40.284 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:40.292 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:40.345 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:40.345 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/305) 2024-04-24T08:32:40.975 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:40.985 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:40.986 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:41.406 INFO:teuthology.orchestra.run.smithi123.stdout:generic/305 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:41.419 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/305 2024-04-24T08:32:41.419 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/305 2024-04-24T08:32:41.419 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:41.429 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:41.465 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:41.465 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/306) 2024-04-24T08:32:42.016 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:42.030 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:42.030 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:42.459 INFO:teuthology.orchestra.run.smithi123.stdout:generic/306 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:42.473 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/306 2024-04-24T08:32:42.473 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/306 2024-04-24T08:32:42.473 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:42.484 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:42.516 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:42.516 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/307) 2024-04-24T08:32:43.053 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:43.067 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:43.067 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:43.510 INFO:teuthology.orchestra.run.smithi123.stdout:generic/307 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:32:43.524 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/307 2024-04-24T08:32:43.524 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/307 2024-04-24T08:32:43.524 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:43.568 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:43.569 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/308) 2024-04-24T08:32:44.122 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:44.135 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:44.686 INFO:teuthology.orchestra.run.smithi123.stdout:generic/308 0s 2024-04-24T08:32:44.707 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/308 2024-04-24T08:32:44.707 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:44.719 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:44.755 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:44.755 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/309) 2024-04-24T08:32:45.269 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:45.282 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:45.282 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:46.935 INFO:teuthology.orchestra.run.smithi123.stdout:generic/309 1s 2024-04-24T08:32:46.958 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/309 2024-04-24T08:32:46.958 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:32:46.969 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:32:47.022 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:32:47.022 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/310) 2024-04-24T08:32:47.682 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:32:47.693 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:32:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:33:05.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:33:05.517 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:33:35.704 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:33:35.713 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:34:05.897 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:34:05.906 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:34:37.084 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:34:37.092 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:34:42.139 INFO:teuthology.orchestra.run.smithi123.stdout:generic/310 115s 2024-04-24T08:34:42.154 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/310 2024-04-24T08:34:42.155 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:34:42.163 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:34:42.211 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:34:42.212 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/311) 2024-04-24T08:34:43.111 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:34:43.128 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:34:43.128 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:34:43.526 INFO:teuthology.orchestra.run.smithi123.stdout:generic/311 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:34:43.537 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/311 2024-04-24T08:34:43.537 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/311 2024-04-24T08:34:43.538 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:34:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:34:43.579 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:34:43.579 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/312) 2024-04-24T08:34:44.122 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:34:44.135 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:34:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:34:44.669 INFO:teuthology.orchestra.run.smithi123.stdout:generic/312 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:34:44.681 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/312 2024-04-24T08:34:44.681 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/312 2024-04-24T08:34:44.681 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:34:44.690 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:34:44.719 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:34:44.720 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/313) 2024-04-24T08:34:45.270 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:34:45.285 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:34:45.285 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:34:49.905 INFO:teuthology.orchestra.run.smithi123.stdout:generic/313 4s 2024-04-24T08:34:49.922 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/313 2024-04-24T08:34:49.922 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:34:49.931 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:34:49.961 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:34:49.961 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/314) 2024-04-24T08:34:50.472 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:34:50.485 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:34:50.486 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:34:51.754 INFO:teuthology.orchestra.run.smithi123.stdout:generic/314 1s 2024-04-24T08:34:51.769 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/314 2024-04-24T08:34:51.769 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:34:51.777 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:34:51.803 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:34:51.803 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/315) 2024-04-24T08:34:52.344 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:34:52.358 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:34:52.358 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:34:55.020 INFO:teuthology.orchestra.run.smithi123.stdout:generic/315 3s 2024-04-24T08:34:55.038 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/315 2024-04-24T08:34:55.038 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:34:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:34:55.081 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:34:55.082 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/316) 2024-04-24T08:34:55.750 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:34:55.762 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:34:55.762 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:00.281 INFO:teuthology.orchestra.run.smithi123.stdout:generic/316 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-24T08:35:00.292 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/316 2024-04-24T08:35:00.293 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/316 2024-04-24T08:35:00.293 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:35:00.302 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:00.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:35:00.330 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/317) 2024-04-24T08:35:01.037 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:35:01.049 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:35:01.049 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:01.467 INFO:teuthology.orchestra.run.smithi123.stdout:generic/317 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:35:01.478 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/317 2024-04-24T08:35:01.478 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/317 2024-04-24T08:35:01.478 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:35:01.487 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:01.518 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:35:01.518 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/318) 2024-04-24T08:35:02.056 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:35:02.070 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:35:02.070 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:02.484 INFO:teuthology.orchestra.run.smithi123.stdout:generic/318 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:35:02.495 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/318 2024-04-24T08:35:02.495 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/318 2024-04-24T08:35:02.495 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:35:02.504 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:02.532 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:35:02.533 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/319) 2024-04-24T08:35:03.068 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:35:03.081 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:35:03.081 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:03.576 INFO:teuthology.orchestra.run.smithi123.stdout:generic/319 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-24T08:35:03.587 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/319 2024-04-24T08:35:03.587 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/319 2024-04-24T08:35:03.587 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:35:03.595 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:03.622 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:35:03.622 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/320) 2024-04-24T08:35:04.158 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:35:04.174 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:35:04.174 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:04.585 INFO:teuthology.orchestra.run.smithi123.stdout:generic/320 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:35:04.597 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/320 2024-04-24T08:35:04.597 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/320 2024-04-24T08:35:04.597 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:35:04.607 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:04.636 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:35:04.637 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/321) 2024-04-24T08:35:05.153 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:35:05.166 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:35:05.166 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:05.666 INFO:teuthology.orchestra.run.smithi123.stdout:generic/321 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:35:05.678 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/321 2024-04-24T08:35:05.679 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/321 2024-04-24T08:35:05.679 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:35:05.690 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:05.722 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:35:05.723 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/322) 2024-04-24T08:35:06.280 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:35:06.294 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:35:06.294 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:06.734 INFO:teuthology.orchestra.run.smithi123.stdout:generic/322 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:35:06.747 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/322 2024-04-24T08:35:06.747 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/322 2024-04-24T08:35:06.748 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:35:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:06.792 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:35:06.792 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/323) 2024-04-24T08:35:07.206 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:35:07.211 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:35:07.304 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:35:07.315 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:35:07.315 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:35:37.395 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:35:37.400 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:36:07.548 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:36:07.553 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:36:38.663 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:36:38.673 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:36:39.621 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T08:36:39.620+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 310937) UID: 0 2024-04-24T08:36:39.622 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T08:36:39.620+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 310937) UID: 0 2024-04-24T08:36:39.622 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T08:36:39.620+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 310937) UID: 0 2024-04-24T08:36:39.623 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T08:36:39.622+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 310937) UID: 0 2024-04-24T08:37:18.353 INFO:teuthology.orchestra.run.smithi123.stdout:generic/323 130s 2024-04-24T08:37:18.375 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/323 2024-04-24T08:37:18.375 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:18.387 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:18.465 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:18.466 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/324) 2024-04-24T08:37:19.267 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:19.278 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:19.278 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:21.460 INFO:teuthology.orchestra.run.smithi123.stdout:generic/324 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:21.474 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/324 2024-04-24T08:37:21.474 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/324 2024-04-24T08:37:21.474 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:21.485 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:21.515 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:21.516 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/325) 2024-04-24T08:37:21.993 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:22.004 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:22.004 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:23.017 INFO:teuthology.orchestra.run.smithi123.stdout:generic/325 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:23.031 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/325 2024-04-24T08:37:23.032 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/325 2024-04-24T08:37:23.032 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:23.043 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:23.075 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:23.075 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/326) 2024-04-24T08:37:23.585 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:23.601 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:23.601 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:23.980 INFO:teuthology.orchestra.run.smithi123.stdout:generic/326 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:23.990 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/326 2024-04-24T08:37:23.990 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/326 2024-04-24T08:37:23.990 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:23.998 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:24.023 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:24.024 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/327) 2024-04-24T08:37:24.493 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:24.506 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:24.506 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:24.889 INFO:teuthology.orchestra.run.smithi123.stdout:generic/327 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:24.900 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/327 2024-04-24T08:37:24.900 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/327 2024-04-24T08:37:24.901 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:24.910 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:24.938 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:24.939 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/328) 2024-04-24T08:37:25.406 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:25.419 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:25.419 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:25.789 INFO:teuthology.orchestra.run.smithi123.stdout:generic/328 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:25.798 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/328 2024-04-24T08:37:25.799 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/328 2024-04-24T08:37:25.799 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:25.829 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:25.830 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/329) 2024-04-24T08:37:26.306 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:26.318 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:26.318 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:26.830 INFO:teuthology.orchestra.run.smithi123.stdout:generic/329 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:26.844 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/329 2024-04-24T08:37:26.844 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/329 2024-04-24T08:37:26.844 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:26.855 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:26.887 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:26.887 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/330) 2024-04-24T08:37:27.360 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:27.370 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:27.371 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:27.727 INFO:teuthology.orchestra.run.smithi123.stdout:generic/330 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:27.739 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/330 2024-04-24T08:37:27.739 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/330 2024-04-24T08:37:27.739 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:27.750 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:27.777 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:27.777 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/331) 2024-04-24T08:37:28.239 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:28.250 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:28.250 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:28.722 INFO:teuthology.orchestra.run.smithi123.stdout:generic/331 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:28.733 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/331 2024-04-24T08:37:28.734 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/331 2024-04-24T08:37:28.734 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:28.742 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:28.768 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:28.768 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/332) 2024-04-24T08:37:29.266 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:29.279 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:29.279 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:29.673 INFO:teuthology.orchestra.run.smithi123.stdout:generic/332 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:29.686 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/332 2024-04-24T08:37:29.687 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/332 2024-04-24T08:37:29.687 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:29.698 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:29.727 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:29.727 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/333) 2024-04-24T08:37:30.198 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:30.210 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:30.210 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:30.581 INFO:teuthology.orchestra.run.smithi123.stdout:generic/333 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:30.592 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/333 2024-04-24T08:37:30.592 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/333 2024-04-24T08:37:30.592 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:30.599 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:30.630 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:30.630 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/334) 2024-04-24T08:37:31.114 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:31.125 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:31.519 INFO:teuthology.orchestra.run.smithi123.stdout:generic/334 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:31.531 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/334 2024-04-24T08:37:31.531 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/334 2024-04-24T08:37:31.531 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:31.539 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:31.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:31.565 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/335) 2024-04-24T08:37:32.023 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:32.034 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:32.034 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:32.480 INFO:teuthology.orchestra.run.smithi123.stdout:generic/335 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:32.490 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/335 2024-04-24T08:37:32.490 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/335 2024-04-24T08:37:32.490 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:32.499 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:32.527 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:32.528 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/336) 2024-04-24T08:37:33.028 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:33.040 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:33.040 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:33.496 INFO:teuthology.orchestra.run.smithi123.stdout:generic/336 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:33.506 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/336 2024-04-24T08:37:33.506 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/336 2024-04-24T08:37:33.506 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:33.514 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:33.541 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:33.542 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/337) 2024-04-24T08:37:34.028 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:34.039 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:34.039 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:34.415 INFO:teuthology.orchestra.run.smithi123.stdout:generic/337 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:34.427 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/337 2024-04-24T08:37:34.427 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/337 2024-04-24T08:37:34.427 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:34.437 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:34.463 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:34.463 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/338) 2024-04-24T08:37:34.984 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:34.996 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:34.996 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:35.378 INFO:teuthology.orchestra.run.smithi123.stdout:generic/338 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:35.388 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/338 2024-04-24T08:37:35.388 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/338 2024-04-24T08:37:35.388 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:35.396 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:35.423 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:35.423 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/339) 2024-04-24T08:37:35.918 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:35.930 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:36.302 INFO:teuthology.orchestra.run.smithi123.stdout:generic/339 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:36.313 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/339 2024-04-24T08:37:36.313 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/339 2024-04-24T08:37:36.313 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:36.321 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:36.347 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:36.348 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/340) 2024-04-24T08:37:36.815 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:36.827 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:36.827 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:37.208 INFO:teuthology.orchestra.run.smithi123.stdout:generic/340 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:37.219 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/340 2024-04-24T08:37:37.219 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/340 2024-04-24T08:37:37.220 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:37.234 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:37.257 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:37.258 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/341) 2024-04-24T08:37:37.711 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:37.722 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:37.722 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:38.109 INFO:teuthology.orchestra.run.smithi123.stdout:generic/341 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:38.120 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/341 2024-04-24T08:37:38.120 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/341 2024-04-24T08:37:38.120 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:38.128 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:38.153 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:38.153 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/342) 2024-04-24T08:37:38.623 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:38.636 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:38.636 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:39.044 INFO:teuthology.orchestra.run.smithi123.stdout:generic/342 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:39.055 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/342 2024-04-24T08:37:39.056 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/342 2024-04-24T08:37:39.056 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:39.065 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:39.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:39.089 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/343) 2024-04-24T08:37:39.525 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:39.537 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:39.537 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:39.942 INFO:teuthology.orchestra.run.smithi123.stdout:generic/343 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:39.954 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/343 2024-04-24T08:37:39.955 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/343 2024-04-24T08:37:39.955 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:39.965 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:39.992 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:39.993 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/344) 2024-04-24T08:37:40.483 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:40.495 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:40.495 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:40.879 INFO:teuthology.orchestra.run.smithi123.stdout:generic/344 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:40.892 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/344 2024-04-24T08:37:40.892 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/344 2024-04-24T08:37:40.892 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:40.902 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:40.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:40.933 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/345) 2024-04-24T08:37:41.456 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:41.470 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:41.470 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:41.836 INFO:teuthology.orchestra.run.smithi123.stdout:generic/345 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:41.847 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/345 2024-04-24T08:37:41.847 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/345 2024-04-24T08:37:41.847 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:41.854 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:41.881 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:41.881 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/346) 2024-04-24T08:37:42.358 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:42.369 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:42.369 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:42.703 INFO:teuthology.orchestra.run.smithi123.stdout:generic/346 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:42.713 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/346 2024-04-24T08:37:42.713 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/346 2024-04-24T08:37:42.713 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:42.721 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:42.746 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:42.747 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/347) 2024-04-24T08:37:43.193 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:43.205 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:43.205 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:43.822 INFO:teuthology.orchestra.run.smithi123.stdout:generic/347 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:43.833 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/347 2024-04-24T08:37:43.833 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/347 2024-04-24T08:37:43.834 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:43.844 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:43.872 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:43.872 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/348) 2024-04-24T08:37:44.343 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:44.355 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:44.356 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:44.816 INFO:teuthology.orchestra.run.smithi123.stdout:generic/348 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:44.828 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/348 2024-04-24T08:37:44.829 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/348 2024-04-24T08:37:44.829 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:44.840 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:44.873 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:44.873 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/349) 2024-04-24T08:37:45.346 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:45.360 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:45.360 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:48.920 INFO:teuthology.orchestra.run.smithi123.stdout:generic/349 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-24T08:37:48.931 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/349 2024-04-24T08:37:48.931 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/349 2024-04-24T08:37:48.931 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:48.940 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:48.967 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:48.968 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/350) 2024-04-24T08:37:49.439 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:49.453 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:49.453 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:54.017 INFO:teuthology.orchestra.run.smithi123.stdout:generic/350 5s 2024-04-24T08:37:54.032 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/350 2024-04-24T08:37:54.032 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:54.064 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:54.064 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/351) 2024-04-24T08:37:54.541 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:54.555 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:57.659 INFO:teuthology.orchestra.run.smithi123.stdout:generic/351 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-24T08:37:57.670 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/351 2024-04-24T08:37:57.671 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/351 2024-04-24T08:37:57.671 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:57.679 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:57.729 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:57.730 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/352) 2024-04-24T08:37:58.444 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:58.456 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:58.857 INFO:teuthology.orchestra.run.smithi123.stdout:generic/352 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:58.868 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/352 2024-04-24T08:37:58.868 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/352 2024-04-24T08:37:58.868 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:58.877 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:58.904 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:58.904 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/353) 2024-04-24T08:37:59.388 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:37:59.401 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:37:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:59.794 INFO:teuthology.orchestra.run.smithi123.stdout:generic/353 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:37:59.803 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/353 2024-04-24T08:37:59.803 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/353 2024-04-24T08:37:59.803 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:37:59.811 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:37:59.838 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:37:59.839 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/354) 2024-04-24T08:38:00.315 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:00.327 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:00.327 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:00.703 INFO:teuthology.orchestra.run.smithi123.stdout:generic/354 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:00.714 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/354 2024-04-24T08:38:00.714 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/354 2024-04-24T08:38:00.714 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:00.724 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:00.749 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:00.750 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/355) 2024-04-24T08:38:01.200 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:01.213 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:01.214 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:03.453 INFO:teuthology.orchestra.run.smithi123.stdout:generic/355 2s 2024-04-24T08:38:03.469 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/355 2024-04-24T08:38:03.469 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:03.477 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:03.504 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:03.504 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/356) 2024-04-24T08:38:04.241 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:04.254 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:04.254 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:04.637 INFO:teuthology.orchestra.run.smithi123.stdout:generic/356 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:04.647 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/356 2024-04-24T08:38:04.647 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/356 2024-04-24T08:38:04.647 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:04.655 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:04.683 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:04.683 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/357) 2024-04-24T08:38:05.163 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:05.175 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:05.175 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:05.560 INFO:teuthology.orchestra.run.smithi123.stdout:generic/357 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:05.571 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/357 2024-04-24T08:38:05.571 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/357 2024-04-24T08:38:05.571 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:05.579 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:05.604 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:05.605 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/358) 2024-04-24T08:38:06.062 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:06.074 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:06.074 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:06.464 INFO:teuthology.orchestra.run.smithi123.stdout:generic/358 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:06.475 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/358 2024-04-24T08:38:06.475 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/358 2024-04-24T08:38:06.475 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:06.483 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:06.509 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:06.509 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/359) 2024-04-24T08:38:06.981 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:06.992 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:06.992 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:07.385 INFO:teuthology.orchestra.run.smithi123.stdout:generic/359 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:07.395 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/359 2024-04-24T08:38:07.396 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/359 2024-04-24T08:38:07.396 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:07.404 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:07.432 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:07.432 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/360) 2024-04-24T08:38:07.885 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:07.899 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:07.899 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:08.718 INFO:teuthology.orchestra.run.smithi123.stdout:generic/360 1s 2024-04-24T08:38:08.737 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/360 2024-04-24T08:38:08.737 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:08.747 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:08.776 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:08.777 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/361) 2024-04-24T08:38:09.280 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:09.292 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:09.292 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:09.853 INFO:teuthology.orchestra.run.smithi123.stdout:generic/361 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/361 2024-04-24T08:38:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/361 2024-04-24T08:38:09.867 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:09.877 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:09.907 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:09.907 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/362) 2024-04-24T08:38:10.390 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:10.401 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:10.401 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:10.753 INFO:teuthology.orchestra.run.smithi123.stdout:generic/362 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:10.762 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/362 2024-04-24T08:38:10.763 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/362 2024-04-24T08:38:10.763 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:10.772 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:10.799 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:10.799 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/363) 2024-04-24T08:38:11.307 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:11.321 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:11.321 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:11.713 INFO:teuthology.orchestra.run.smithi123.stdout:generic/363 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:11.724 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/363 2024-04-24T08:38:11.724 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/363 2024-04-24T08:38:11.725 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:11.733 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:11.760 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:11.760 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/364) 2024-04-24T08:38:12.279 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:12.291 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:12.291 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:12.659 INFO:teuthology.orchestra.run.smithi123.stdout:generic/364 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:12.670 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/364 2024-04-24T08:38:12.670 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/364 2024-04-24T08:38:12.670 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:12.680 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:12.706 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:12.707 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/365) 2024-04-24T08:38:13.159 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:13.169 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:13.170 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:13.520 INFO:teuthology.orchestra.run.smithi123.stdout:generic/365 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:13.529 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/365 2024-04-24T08:38:13.529 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/365 2024-04-24T08:38:13.529 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:13.536 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:13.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:13.562 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/366) 2024-04-24T08:38:14.007 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:14.017 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:14.017 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:14.366 INFO:teuthology.orchestra.run.smithi123.stdout:generic/366 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:14.377 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/366 2024-04-24T08:38:14.377 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/366 2024-04-24T08:38:14.377 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:14.387 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:14.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:14.413 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/367) 2024-04-24T08:38:14.912 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:14.923 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:14.923 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:15.274 INFO:teuthology.orchestra.run.smithi123.stdout:generic/367 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:15.284 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/367 2024-04-24T08:38:15.285 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/367 2024-04-24T08:38:15.285 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:15.292 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:15.317 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:15.317 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/368) 2024-04-24T08:38:15.818 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:15.832 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:15.832 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:16.213 INFO:teuthology.orchestra.run.smithi123.stdout:generic/368 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:16.225 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/368 2024-04-24T08:38:16.225 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/368 2024-04-24T08:38:16.225 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:16.234 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:16.265 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:16.265 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/369) 2024-04-24T08:38:16.734 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:16.746 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:17.113 INFO:teuthology.orchestra.run.smithi123.stdout:generic/369 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:17.127 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/369 2024-04-24T08:38:17.127 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/369 2024-04-24T08:38:17.127 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:17.168 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:17.168 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/370) 2024-04-24T08:38:17.628 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:17.641 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:17.641 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:18.014 INFO:teuthology.orchestra.run.smithi123.stdout:generic/370 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:18.026 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/370 2024-04-24T08:38:18.026 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/370 2024-04-24T08:38:18.026 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:18.064 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:18.064 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/371) 2024-04-24T08:38:18.540 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:18.553 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:18.553 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:18.922 INFO:teuthology.orchestra.run.smithi123.stdout:generic/371 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:18.933 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/371 2024-04-24T08:38:18.933 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/371 2024-04-24T08:38:18.933 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:18.940 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:18.968 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:18.968 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/372) 2024-04-24T08:38:19.446 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:19.458 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:19.458 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:19.866 INFO:teuthology.orchestra.run.smithi123.stdout:generic/372 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:19.877 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/372 2024-04-24T08:38:19.877 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/372 2024-04-24T08:38:19.877 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:19.886 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:19.914 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:19.914 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/373) 2024-04-24T08:38:20.400 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:20.414 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:20.414 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:20.823 INFO:teuthology.orchestra.run.smithi123.stdout:generic/373 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:20.834 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/373 2024-04-24T08:38:20.834 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/373 2024-04-24T08:38:20.835 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:20.843 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:20.871 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:20.872 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/374) 2024-04-24T08:38:21.335 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:21.347 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:21.347 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:21.741 INFO:teuthology.orchestra.run.smithi123.stdout:generic/374 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:21.752 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/374 2024-04-24T08:38:21.752 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/374 2024-04-24T08:38:21.752 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:21.760 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:21.787 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:21.787 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/375) 2024-04-24T08:38:22.248 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:22.259 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:22.259 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:22.785 INFO:teuthology.orchestra.run.smithi123.stdout:generic/375 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-24T08:38:22.796 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/375 2024-04-24T08:38:22.796 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/375 2024-04-24T08:38:22.796 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:22.804 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:22.829 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:22.830 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/376) 2024-04-24T08:38:23.296 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:23.307 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:23.307 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:23.858 INFO:teuthology.orchestra.run.smithi123.stdout:generic/376 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:23.871 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/376 2024-04-24T08:38:23.871 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/376 2024-04-24T08:38:23.871 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:23.881 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:23.910 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:23.911 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/377) 2024-04-24T08:38:24.387 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:24.398 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:24.398 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:24.750 INFO:teuthology.orchestra.run.smithi123.stdout:generic/377 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:24.760 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/377 2024-04-24T08:38:24.760 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/377 2024-04-24T08:38:24.760 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:24.794 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:24.794 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/378) 2024-04-24T08:38:25.243 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:25.253 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:25.253 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:26.400 INFO:teuthology.orchestra.run.smithi123.stdout:generic/378 1s 2024-04-24T08:38:26.415 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/378 2024-04-24T08:38:26.415 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:26.423 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:26.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:26.454 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/379) 2024-04-24T08:38:27.164 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:27.175 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:27.538 INFO:teuthology.orchestra.run.smithi123.stdout:generic/379 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:27.806 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/379 2024-04-24T08:38:27.806 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/379 2024-04-24T08:38:27.806 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:27.806 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:27.806 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:27.807 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/380) 2024-04-24T08:38:28.314 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:28.326 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:28.326 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:28.714 INFO:teuthology.orchestra.run.smithi123.stdout:generic/380 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:28.727 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/380 2024-04-24T08:38:28.728 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/380 2024-04-24T08:38:28.728 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:28.736 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:28.762 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:28.763 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/381) 2024-04-24T08:38:29.255 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:29.269 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:29.269 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:29.669 INFO:teuthology.orchestra.run.smithi123.stdout:generic/381 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:29.680 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/381 2024-04-24T08:38:29.680 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/381 2024-04-24T08:38:29.680 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:29.689 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:29.716 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:29.716 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/382) 2024-04-24T08:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:30.643 INFO:teuthology.orchestra.run.smithi123.stdout:generic/382 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:30.656 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/382 2024-04-24T08:38:30.657 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/382 2024-04-24T08:38:30.657 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:30.668 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:30.697 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:30.697 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/383) 2024-04-24T08:38:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:31.210 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:31.211 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:31.621 INFO:teuthology.orchestra.run.smithi123.stdout:generic/383 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:31.632 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/383 2024-04-24T08:38:31.632 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/383 2024-04-24T08:38:31.632 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:31.642 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:31.670 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:31.671 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/384) 2024-04-24T08:38:32.142 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:32.155 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:32.155 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:32.545 INFO:teuthology.orchestra.run.smithi123.stdout:generic/384 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-24T08:38:32.558 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/384 2024-04-24T08:38:32.558 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/384 2024-04-24T08:38:32.558 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:32.569 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:32.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:32.598 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/385) 2024-04-24T08:38:33.079 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:33.091 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:33.091 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:33.478 INFO:teuthology.orchestra.run.smithi123.stdout:generic/385 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:33.488 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/385 2024-04-24T08:38:33.489 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/385 2024-04-24T08:38:33.489 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:33.497 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:33.522 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:33.523 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/386) 2024-04-24T08:38:34.021 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:34.034 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:34.034 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:34.425 INFO:teuthology.orchestra.run.smithi123.stdout:generic/386 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-24T08:38:34.435 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/386 2024-04-24T08:38:34.435 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/386 2024-04-24T08:38:34.436 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:34.443 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:34.473 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:34.474 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/387) 2024-04-24T08:38:34.956 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:34.967 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:34.967 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:35.362 INFO:teuthology.orchestra.run.smithi123.stdout:generic/387 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:35.373 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/387 2024-04-24T08:38:35.373 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/387 2024-04-24T08:38:35.373 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:35.381 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:35.408 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:35.408 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/388) 2024-04-24T08:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:36.297 INFO:teuthology.orchestra.run.smithi123.stdout:generic/388 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:36.307 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/388 2024-04-24T08:38:36.307 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/388 2024-04-24T08:38:36.307 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:36.315 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:36.339 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:36.339 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/389) 2024-04-24T08:38:36.826 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:36.839 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:36.839 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:37.233 INFO:teuthology.orchestra.run.smithi123.stdout:generic/389 [not run] O_TMPFILE is not supported 2024-04-24T08:38:37.245 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/389 2024-04-24T08:38:37.245 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/389 2024-04-24T08:38:37.245 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:37.255 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:37.281 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:37.281 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/390) 2024-04-24T08:38:37.753 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:37.765 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:37.765 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:38.185 INFO:teuthology.orchestra.run.smithi123.stdout:generic/390 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:38:38.196 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/390 2024-04-24T08:38:38.196 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/390 2024-04-24T08:38:38.196 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:38:38.206 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:38:38.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:38:38.233 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/391) 2024-04-24T08:38:38.701 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:38:38.714 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:38:38.714 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:39:51.575 INFO:teuthology.orchestra.run.smithi123.stdout:generic/391 73s 2024-04-24T08:39:51.651 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/391 2024-04-24T08:39:51.651 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:39:51.663 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:39:51.715 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:39:51.716 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/392) 2024-04-24T08:39:53.093 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:39:53.105 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:39:53.105 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:39:53.524 INFO:teuthology.orchestra.run.smithi123.stdout:generic/392 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:39:53.535 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/392 2024-04-24T08:39:53.535 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/392 2024-04-24T08:39:53.535 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:39:53.543 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:39:53.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:39:53.573 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/393) 2024-04-24T08:39:54.141 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:39:54.154 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:39:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:39:54.543 INFO:teuthology.orchestra.run.smithi123.stdout:generic/393 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:39:54.556 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/393 2024-04-24T08:39:54.556 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/393 2024-04-24T08:39:54.556 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:39:54.566 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:39:54.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:39:54.597 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/394) 2024-04-24T08:39:55.075 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:39:55.086 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:39:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:39:55.676 INFO:teuthology.orchestra.run.smithi123.stdout:generic/394 0s 2024-04-24T08:39:55.693 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/394 2024-04-24T08:39:55.693 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:39:55.703 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:39:55.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:39:55.733 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/395) 2024-04-24T08:39:56.562 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:39:56.578 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:39:56.578 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:39:57.004 INFO:teuthology.orchestra.run.smithi123.stdout:generic/395 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:39:57.014 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/395 2024-04-24T08:39:57.014 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/395 2024-04-24T08:39:57.014 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:39:57.022 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:39:57.043 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:39:57.043 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/396) 2024-04-24T08:39:57.533 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:39:57.544 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:39:57.544 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:39:58.053 INFO:teuthology.orchestra.run.smithi123.stdout:generic/396 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:39:58.066 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/396 2024-04-24T08:39:58.067 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/396 2024-04-24T08:39:58.067 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:39:58.076 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:39:58.106 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:39:58.106 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/397) 2024-04-24T08:39:58.642 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:39:58.655 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:39:58.656 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:39:59.219 INFO:teuthology.orchestra.run.smithi123.stdout:generic/397 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:39:59.231 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/397 2024-04-24T08:39:59.231 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/397 2024-04-24T08:39:59.231 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:39:59.240 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:39:59.272 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:39:59.272 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/398) 2024-04-24T08:39:59.828 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:39:59.843 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:39:59.843 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:00.395 INFO:teuthology.orchestra.run.smithi123.stdout:generic/398 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:00.407 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/398 2024-04-24T08:40:00.407 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/398 2024-04-24T08:40:00.407 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:00.416 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:00.445 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:00.445 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/399) 2024-04-24T08:40:00.986 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:00.999 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:01.000 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:01.529 INFO:teuthology.orchestra.run.smithi123.stdout:generic/399 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:01.543 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/399 2024-04-24T08:40:01.543 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/399 2024-04-24T08:40:01.543 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:01.589 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:01.589 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/400) 2024-04-24T08:40:02.151 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:02.164 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:02.567 INFO:teuthology.orchestra.run.smithi123.stdout:generic/400 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-24T08:40:02.580 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/400 2024-04-24T08:40:02.581 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/400 2024-04-24T08:40:02.581 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:02.592 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:02.627 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:02.627 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/401) 2024-04-24T08:40:03.164 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:03.176 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:03.177 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:03.568 INFO:teuthology.orchestra.run.smithi123.stdout:generic/401 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:03.582 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/401 2024-04-24T08:40:03.582 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/401 2024-04-24T08:40:03.582 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:03.592 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:03.622 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:03.622 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/402) 2024-04-24T08:40:04.163 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:04.176 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:04.176 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:04.580 INFO:teuthology.orchestra.run.smithi123.stdout:generic/402 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:04.591 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/402 2024-04-24T08:40:04.591 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/402 2024-04-24T08:40:04.591 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:04.599 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:04.629 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:04.630 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/403) 2024-04-24T08:40:05.178 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:05.189 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:05.189 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:05.620 INFO:teuthology.orchestra.run.smithi123.stdout:generic/403 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:05.634 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/403 2024-04-24T08:40:05.634 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/403 2024-04-24T08:40:05.635 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:05.646 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:05.676 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:05.676 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/404) 2024-04-24T08:40:06.193 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:06.207 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:06.207 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:09.400 INFO:teuthology.orchestra.run.smithi123.stdout:generic/404 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-24T08:40:09.412 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/404 2024-04-24T08:40:09.412 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/404 2024-04-24T08:40:09.412 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:09.422 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:09.465 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:09.465 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/405) 2024-04-24T08:40:09.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:40:09.608 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:40:09.808 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T08:40:09.805+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 356305) UID: 0 2024-04-24T08:40:09.810 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T08:40:09.805+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 356305) UID: 0 2024-04-24T08:40:09.812 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T08:40:09.811+0000 7ff33a200640 -1 Fail to open '/proc/356305/cmdline' error = (2) No such file or directory 2024-04-24T08:40:09.812 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T08:40:09.812+0000 7ff33a200640 -1 received signal: Hangup from (PID: 356305) UID: 0 2024-04-24T08:40:09.816 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T08:40:09.807+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 356305) UID: 0 2024-04-24T08:40:10.188 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:10.199 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:10.199 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:10.766 INFO:teuthology.orchestra.run.smithi123.stdout:generic/405 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:10.778 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/405 2024-04-24T08:40:10.779 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/405 2024-04-24T08:40:10.779 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:10.790 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:10.818 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:10.819 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/406) 2024-04-24T08:40:11.281 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:11.293 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:11.293 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:11.656 INFO:teuthology.orchestra.run.smithi123.stdout:generic/406 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:11.665 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/406 2024-04-24T08:40:11.665 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/406 2024-04-24T08:40:11.665 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:11.672 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:11.708 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:11.708 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/407) 2024-04-24T08:40:12.192 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:12.204 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:12.204 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:12.706 INFO:teuthology.orchestra.run.smithi123.stdout:generic/407 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:40:12.716 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/407 2024-04-24T08:40:12.716 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/407 2024-04-24T08:40:12.717 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:12.725 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:12.751 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:12.751 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/408) 2024-04-24T08:40:13.478 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:13.491 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:13.491 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:14.012 INFO:teuthology.orchestra.run.smithi123.stdout:generic/408 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-24T08:40:14.022 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/408 2024-04-24T08:40:14.023 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/408 2024-04-24T08:40:14.023 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:14.031 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:14.059 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:14.059 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/409) 2024-04-24T08:40:14.770 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:14.781 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:14.781 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:15.147 INFO:teuthology.orchestra.run.smithi123.stdout:generic/409 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:15.159 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/409 2024-04-24T08:40:15.159 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/409 2024-04-24T08:40:15.160 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:15.172 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:15.203 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:15.203 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/410) 2024-04-24T08:40:15.676 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:15.688 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:15.688 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:16.084 INFO:teuthology.orchestra.run.smithi123.stdout:generic/410 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:16.094 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/410 2024-04-24T08:40:16.094 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/410 2024-04-24T08:40:16.094 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:16.127 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:16.127 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/411) 2024-04-24T08:40:16.627 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:16.640 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:16.640 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:17.014 INFO:teuthology.orchestra.run.smithi123.stdout:generic/411 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:17.023 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/411 2024-04-24T08:40:17.023 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/411 2024-04-24T08:40:17.023 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:17.054 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:17.054 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/412) 2024-04-24T08:40:17.530 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:17.542 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:17.542 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:17.897 INFO:teuthology.orchestra.run.smithi123.stdout:generic/412 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:17.907 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/412 2024-04-24T08:40:17.907 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/412 2024-04-24T08:40:17.907 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:17.916 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:17.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:17.942 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/413) 2024-04-24T08:40:18.413 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:18.427 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:18.427 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:18.774 INFO:teuthology.orchestra.run.smithi123.stdout:generic/413 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:18.784 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/413 2024-04-24T08:40:18.784 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/413 2024-04-24T08:40:18.785 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:18.793 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:18.817 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:18.817 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/414) 2024-04-24T08:40:19.288 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:19.301 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:19.301 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:19.673 INFO:teuthology.orchestra.run.smithi123.stdout:generic/414 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:19.684 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/414 2024-04-24T08:40:19.684 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/414 2024-04-24T08:40:19.684 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:19.692 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:19.716 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:19.716 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/415) 2024-04-24T08:40:20.214 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:20.227 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:20.227 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:20.591 INFO:teuthology.orchestra.run.smithi123.stdout:generic/415 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:20.600 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/415 2024-04-24T08:40:20.601 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/415 2024-04-24T08:40:20.601 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:20.608 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:20.632 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:20.632 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/416) 2024-04-24T08:40:21.121 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:21.132 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:21.525 INFO:teuthology.orchestra.run.smithi123.stdout:generic/416 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:21.535 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/416 2024-04-24T08:40:21.535 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/416 2024-04-24T08:40:21.535 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:21.544 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:21.572 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:21.572 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/417) 2024-04-24T08:40:22.079 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:22.091 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:22.091 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:22.468 INFO:teuthology.orchestra.run.smithi123.stdout:generic/417 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:22.477 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/417 2024-04-24T08:40:22.478 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/417 2024-04-24T08:40:22.478 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:22.485 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:22.521 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:22.521 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/418) 2024-04-24T08:40:22.995 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:23.007 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:23.007 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:23.549 INFO:teuthology.orchestra.run.smithi123.stdout:generic/418 [not run] require ceph-fuse to be valid block disk 2024-04-24T08:40:23.562 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/418 2024-04-24T08:40:23.563 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/418 2024-04-24T08:40:23.563 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:23.573 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:23.602 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:23.603 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/419) 2024-04-24T08:40:24.089 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:24.103 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:24.103 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:24.570 INFO:teuthology.orchestra.run.smithi123.stdout:generic/419 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:24.582 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/419 2024-04-24T08:40:24.582 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/419 2024-04-24T08:40:24.582 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:24.592 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:24.616 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:24.617 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/420) 2024-04-24T08:40:25.093 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:25.106 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:25.106 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:28.464 INFO:teuthology.orchestra.run.smithi123.stdout:generic/420 3s 2024-04-24T08:40:28.479 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/420 2024-04-24T08:40:28.479 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:28.487 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:28.514 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:28.515 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/421) 2024-04-24T08:40:29.251 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:29.263 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:29.263 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:29.701 INFO:teuthology.orchestra.run.smithi123.stdout:generic/421 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:29.713 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/421 2024-04-24T08:40:29.713 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/421 2024-04-24T08:40:29.713 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:29.721 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:29.748 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:29.749 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/422) 2024-04-24T08:40:30.236 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:30.250 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:30.635 INFO:teuthology.orchestra.run.smithi123.stdout:generic/422 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:30.648 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/422 2024-04-24T08:40:30.648 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/422 2024-04-24T08:40:30.648 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:30.657 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:30.686 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:30.687 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/423) 2024-04-24T08:40:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:31.180 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:31.990 INFO:teuthology.orchestra.run.smithi123.stdout:generic/423 0s 2024-04-24T08:40:32.005 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/423 2024-04-24T08:40:32.006 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:32.013 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:32.039 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:32.039 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/424) 2024-04-24T08:40:32.775 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:32.786 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:32.787 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:33.270 INFO:teuthology.orchestra.run.smithi123.stdout:generic/424 [not run] file system doesn't support any of /bin/chattr +a/+c/+d/+i 2024-04-24T08:40:33.283 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/424 2024-04-24T08:40:33.284 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/424 2024-04-24T08:40:33.284 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:33.296 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:33.325 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:33.326 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/425) 2024-04-24T08:40:33.788 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:33.800 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:33.800 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:34.198 INFO:teuthology.orchestra.run.smithi123.stdout:generic/425 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:40:34.208 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/425 2024-04-24T08:40:34.208 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/425 2024-04-24T08:40:34.208 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:40:34.216 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:40:34.241 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:40:34.242 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/426) 2024-04-24T08:40:34.724 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:40:34.737 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:40:34.737 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:41:57.116 INFO:teuthology.orchestra.run.smithi123.stdout:generic/426 82s 2024-04-24T08:41:57.293 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/426 2024-04-24T08:41:57.293 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:41:57.318 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:41:57.539 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:41:57.540 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/427) 2024-04-24T08:42:00.070 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:42:00.083 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:42:00.083 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:00.538 INFO:teuthology.orchestra.run.smithi123.stdout:generic/427 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:42:00.548 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/427 2024-04-24T08:42:00.548 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/427 2024-04-24T08:42:00.548 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:42:00.555 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:00.580 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:42:00.581 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/428) 2024-04-24T08:42:01.045 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:42:01.057 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:42:01.058 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:01.621 INFO:teuthology.orchestra.run.smithi123.stdout:generic/428 0s 2024-04-24T08:42:01.635 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/428 2024-04-24T08:42:01.636 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:42:01.642 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:01.668 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:42:01.668 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/429) 2024-04-24T08:42:02.132 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:42:02.146 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:42:02.146 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:02.602 INFO:teuthology.orchestra.run.smithi123.stdout:generic/429 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:42:02.614 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/429 2024-04-24T08:42:02.614 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/429 2024-04-24T08:42:02.615 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:42:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:02.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:42:02.654 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/430) 2024-04-24T08:42:03.125 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:42:03.136 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:42:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:03.855 INFO:teuthology.orchestra.run.smithi123.stdout:generic/430 0s 2024-04-24T08:42:03.870 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/430 2024-04-24T08:42:03.870 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:42:03.878 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:03.906 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:42:03.906 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/431) 2024-04-24T08:42:04.622 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:42:04.633 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:42:04.633 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:05.273 INFO:teuthology.orchestra.run.smithi123.stdout:generic/431 1s 2024-04-24T08:42:05.288 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/431 2024-04-24T08:42:05.288 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:42:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:05.326 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:42:05.326 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/432) 2024-04-24T08:42:06.060 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:42:06.071 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:42:06.071 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:06.713 INFO:teuthology.orchestra.run.smithi123.stdout:generic/432 0s 2024-04-24T08:42:06.726 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/432 2024-04-24T08:42:06.726 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:42:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:06.794 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:42:06.794 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/433) 2024-04-24T08:42:07.492 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:42:07.505 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:42:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:08.070 INFO:teuthology.orchestra.run.smithi123.stdout:generic/433 1s 2024-04-24T08:42:08.083 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/433 2024-04-24T08:42:08.084 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:42:08.090 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:08.115 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:42:08.115 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/434) 2024-04-24T08:42:08.835 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:42:08.847 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:42:08.847 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:09.427 INFO:teuthology.orchestra.run.smithi123.stdout:generic/434 - output mismatch (see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/434.out.bad) 2024-04-24T08:42:09.432 INFO:teuthology.orchestra.run.smithi123.stdout: --- tests/generic/434.out 2024-04-24 05:47:53.291731626 +0000 2024-04-24T08:42:09.433 INFO:teuthology.orchestra.run.smithi123.stdout: +++ /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/434.out.bad 2024-04-24 08:42:09.423528655 +0000 2024-04-24T08:42:09.433 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -9,6 +9,6 @@ 2024-04-24T08:42:09.433 INFO:teuthology.orchestra.run.smithi123.stdout: copy_range: Bad file descriptor 2024-04-24T08:42:09.433 INFO:teuthology.orchestra.run.smithi123.stdout: d41d8cd98f00b204e9800998ecf8427e TEST_DIR/test-434/copy 2024-04-24T08:42:09.433 INFO:teuthology.orchestra.run.smithi123.stdout: Try to copy to a device 2024-04-24T08:42:09.433 INFO:teuthology.orchestra.run.smithi123.stdout: -copy_range: Invalid argument 2024-04-24T08:42:09.433 INFO:teuthology.orchestra.run.smithi123.stdout: +/tmp/tmp.CHueoLyoRxtest/test-434/dev1: Permission denied 2024-04-24T08:42:09.433 INFO:teuthology.orchestra.run.smithi123.stdout: Try to copy to a fifo 2024-04-24T08:42:09.433 INFO:teuthology.orchestra.run.smithi123.stdout: copy_range: Invalid argument 2024-04-24T08:42:09.440 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/434 2024-04-24T08:42:09.440 INFO:teuthology.orchestra.run.smithi123.stdout:Failures: generic/434 2024-04-24T08:42:09.440 INFO:teuthology.orchestra.run.smithi123.stdout:Failed 1 of 1 tests 2024-04-24T08:42:09.449 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:09.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:42:09.479 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-24T08:42:09.479 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-24T08:42:09.479 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/435) 2024-04-24T08:42:10.202 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:42:10.214 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:42:10.214 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:10.661 INFO:teuthology.orchestra.run.smithi123.stdout:generic/435 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:42:10.671 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/435 2024-04-24T08:42:10.671 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/435 2024-04-24T08:42:10.671 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:42:10.678 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:10.704 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:42:10.704 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/436) 2024-04-24T08:42:11.195 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:42:11.206 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:42:11.207 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:13.363 INFO:teuthology.orchestra.run.smithi123.stdout:generic/436 2s 2024-04-24T08:42:13.377 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/436 2024-04-24T08:42:13.377 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:42:13.386 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:13.428 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:42:13.428 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/437) 2024-04-24T08:42:14.131 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:42:14.142 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:42:14.142 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:15.364 INFO:teuthology.orchestra.run.smithi123.stdout:generic/437 1s 2024-04-24T08:42:15.381 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/437 2024-04-24T08:42:15.381 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:42:15.389 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:42:15.416 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:42:15.417 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/438) 2024-04-24T08:42:15.901 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:42:15.913 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:42:15.913 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:32.455 INFO:teuthology.orchestra.run.smithi123.stdout:generic/438 197s 2024-04-24T08:45:32.470 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/438 2024-04-24T08:45:32.470 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:32.477 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:32.505 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:32.505 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/439) 2024-04-24T08:45:32.988 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:33.001 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:33.401 INFO:teuthology.orchestra.run.smithi123.stdout:generic/439 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:45:33.414 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/439 2024-04-24T08:45:33.415 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/439 2024-04-24T08:45:33.415 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:33.423 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:33.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:33.452 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/440) 2024-04-24T08:45:33.931 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:33.943 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:33.944 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:34.423 INFO:teuthology.orchestra.run.smithi123.stdout:generic/440 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:45:34.433 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/440 2024-04-24T08:45:34.434 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/440 2024-04-24T08:45:34.434 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:34.441 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:34.477 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:34.477 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/441) 2024-04-24T08:45:34.949 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:34.960 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:34.961 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:35.499 INFO:teuthology.orchestra.run.smithi123.stdout:generic/441 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:45:35.510 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/441 2024-04-24T08:45:35.510 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/441 2024-04-24T08:45:35.510 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:35.518 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:35.546 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:35.546 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/442) 2024-04-24T08:45:36.052 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:36.066 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:36.067 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:36.488 INFO:teuthology.orchestra.run.smithi123.stdout:generic/442 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:45:36.498 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/442 2024-04-24T08:45:36.498 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/442 2024-04-24T08:45:36.498 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:36.506 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:36.531 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:36.531 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/443) 2024-04-24T08:45:37.038 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:37.049 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:37.049 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:37.493 INFO:teuthology.orchestra.run.smithi123.stdout:generic/443 0s 2024-04-24T08:45:37.509 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/443 2024-04-24T08:45:37.510 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:37.518 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:37.544 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:37.544 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/444) 2024-04-24T08:45:38.053 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:38.066 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:38.066 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:38.506 INFO:teuthology.orchestra.run.smithi123.stdout:generic/444 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-24T08:45:38.518 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/444 2024-04-24T08:45:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/444 2024-04-24T08:45:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:38.529 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:38.557 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:38.558 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/445) 2024-04-24T08:45:39.033 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:39.044 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:39.044 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:41.468 INFO:teuthology.orchestra.run.smithi123.stdout:generic/445 2s 2024-04-24T08:45:41.483 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/445 2024-04-24T08:45:41.483 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:41.491 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:41.520 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:41.520 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/446) 2024-04-24T08:45:42.242 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:42.250 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:42.251 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:42.568 INFO:teuthology.orchestra.run.smithi123.stdout:generic/446 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:45:42.578 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/446 2024-04-24T08:45:42.578 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/446 2024-04-24T08:45:42.578 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:42.585 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:42.609 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:42.610 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/447) 2024-04-24T08:45:43.100 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:43.112 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:43.112 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:43.520 INFO:teuthology.orchestra.run.smithi123.stdout:generic/447 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:45:43.531 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/447 2024-04-24T08:45:43.531 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/447 2024-04-24T08:45:43.531 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:43.567 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:43.567 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/448) 2024-04-24T08:45:44.053 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:44.065 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:46.484 INFO:teuthology.orchestra.run.smithi123.stdout:generic/448 2s 2024-04-24T08:45:46.499 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/448 2024-04-24T08:45:46.499 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:46.507 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:46.539 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:46.539 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/449) 2024-04-24T08:45:48.061 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:48.075 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:48.867 INFO:teuthology.orchestra.run.smithi123.stdout:generic/449 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:45:48.879 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/449 2024-04-24T08:45:48.879 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/449 2024-04-24T08:45:48.879 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:48.887 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:48.916 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:48.916 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/450) 2024-04-24T08:45:49.394 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:49.411 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:49.411 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:52.676 INFO:teuthology.orchestra.run.smithi123.stdout:generic/450 3s 2024-04-24T08:45:52.691 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/450 2024-04-24T08:45:52.692 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:52.699 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:52.726 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:52.727 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/451) 2024-04-24T08:45:53.461 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:53.472 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:53.472 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:54.319 INFO:teuthology.orchestra.run.smithi123.stdout:generic/451 - output mismatch (see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/451.out.bad) 2024-04-24T08:45:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: --- tests/generic/451.out 2024-04-24 05:47:53.293731579 +0000 2024-04-24T08:45:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: +++ /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/451.out.bad 2024-04-24 08:45:54.315330307 +0000 2024-04-24T08:45:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -1,2 +1,5 @@ 2024-04-24T08:45:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: QA output created by 451 2024-04-24T08:45:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: +get stale data from buffer read 2024-04-24T08:45:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: +00000000 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 UUUUUUUUUUUUUUUU 2024-04-24T08:45:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: +000a0000 2024-04-24T08:45:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: Silence is golden 2024-04-24T08:45:54.331 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/451 2024-04-24T08:45:54.331 INFO:teuthology.orchestra.run.smithi123.stdout:Failures: generic/451 2024-04-24T08:45:54.331 INFO:teuthology.orchestra.run.smithi123.stdout:Failed 1 of 1 tests 2024-04-24T08:45:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:54.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:45:54.362 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-24T08:45:54.362 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-24T08:45:54.362 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/452) 2024-04-24T08:45:55.119 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:55.139 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:55.439 INFO:teuthology.orchestra.run.smithi123.stdout:generic/452 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:45:55.448 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/452 2024-04-24T08:45:55.448 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/452 2024-04-24T08:45:55.448 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:55.455 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:55.477 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:55.477 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/453) 2024-04-24T08:45:55.963 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:55.974 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:55.974 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:56.338 INFO:teuthology.orchestra.run.smithi123.stdout:generic/453 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:45:56.349 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/453 2024-04-24T08:45:56.349 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/453 2024-04-24T08:45:56.349 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:56.358 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:56.383 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:56.384 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/454) 2024-04-24T08:45:56.872 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:56.883 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:56.883 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:57.223 INFO:teuthology.orchestra.run.smithi123.stdout:generic/454 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:45:57.234 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/454 2024-04-24T08:45:57.234 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/454 2024-04-24T08:45:57.234 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:57.242 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:57.269 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:57.269 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/455) 2024-04-24T08:45:57.741 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:57.753 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:57.753 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:58.263 INFO:teuthology.orchestra.run.smithi123.stdout:generic/455 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:45:58.275 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/455 2024-04-24T08:45:58.275 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/455 2024-04-24T08:45:58.275 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:58.283 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:58.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:58.309 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/456) 2024-04-24T08:45:58.801 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:58.814 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:58.815 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:59.209 INFO:teuthology.orchestra.run.smithi123.stdout:generic/456 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:45:59.223 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/456 2024-04-24T08:45:59.223 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/456 2024-04-24T08:45:59.223 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:45:59.228 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:45:59.255 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:45:59.255 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/457) 2024-04-24T08:45:59.747 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:45:59.759 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:45:59.759 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:00.266 INFO:teuthology.orchestra.run.smithi123.stdout:generic/457 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:46:00.277 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/457 2024-04-24T08:46:00.278 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/457 2024-04-24T08:46:00.278 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:46:00.286 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:00.314 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:46:00.314 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/458) 2024-04-24T08:46:00.781 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:46:00.795 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:46:00.795 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:01.175 INFO:teuthology.orchestra.run.smithi123.stdout:generic/458 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:46:01.186 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/458 2024-04-24T08:46:01.186 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/458 2024-04-24T08:46:01.186 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:46:01.196 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:01.225 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:46:01.225 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/459) 2024-04-24T08:46:01.702 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:46:01.714 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:46:01.714 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:02.502 INFO:teuthology.orchestra.run.smithi123.stdout:generic/459 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:46:02.515 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/459 2024-04-24T08:46:02.515 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/459 2024-04-24T08:46:02.515 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:46:02.526 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:02.565 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:46:02.565 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/460) 2024-04-24T08:46:03.046 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:46:03.057 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:46:03.057 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:03.464 INFO:teuthology.orchestra.run.smithi123.stdout:generic/460 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:46:03.475 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/460 2024-04-24T08:46:03.476 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/460 2024-04-24T08:46:03.476 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:46:03.484 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:03.509 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:46:03.509 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/461) 2024-04-24T08:46:04.000 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:46:04.012 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:46:04.012 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:04.421 INFO:teuthology.orchestra.run.smithi123.stdout:generic/461 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:46:04.433 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/461 2024-04-24T08:46:04.433 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/461 2024-04-24T08:46:04.433 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:46:04.444 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:04.476 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:46:04.476 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/462) 2024-04-24T08:46:04.953 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:46:04.965 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:46:04.965 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:05.425 INFO:teuthology.orchestra.run.smithi123.stdout:generic/462 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:46:05.434 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/462 2024-04-24T08:46:05.435 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/462 2024-04-24T08:46:05.435 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:46:05.442 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:05.468 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:46:05.468 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/463) 2024-04-24T08:46:05.989 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:46:05.999 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:46:06.000 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:06.494 INFO:teuthology.orchestra.run.smithi123.stdout:generic/463 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T08:46:06.505 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/463 2024-04-24T08:46:06.505 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/463 2024-04-24T08:46:06.505 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:46:06.513 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:06.540 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:46:06.540 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/464) 2024-04-24T08:46:07.226 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:46:07.237 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:46:07.237 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:07.590 INFO:teuthology.orchestra.run.smithi123.stdout:generic/464 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:46:07.599 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/464 2024-04-24T08:46:07.599 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/464 2024-04-24T08:46:07.599 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:46:07.606 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:07.675 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:46:07.676 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/465) 2024-04-24T08:46:08.124 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:46:08.134 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:46:08.135 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:38.441 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:46:38.449 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:46:38.680 INFO:teuthology.orchestra.run.smithi123.stdout:generic/465 30s 2024-04-24T08:46:38.715 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/465 2024-04-24T08:46:38.716 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:46:38.730 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:38.782 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:46:38.783 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/466) 2024-04-24T08:46:39.315 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:46:39.327 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:46:39.327 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:generic/466 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:46:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/466 2024-04-24T08:46:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/466 2024-04-24T08:46:39.652 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:46:39.662 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:39.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:46:39.693 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/467) 2024-04-24T08:46:40.225 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:46:40.239 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:46:40.239 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:49.666 INFO:teuthology.orchestra.run.smithi123.stdout:generic/467 9s 2024-04-24T08:46:49.738 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/467 2024-04-24T08:46:49.738 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:46:49.750 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:49.898 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:46:49.899 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/468) 2024-04-24T08:46:51.427 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:46:51.442 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:46:51.442 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:51.892 INFO:teuthology.orchestra.run.smithi123.stdout:generic/468 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:46:51.901 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/468 2024-04-24T08:46:51.901 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/468 2024-04-24T08:46:51.901 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:46:51.909 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:46:51.935 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:46:51.935 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/469) 2024-04-24T08:46:52.500 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:46:52.514 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:46:52.514 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:47:01.414 INFO:teuthology.orchestra.run.smithi123.stdout:generic/469 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-24T08:47:01.428 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/469 2024-04-24T08:47:01.428 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/469 2024-04-24T08:47:01.428 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:47:01.440 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:47:01.492 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:47:01.492 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/470) 2024-04-24T08:47:02.202 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:47:02.213 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:47:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:47:02.691 INFO:teuthology.orchestra.run.smithi123.stdout:generic/470 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:47:02.707 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/470 2024-04-24T08:47:02.707 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/470 2024-04-24T08:47:02.708 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:47:02.719 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:47:02.752 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:47:02.752 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/471) 2024-04-24T08:47:03.290 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:47:03.304 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:47:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:47:08.691 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:47:08.700 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:47:38.856 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:47:38.865 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:48:09.043 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:48:09.052 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:48:39.256 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:48:39.261 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:49:09.419 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:49:09.429 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:49:20.265 INFO:teuthology.orchestra.run.smithi123.stdout:generic/471 137s 2024-04-24T08:49:20.286 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/471 2024-04-24T08:49:20.286 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:49:20.298 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:20.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:49:20.331 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/472) 2024-04-24T08:49:21.297 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:49:21.312 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:49:21.312 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:21.649 INFO:teuthology.orchestra.run.smithi123.stdout:generic/472 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:49:21.658 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/472 2024-04-24T08:49:21.658 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/472 2024-04-24T08:49:21.658 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:49:21.666 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:21.689 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:49:21.689 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/473) 2024-04-24T08:49:22.197 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:49:22.209 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:49:22.209 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:22.730 INFO:teuthology.orchestra.run.smithi123.stdout:generic/473 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-24T08:49:22.745 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/473 2024-04-24T08:49:22.745 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/473 2024-04-24T08:49:22.745 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:49:22.756 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:22.792 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:49:22.792 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/474) 2024-04-24T08:49:23.332 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:49:23.344 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:49:23.345 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:23.790 INFO:teuthology.orchestra.run.smithi123.stdout:generic/474 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:49:23.802 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/474 2024-04-24T08:49:23.802 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/474 2024-04-24T08:49:23.802 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:49:23.812 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:23.845 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:49:23.846 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/475) 2024-04-24T08:49:24.379 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:49:24.391 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:49:24.391 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:24.899 INFO:teuthology.orchestra.run.smithi123.stdout:generic/475 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:49:24.912 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/475 2024-04-24T08:49:24.912 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/475 2024-04-24T08:49:24.912 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:49:24.921 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:24.948 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:49:24.949 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/476) 2024-04-24T08:49:25.469 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:49:25.483 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:49:25.483 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:25.929 INFO:teuthology.orchestra.run.smithi123.stdout:generic/476 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:49:25.941 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/476 2024-04-24T08:49:25.941 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/476 2024-04-24T08:49:25.941 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:49:25.949 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:25.975 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:49:25.976 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/477) 2024-04-24T08:49:26.517 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:49:26.531 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:49:26.531 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:38.905 INFO:teuthology.orchestra.run.smithi123.stdout:generic/477 12s 2024-04-24T08:49:38.993 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/477 2024-04-24T08:49:38.993 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:49:39.003 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:39.096 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:49:39.096 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/478) 2024-04-24T08:49:39.578 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:49:39.585 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:49:40.608 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:49:40.622 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:49:40.622 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:52.727 INFO:teuthology.orchestra.run.smithi123.stdout:generic/478 - output mismatch (see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/478.out.bad) 2024-04-24T08:49:52.739 INFO:teuthology.orchestra.run.smithi123.stdout: --- tests/generic/478.out 2024-04-24 05:47:53.298731461 +0000 2024-04-24T08:49:52.739 INFO:teuthology.orchestra.run.smithi123.stdout: +++ /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/478.out.bad 2024-04-24 08:49:52.694801874 +0000 2024-04-24T08:49:52.739 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -10,7 +10,7 @@ 2024-04-24T08:49:52.739 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.739 INFO:teuthology.orchestra.run.smithi123.stdout: get wrlck 2024-04-24T08:49:52.739 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.739 INFO:teuthology.orchestra.run.smithi123.stdout: -lock could be placed 2024-04-24T08:49:52.739 INFO:teuthology.orchestra.run.smithi123.stdout: +get wrlck 2024-04-24T08:49:52.739 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.739 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.739 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.739 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -25,7 +25,7 @@ 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: get wrlck 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: -lock could be placed 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: +get wrlck 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -37,7 +37,7 @@ 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: get rdlck 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: get rdlck 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: -lock could be placed 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: +get rdlck 2024-04-24T08:49:52.740 INFO:teuthology.orchestra.run.smithi123.stdout: get rdlck 2024-04-24T08:49:52.741 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.741 INFO:teuthology.orchestra.run.smithi123.stdout: get rdlck 2024-04-24T08:49:52.741 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -46,7 +46,7 @@ 2024-04-24T08:49:52.741 INFO:teuthology.orchestra.run.smithi123.stdout: get rdlck 2024-04-24T08:49:52.741 INFO:teuthology.orchestra.run.smithi123.stdout: get rdlck 2024-04-24T08:49:52.741 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.741 INFO:teuthology.orchestra.run.smithi123.stdout: -lock could be placed 2024-04-24T08:49:52.741 INFO:teuthology.orchestra.run.smithi123.stdout: +get rdlck 2024-04-24T08:49:52.741 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.741 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.741 INFO:teuthology.orchestra.run.smithi123.stdout: lock could be placed 2024-04-24T08:49:52.749 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/478 2024-04-24T08:49:52.750 INFO:teuthology.orchestra.run.smithi123.stdout:Failures: generic/478 2024-04-24T08:49:52.750 INFO:teuthology.orchestra.run.smithi123.stdout:Failed 1 of 1 tests 2024-04-24T08:49:52.761 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:52.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:49:52.796 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-24T08:49:52.796 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-24T08:49:52.797 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/479) 2024-04-24T08:49:53.350 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:49:53.363 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:49:53.363 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:53.878 INFO:teuthology.orchestra.run.smithi123.stdout:generic/479 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:49:53.890 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/479 2024-04-24T08:49:53.890 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/479 2024-04-24T08:49:53.890 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:49:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:53.930 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:49:53.930 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/480) 2024-04-24T08:49:54.456 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:49:54.468 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:49:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:54.918 INFO:teuthology.orchestra.run.smithi123.stdout:generic/480 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:49:54.929 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/480 2024-04-24T08:49:54.930 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/480 2024-04-24T08:49:54.930 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:49:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:54.965 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:49:54.966 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/481) 2024-04-24T08:49:55.507 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:49:55.521 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:49:55.521 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:55.990 INFO:teuthology.orchestra.run.smithi123.stdout:generic/481 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:49:56.005 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/481 2024-04-24T08:49:56.005 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/481 2024-04-24T08:49:56.005 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:49:56.017 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:56.049 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:49:56.049 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/482) 2024-04-24T08:49:56.593 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:49:56.606 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:49:56.606 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:57.208 INFO:teuthology.orchestra.run.smithi123.stdout:generic/482 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:49:57.222 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/482 2024-04-24T08:49:57.222 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/482 2024-04-24T08:49:57.223 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:49:57.234 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:57.265 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:49:57.266 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/483) 2024-04-24T08:49:57.794 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:49:57.808 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:49:57.808 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:58.270 INFO:teuthology.orchestra.run.smithi123.stdout:generic/483 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:49:58.283 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/483 2024-04-24T08:49:58.283 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/483 2024-04-24T08:49:58.284 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:49:58.296 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:58.326 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:49:58.326 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/484) 2024-04-24T08:49:58.853 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:49:58.865 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:49:58.865 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:59.345 INFO:teuthology.orchestra.run.smithi123.stdout:generic/484 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:49:59.358 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/484 2024-04-24T08:49:59.359 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/484 2024-04-24T08:49:59.359 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:49:59.370 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:49:59.401 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:49:59.402 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/485) 2024-04-24T08:49:59.967 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:49:59.982 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:49:59.982 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:02.671 INFO:teuthology.orchestra.run.smithi123.stdout:generic/485 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-24T08:50:02.683 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/485 2024-04-24T08:50:02.683 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/485 2024-04-24T08:50:02.683 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:02.735 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:02.735 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/486) 2024-04-24T08:50:03.420 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:03.431 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:03.431 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:03.837 INFO:teuthology.orchestra.run.smithi123.stdout:generic/486 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:03.847 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/486 2024-04-24T08:50:03.847 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/486 2024-04-24T08:50:03.847 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:03.858 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:03.891 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:03.892 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/487) 2024-04-24T08:50:04.434 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:04.448 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:04.448 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:04.941 INFO:teuthology.orchestra.run.smithi123.stdout:generic/487 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:04.954 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/487 2024-04-24T08:50:04.954 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/487 2024-04-24T08:50:04.955 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:04.966 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:04.997 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:04.997 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/488) 2024-04-24T08:50:05.539 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:05.551 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:05.551 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:05.989 INFO:teuthology.orchestra.run.smithi123.stdout:generic/488 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:06.001 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/488 2024-04-24T08:50:06.001 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/488 2024-04-24T08:50:06.001 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:06.011 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:06.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:06.046 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/489) 2024-04-24T08:50:06.612 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:06.625 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:07.035 INFO:teuthology.orchestra.run.smithi123.stdout:generic/489 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:07.049 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/489 2024-04-24T08:50:07.049 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/489 2024-04-24T08:50:07.049 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:07.060 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:07.091 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:07.092 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/490) 2024-04-24T08:50:07.617 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:07.629 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:07.629 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:09.826 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:50:09.835 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:50:10.196 INFO:teuthology.orchestra.run.smithi123.stdout:generic/490 3s 2024-04-24T08:50:10.214 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/490 2024-04-24T08:50:10.214 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:10.222 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:10.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:10.252 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/491) 2024-04-24T08:50:11.008 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:11.021 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:11.021 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:11.468 INFO:teuthology.orchestra.run.smithi123.stdout:generic/491 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:11.478 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/491 2024-04-24T08:50:11.478 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/491 2024-04-24T08:50:11.478 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:11.486 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:11.513 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:11.513 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/492) 2024-04-24T08:50:12.042 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:12.053 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:12.053 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:generic/492 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/492 2024-04-24T08:50:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/492 2024-04-24T08:50:12.460 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:12.469 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:12.498 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:12.498 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/493) 2024-04-24T08:50:12.979 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:12.991 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:12.991 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:13.349 INFO:teuthology.orchestra.run.smithi123.stdout:generic/493 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:13.361 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/493 2024-04-24T08:50:13.361 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/493 2024-04-24T08:50:13.361 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:13.372 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:13.398 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:13.398 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/494) 2024-04-24T08:50:13.873 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:13.885 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:13.886 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:14.252 INFO:teuthology.orchestra.run.smithi123.stdout:generic/494 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:14.261 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/494 2024-04-24T08:50:14.261 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/494 2024-04-24T08:50:14.261 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:14.270 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:14.298 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:14.299 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/495) 2024-04-24T08:50:14.807 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:14.818 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:14.818 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:15.254 INFO:teuthology.orchestra.run.smithi123.stdout:generic/495 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:15.265 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/495 2024-04-24T08:50:15.265 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/495 2024-04-24T08:50:15.265 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:15.274 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:15.304 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:15.304 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/496) 2024-04-24T08:50:15.835 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:15.850 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:16.323 INFO:teuthology.orchestra.run.smithi123.stdout:generic/496 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:16.337 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/496 2024-04-24T08:50:16.337 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/496 2024-04-24T08:50:16.337 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:16.349 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:16.381 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:16.381 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/497) 2024-04-24T08:50:16.924 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:16.938 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:16.938 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:17.344 INFO:teuthology.orchestra.run.smithi123.stdout:generic/497 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:17.356 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/497 2024-04-24T08:50:17.356 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/497 2024-04-24T08:50:17.356 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:17.367 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:17.403 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:17.403 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/498) 2024-04-24T08:50:17.966 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:17.979 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:17.979 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:18.440 INFO:teuthology.orchestra.run.smithi123.stdout:generic/498 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:18.450 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/498 2024-04-24T08:50:18.450 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/498 2024-04-24T08:50:18.450 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:18.459 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:18.499 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:18.499 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/499) 2024-04-24T08:50:19.021 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:19.035 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:19.035 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:19.453 INFO:teuthology.orchestra.run.smithi123.stdout:generic/499 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:19.464 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/499 2024-04-24T08:50:19.464 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/499 2024-04-24T08:50:19.464 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:19.473 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:19.503 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:19.504 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/500) 2024-04-24T08:50:20.033 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:20.046 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:20.046 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:20.591 INFO:teuthology.orchestra.run.smithi123.stdout:generic/500 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:20.602 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/500 2024-04-24T08:50:20.602 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/500 2024-04-24T08:50:20.602 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:20.613 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:20.643 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:20.643 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/501) 2024-04-24T08:50:21.207 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:21.221 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:21.221 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:21.684 INFO:teuthology.orchestra.run.smithi123.stdout:generic/501 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:21.696 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/501 2024-04-24T08:50:21.696 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/501 2024-04-24T08:50:21.697 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:21.707 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:21.740 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:21.741 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/502) 2024-04-24T08:50:22.278 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:22.291 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:22.291 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:22.752 INFO:teuthology.orchestra.run.smithi123.stdout:generic/502 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:22.766 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/502 2024-04-24T08:50:22.766 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/502 2024-04-24T08:50:22.766 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:22.778 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:22.808 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:22.808 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/503) 2024-04-24T08:50:23.339 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:23.353 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:23.353 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:23.785 INFO:teuthology.orchestra.run.smithi123.stdout:generic/503 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:23.798 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/503 2024-04-24T08:50:23.799 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/503 2024-04-24T08:50:23.799 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:23.810 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:23.841 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:23.841 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/504) 2024-04-24T08:50:24.394 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:24.407 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:24.407 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:24.892 INFO:teuthology.orchestra.run.smithi123.stdout:generic/504 - output mismatch (see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/504.out.bad) 2024-04-24T08:50:24.899 INFO:teuthology.orchestra.run.smithi123.stdout: --- tests/generic/504.out 2024-04-24 05:47:53.302731367 +0000 2024-04-24T08:50:24.899 INFO:teuthology.orchestra.run.smithi123.stdout: +++ /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/504.out.bad 2024-04-24 08:50:24.889062619 +0000 2024-04-24T08:50:24.899 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -1,2 +1,3 @@ 2024-04-24T08:50:24.899 INFO:teuthology.orchestra.run.smithi123.stdout: QA output created by 504 2024-04-24T08:50:24.899 INFO:teuthology.orchestra.run.smithi123.stdout: +lock info not found 2024-04-24T08:50:24.899 INFO:teuthology.orchestra.run.smithi123.stdout: Silence is golden 2024-04-24T08:50:24.906 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/504 2024-04-24T08:50:24.907 INFO:teuthology.orchestra.run.smithi123.stdout:Failures: generic/504 2024-04-24T08:50:24.907 INFO:teuthology.orchestra.run.smithi123.stdout:Failed 1 of 1 tests 2024-04-24T08:50:24.915 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:24.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:50:24.947 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-24T08:50:24.947 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-24T08:50:24.947 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/505) 2024-04-24T08:50:25.514 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:25.526 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:25.526 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:25.987 INFO:teuthology.orchestra.run.smithi123.stdout:generic/505 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:25.997 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/505 2024-04-24T08:50:25.997 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/505 2024-04-24T08:50:25.997 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:26.006 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:26.040 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:26.040 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/506) 2024-04-24T08:50:26.547 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:26.560 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:26.560 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:27.020 INFO:teuthology.orchestra.run.smithi123.stdout:generic/506 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:27.033 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/506 2024-04-24T08:50:27.034 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/506 2024-04-24T08:50:27.034 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:27.046 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:27.077 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:27.077 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/507) 2024-04-24T08:50:27.613 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:27.627 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:27.627 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:28.164 INFO:teuthology.orchestra.run.smithi123.stdout:generic/507 [not run] file system doesn't support chattr +ASai 2024-04-24T08:50:28.179 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/507 2024-04-24T08:50:28.179 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/507 2024-04-24T08:50:28.179 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:28.191 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:28.223 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:28.223 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/508) 2024-04-24T08:50:28.755 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:28.768 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:28.768 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:29.234 INFO:teuthology.orchestra.run.smithi123.stdout:generic/508 [not run] lsattr not supported by test filesystem type: ceph-fuse 2024-04-24T08:50:29.244 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/508 2024-04-24T08:50:29.244 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/508 2024-04-24T08:50:29.244 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:29.253 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:29.281 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:29.281 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/509) 2024-04-24T08:50:29.837 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:29.851 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:29.851 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:30.315 INFO:teuthology.orchestra.run.smithi123.stdout:generic/509 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:30.327 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/509 2024-04-24T08:50:30.327 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/509 2024-04-24T08:50:30.327 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:30.336 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:30.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:30.369 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/510) 2024-04-24T08:50:30.916 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:30.928 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:30.929 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:31.368 INFO:teuthology.orchestra.run.smithi123.stdout:generic/510 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:31.382 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/510 2024-04-24T08:50:31.382 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/510 2024-04-24T08:50:31.382 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:31.393 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:31.423 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:31.423 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/511) 2024-04-24T08:50:31.956 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:31.970 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:31.971 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:32.440 INFO:teuthology.orchestra.run.smithi123.stdout:generic/511 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:32.452 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/511 2024-04-24T08:50:32.452 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/511 2024-04-24T08:50:32.452 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:32.460 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:32.488 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:32.489 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/512) 2024-04-24T08:50:33.011 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:33.026 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:33.463 INFO:teuthology.orchestra.run.smithi123.stdout:generic/512 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:33.474 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/512 2024-04-24T08:50:33.474 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/512 2024-04-24T08:50:33.474 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:33.482 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:33.510 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:33.511 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/513) 2024-04-24T08:50:34.039 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:34.053 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:34.053 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:34.491 INFO:teuthology.orchestra.run.smithi123.stdout:generic/513 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:34.503 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/513 2024-04-24T08:50:34.503 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/513 2024-04-24T08:50:34.503 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:34.514 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:34.545 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:34.545 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/514) 2024-04-24T08:50:35.065 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:35.078 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:35.079 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:35.489 INFO:teuthology.orchestra.run.smithi123.stdout:generic/514 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:35.501 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/514 2024-04-24T08:50:35.501 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/514 2024-04-24T08:50:35.501 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:35.511 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:35.540 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:35.540 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/515) 2024-04-24T08:50:36.059 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:36.071 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:36.071 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:36.533 INFO:teuthology.orchestra.run.smithi123.stdout:generic/515 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:36.545 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/515 2024-04-24T08:50:36.546 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/515 2024-04-24T08:50:36.546 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:36.557 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:36.588 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:36.588 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/516) 2024-04-24T08:50:37.111 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:37.126 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:37.126 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:37.745 INFO:teuthology.orchestra.run.smithi123.stdout:generic/516 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-24T08:50:37.759 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/516 2024-04-24T08:50:37.759 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/516 2024-04-24T08:50:37.759 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:37.770 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:37.830 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:37.830 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/517) 2024-04-24T08:50:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:38.514 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:38.514 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:38.982 INFO:teuthology.orchestra.run.smithi123.stdout:generic/517 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:38.996 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/517 2024-04-24T08:50:38.997 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/517 2024-04-24T08:50:38.997 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:39.008 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:39.039 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:39.039 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/518) 2024-04-24T08:50:39.621 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:39.635 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:39.635 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:39.944 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:50:39.950 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:50:40.052 INFO:teuthology.orchestra.run.smithi123.stdout:generic/518 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:40.063 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/518 2024-04-24T08:50:40.063 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/518 2024-04-24T08:50:40.063 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:40.072 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:40.100 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:40.100 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/519) 2024-04-24T08:50:40.685 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:40.696 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:40.696 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:41.111 INFO:teuthology.orchestra.run.smithi123.stdout:generic/519 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:41.121 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/519 2024-04-24T08:50:41.121 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/519 2024-04-24T08:50:41.121 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:41.129 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:41.157 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:41.158 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/520) 2024-04-24T08:50:41.679 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:41.692 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:41.692 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:42.144 INFO:teuthology.orchestra.run.smithi123.stdout:generic/520 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T08:50:42.155 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/520 2024-04-24T08:50:42.155 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/520 2024-04-24T08:50:42.155 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T08:50:42.164 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:50:42.191 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T08:50:42.192 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/521) 2024-04-24T08:50:42.693 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T08:50:42.705 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T08:50:42.705 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T08:51:10.142 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:51:10.151 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:51:40.346 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:51:40.354 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:52:10.478 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:52:10.487 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:52:40.608 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:52:40.617 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:53:10.738 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:53:10.746 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:53:41.473 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:53:41.482 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:54:11.599 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:54:11.608 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:54:41.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:54:41.738 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:55:11.851 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:55:11.860 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:55:41.998 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:55:42.007 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:56:12.238 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:56:12.246 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:56:42.383 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:56:42.395 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:57:12.531 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:57:12.541 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:57:42.661 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:57:42.669 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:58:12.783 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:58:12.793 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T08:58:12.898 INFO:tasks.ceph.mds.a.smithi123.stderr:2024-04-24T08:58:12.898+0000 7f7cdcc00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 427950) UID: 0 2024-04-24T08:58:12.898 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-24T08:58:12.898+0000 7f20c3400640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 427950) UID: 0 2024-04-24T09:01:03.843 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:01:03.852 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:01:34.069 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:01:34.081 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:02:04.201 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:02:04.210 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:02:34.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:02:34.339 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:03:04.461 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:03:04.469 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:03:34.600 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:03:34.609 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:04:04.748 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:04:04.757 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:04:34.885 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:04:34.893 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:05:05.024 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:05:05.035 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:05:35.154 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:05:35.163 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:06:05.369 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:06:05.377 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:06:35.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:06:35.511 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:07:05.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:07:05.828 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:07:35.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:07:35.961 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:08:06.097 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:08:06.108 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:08:36.230 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:08:36.238 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:09:06.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:09:06.360 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:09:36.472 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:09:36.510 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:10:06.635 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:10:06.644 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:10:36.775 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:10:36.785 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:11:06.913 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:11:06.924 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:11:37.043 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:11:37.052 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:12:07.167 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:12:07.176 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:12:37.290 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:12:37.299 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:13:07.403 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:13:07.412 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:13:37.547 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:13:37.556 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:14:07.679 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:14:07.688 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:14:37.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:14:37.848 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:15:07.973 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:15:07.981 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:15:38.095 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:15:38.103 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:16:08.216 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:16:08.226 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:16:38.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:16:38.359 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:17:08.478 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:17:08.488 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:17:38.601 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:17:38.609 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:18:08.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:18:08.738 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:18:38.871 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:18:38.880 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:19:09.000 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:19:09.008 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:19:39.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:19:39.680 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:20:09.801 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:20:09.810 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:20:39.933 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:20:39.942 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:21:10.061 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:21:10.069 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:21:40.193 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:21:40.201 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:22:10.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:22:10.359 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:22:40.491 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:22:40.499 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:23:10.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:23:10.819 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:23:41.273 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:23:41.283 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:24:11.400 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:24:11.409 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:24:41.522 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:24:41.531 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:25:11.644 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:25:11.652 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:25:41.767 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:25:41.776 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:26:11.897 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:26:11.906 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:26:42.028 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:26:42.037 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:27:12.156 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:27:12.165 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:27:42.280 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:27:42.289 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:28:12.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:28:12.410 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:28:42.524 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:28:42.533 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:29:12.655 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:29:12.664 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:29:42.802 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:29:42.836 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:30:12.957 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:30:12.966 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:30:43.079 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:30:43.088 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:31:13.200 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:31:13.210 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:31:43.322 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:31:43.331 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:32:13.469 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:32:13.479 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:32:43.592 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:32:43.601 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:33:13.822 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:33:13.831 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:33:43.954 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:33:43.963 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:34:14.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:34:14.085 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:34:44.197 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:34:44.205 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:35:14.328 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:35:14.339 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:35:44.458 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:35:44.467 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:36:14.599 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:36:14.609 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:36:44.718 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:36:44.729 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:37:14.850 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:37:14.858 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:37:44.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:37:44.982 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:38:15.101 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:38:15.110 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:38:45.614 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:38:45.626 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:39:15.757 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:39:15.766 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:39:45.884 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:39:45.894 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:40:16.007 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:40:16.016 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:40:46.145 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:40:46.154 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:41:16.363 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:41:16.372 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:41:46.505 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:41:46.514 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:42:16.628 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:42:16.636 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:42:46.749 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:42:46.759 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:43:16.879 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:43:16.887 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:43:47.004 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:43:47.012 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:43:47.125 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T09:43:47.125+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47372) UID: 0 2024-04-24T09:43:47.126 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T09:43:47.125+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47372) UID: 0 2024-04-24T09:43:47.126 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T09:43:47.125+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47372) UID: 0 2024-04-24T09:43:47.126 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T09:43:47.125+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47372) UID: 0 2024-04-24T09:46:38.872 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:46:38.882 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:47:09.030 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:47:09.040 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:47:39.161 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:47:39.182 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:48:09.301 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:48:09.312 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:48:39.441 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:48:39.451 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:49:09.588 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:49:09.599 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:49:39.707 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:49:39.717 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:50:09.855 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:50:09.864 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:50:39.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:50:40.006 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:51:10.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:51:10.142 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:51:40.254 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:51:40.263 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:52:10.385 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:52:10.396 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:52:40.516 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:52:40.526 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:53:10.664 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:53:10.673 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:53:40.794 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:53:40.804 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:54:10.924 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:54:10.933 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:54:41.047 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:54:41.055 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:55:11.176 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:55:11.185 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:55:41.306 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:55:41.315 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:56:11.447 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:56:11.456 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:56:41.576 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:56:41.584 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:57:11.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:57:11.714 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:57:41.828 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:57:41.880 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:58:12.025 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:58:12.033 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:58:42.153 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:58:42.165 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:59:12.302 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:59:12.312 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:59:42.445 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T09:59:42.456 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:00:12.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:00:12.583 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:00:42.695 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:00:42.704 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:01:12.818 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:01:12.827 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:01:42.945 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:01:42.955 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:02:13.084 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:02:13.093 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:02:43.219 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:02:43.229 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:03:13.349 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:03:13.360 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:03:43.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:03:43.480 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:04:13.594 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:04:13.603 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:04:43.713 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:04:43.722 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:05:13.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:05:13.846 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:05:43.966 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:05:43.975 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:06:14.107 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:06:14.117 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:06:44.238 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:06:44.247 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:07:14.367 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:07:14.375 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:07:44.490 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:07:44.501 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:08:15.154 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:08:15.163 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:08:45.283 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:08:45.292 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:09:15.425 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:09:15.436 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:09:45.551 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:09:45.581 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:10:15.689 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:10:15.699 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:10:45.822 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:10:45.831 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:11:15.945 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:11:15.954 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:11:46.083 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:11:46.093 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:12:16.205 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:12:16.214 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:12:46.328 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:12:46.336 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:13:16.467 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:13:16.475 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:13:46.604 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:13:46.615 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:14:16.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:14:16.739 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:14:46.886 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:14:46.896 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:15:17.007 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:15:17.016 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:15:47.151 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:15:47.160 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:16:17.280 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:16:17.289 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:16:47.403 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:16:47.412 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:17:17.528 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:17:17.538 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:17:47.659 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:17:47.669 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:18:17.780 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:18:17.789 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:18:47.902 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:18:47.911 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:19:18.041 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:19:18.050 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:19:48.171 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:19:48.181 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:20:18.302 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:20:18.312 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:20:48.476 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:20:48.486 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:21:18.647 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:21:18.656 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:21:48.781 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:21:48.790 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:22:18.903 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:22:18.912 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:22:49.028 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:22:49.036 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:23:19.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:23:19.299 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:23:49.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:23:49.423 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:24:19.818 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:24:19.828 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:24:49.963 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:24:49.971 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:25:20.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:25:20.097 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:25:50.209 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:25:50.218 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:26:20.332 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:26:20.341 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:26:50.455 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:26:50.465 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:27:20.585 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:27:20.594 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:27:50.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:27:50.715 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:28:20.861 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:28:20.870 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:28:50.983 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:28:50.991 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:29:21.128 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:29:21.138 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:29:51.248 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:29:51.257 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:30:21.383 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:30:21.392 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:30:51.505 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:30:51.513 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:31:21.660 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:31:21.669 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:31:51.781 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:31:51.790 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:32:21.905 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:32:21.915 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:32:52.026 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:32:52.037 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:33:22.164 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:33:22.173 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:33:52.295 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:33:52.303 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:34:22.442 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:34:22.450 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:34:52.566 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:34:52.575 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:35:22.694 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:35:22.703 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:35:52.825 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:35:52.834 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:36:22.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:36:22.981 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:36:53.111 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:36:53.120 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:37:23.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:37:23.250 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:37:53.371 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:37:53.380 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:38:23.509 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:38:23.518 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:38:53.630 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:38:53.638 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:39:23.753 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:39:23.761 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:39:54.595 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:39:54.604 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:40:24.749 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:40:24.758 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:40:54.870 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:40:54.879 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:41:25.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:41:25.061 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:41:55.188 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:41:55.198 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:42:25.310 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:42:25.318 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:42:55.459 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:42:55.467 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:43:25.615 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:43:25.624 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:43:55.769 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:43:55.778 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:43:55.876 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T10:43:55.881+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49540) UID: 0 2024-04-24T10:43:55.877 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T10:43:55.881+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49540) UID: 0 2024-04-24T10:43:55.877 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T10:43:55.881+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49540) UID: 0 2024-04-24T10:43:55.877 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T10:43:55.881+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49540) UID: 0 2024-04-24T10:47:07.258 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:47:07.268 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:47:07.510 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T10:47:07.515+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49565) UID: 0 2024-04-24T10:47:07.510 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T10:47:07.515+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49565) UID: 0 2024-04-24T10:47:07.511 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T10:47:07.515+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49565) UID: 0 2024-04-24T10:47:07.511 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T10:47:07.515+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49565) UID: 0 2024-04-24T10:50:04.753 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:50:04.762 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:50:34.955 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:50:34.967 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:51:05.081 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:51:05.091 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:51:35.221 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:51:35.231 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:52:05.407 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:52:05.416 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:52:36.180 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:52:36.190 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:53:06.308 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:53:06.317 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:53:36.649 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:53:36.658 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:53:36.759 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T10:53:36.761+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 431905) UID: 0 2024-04-24T10:53:36.760 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T10:53:36.761+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 431905) UID: 0 2024-04-24T10:53:36.760 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T10:53:36.761+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 431905) UID: 0 2024-04-24T10:53:36.760 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T10:53:36.761+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 431905) UID: 0 2024-04-24T10:56:25.920 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:56:25.930 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:56:56.112 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:56:56.122 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:57:26.694 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:57:26.703 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:57:56.807 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:57:56.815 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:58:26.929 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:58:26.937 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:58:57.059 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:58:57.068 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:59:10.227 INFO:teuthology.orchestra.run.smithi123.stdout:generic/521 7708s 2024-04-24T10:59:10.274 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/521 2024-04-24T10:59:10.274 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T10:59:10.285 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T10:59:10.334 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T10:59:10.334 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/522) 2024-04-24T10:59:11.172 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T10:59:11.184 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T10:59:11.184 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T10:59:27.189 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:59:27.200 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:59:57.322 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T10:59:57.333 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:00:27.525 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:00:27.534 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:00:57.697 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:00:57.707 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:01:27.827 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:01:27.837 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:01:58.017 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:01:58.026 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:02:28.249 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:02:28.258 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:02:58.402 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:02:58.412 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:03:28.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:03:28.560 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:03:58.679 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:03:58.688 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:04:28.819 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:04:28.830 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:04:58.958 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:04:58.966 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:05:29.147 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:05:29.155 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:05:59.369 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:05:59.380 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:06:29.524 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:06:29.534 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:06:59.713 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:06:59.723 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:07:29.843 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:07:29.852 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:08:00.046 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:08:00.055 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:08:30.428 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:08:30.439 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:09:00.567 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:09:00.577 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:09:30.723 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:09:30.731 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:10:00.887 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:10:00.896 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:10:31.060 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:10:31.071 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:11:01.231 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:11:01.240 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:11:31.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:11:31.387 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:12:01.551 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:12:01.560 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:12:31.689 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:12:31.697 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:13:01.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:13:01.820 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:13:31.943 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:13:31.952 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:14:02.064 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:14:02.073 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:14:32.271 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:14:32.279 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:15:02.418 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:15:02.427 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:15:32.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:15:32.584 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:16:02.764 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:16:02.774 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:16:32.893 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:16:32.901 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:17:03.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:17:03.084 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:17:33.197 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:17:33.207 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:18:03.359 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:18:03.367 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:18:33.483 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:18:33.494 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:19:03.640 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:19:03.650 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:19:33.794 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:19:33.805 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:20:04.784 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:20:04.793 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:20:34.934 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:20:34.943 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:21:05.107 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:21:05.116 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:21:35.240 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:21:35.249 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:22:05.367 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:22:05.377 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:22:35.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:22:35.514 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:23:05.681 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:23:05.692 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:23:35.808 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:23:35.817 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:24:05.956 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:24:05.966 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:24:36.154 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:24:36.164 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:25:06.368 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:25:06.377 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:25:36.590 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:25:36.599 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:26:06.712 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:26:06.721 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:26:36.885 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:26:36.894 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:27:07.109 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:27:07.117 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:27:37.439 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:27:37.449 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:27:37.540 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T11:27:37.541+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 433559) UID: 0 2024-04-24T11:27:37.541 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T11:27:37.541+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 433559) UID: 0 2024-04-24T11:27:37.541 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T11:27:37.541+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 433559) UID: 0 2024-04-24T11:27:37.542 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T11:27:37.541+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 433559) UID: 0 2024-04-24T11:31:00.659 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:31:00.667 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:31:30.897 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:31:30.907 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:32:01.033 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:32:01.043 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:32:31.223 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:32:31.231 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:33:01.725 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:33:01.734 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:33:31.850 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:33:31.860 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:34:02.181 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:34:02.192 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:34:32.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:34:32.387 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:35:02.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:35:02.559 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:35:32.739 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:35:32.749 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:36:02.921 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:36:02.930 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:36:33.808 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:36:33.817 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:37:04.020 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:37:04.029 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:37:34.161 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:37:34.170 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:38:04.358 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:38:04.366 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:38:34.572 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:38:34.584 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:39:04.779 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:39:04.789 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:39:34.917 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:39:34.926 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:40:05.080 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:40:05.089 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:40:35.221 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:40:35.231 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:41:05.366 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:41:05.379 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:41:35.581 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:41:35.590 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:42:05.728 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:42:05.736 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:42:35.886 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:42:35.896 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:43:06.030 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:43:06.039 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:43:06.130 INFO:tasks.ceph.mds.a.smithi123.stderr:2024-04-24T11:43:06.131+0000 7f7cdcc00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 434071) UID: 0 2024-04-24T11:43:06.130 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-24T11:43:06.131+0000 7f20c3400640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 434071) UID: 0 2024-04-24T11:46:02.784 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:46:02.793 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:46:33.016 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:46:33.025 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:47:03.156 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:47:03.167 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:47:33.302 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:47:33.329 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:48:03.509 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:48:03.519 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:48:33.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:48:33.682 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:49:03.803 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:49:03.813 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:49:33.974 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:49:33.984 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:50:04.147 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:50:04.156 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:50:34.328 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:50:34.337 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:51:04.475 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:51:04.484 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:51:34.598 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:51:34.607 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:52:04.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:52:04.820 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:52:35.020 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:52:35.030 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:53:05.222 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:53:05.230 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:53:35.337 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:53:35.346 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:54:05.550 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:54:05.562 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:54:36.292 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:54:36.303 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:55:06.436 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:55:06.444 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:55:36.574 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:55:36.585 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:56:06.722 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:56:06.731 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:56:36.902 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:56:36.912 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:57:07.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:57:07.046 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:57:37.222 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:57:37.233 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:58:07.397 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:58:07.405 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:58:37.529 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:58:37.538 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:59:07.704 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:59:07.713 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:59:37.868 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T11:59:37.881 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:00:08.007 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:00:08.015 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:00:38.137 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:00:38.147 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:01:08.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:01:08.268 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:01:38.390 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:01:38.399 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:02:08.563 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:02:08.573 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:02:38.902 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:02:38.913 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:03:09.032 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:03:09.042 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:03:39.170 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:03:39.178 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:04:09.309 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:04:09.322 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:04:39.464 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:04:39.472 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:05:09.594 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:05:09.603 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:05:39.734 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:05:39.744 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:06:09.873 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:06:09.883 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:06:40.011 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:06:40.021 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:07:10.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:07:10.184 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:07:40.296 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:07:40.305 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:08:10.436 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:08:10.446 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:08:40.583 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:08:40.592 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:09:10.756 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:09:10.766 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:09:40.945 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:09:40.954 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:10:11.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:10:11.143 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:10:41.290 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:10:41.299 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:11:11.454 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:11:11.463 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:11:41.643 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:11:41.652 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:12:11.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:12:11.775 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:12:41.978 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:12:41.987 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:13:12.134 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:13:12.143 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:13:42.340 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:13:42.348 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:13:42.455 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T12:13:42.455+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 435131) UID: 0 2024-04-24T12:13:42.455 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T12:13:42.455+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 435131) UID: 0 2024-04-24T12:13:42.455 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T12:13:42.455+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 435131) UID: 0 2024-04-24T12:13:42.456 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T12:13:42.455+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 435131) UID: 0 2024-04-24T12:16:48.072 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:16:48.082 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:17:18.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:17:18.413 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:17:48.532 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:17:48.541 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:18:18.690 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:18:18.703 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:18:48.896 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:18:48.905 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:19:19.021 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:19:19.033 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:19:49.208 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:19:49.217 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:20:19.362 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:20:19.403 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:20:49.530 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:20:49.539 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:21:19.698 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:21:19.708 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:21:49.887 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:21:49.896 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:22:20.043 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:22:20.052 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:22:50.173 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:22:50.182 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:23:20.372 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:23:20.382 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:23:50.537 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:23:50.546 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:24:20.698 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:24:20.709 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:24:50.829 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:24:50.839 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:25:20.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:25:20.958 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:25:51.106 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:25:51.116 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:26:21.245 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:26:21.254 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:26:51.427 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:26:51.436 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:27:21.622 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:27:21.631 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:27:51.761 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:27:51.769 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:28:21.959 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:28:21.969 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:28:52.136 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:28:52.145 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:29:22.328 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:29:22.337 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:29:52.475 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:29:52.484 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:30:22.604 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:30:22.612 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:30:52.821 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:30:52.830 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:31:22.992 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:31:23.001 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:31:53.198 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:31:53.208 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:32:23.335 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:32:23.344 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:32:53.491 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:32:53.500 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:33:23.613 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:33:23.623 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:33:53.751 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:33:53.763 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:34:24.590 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:34:24.599 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:34:24.687 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T12:34:24.687+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 435813) UID: 0 2024-04-24T12:34:24.687 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T12:34:24.687+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 435813) UID: 0 2024-04-24T12:34:24.688 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T12:34:24.687+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 435813) UID: 0 2024-04-24T12:34:24.688 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T12:34:24.687+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 435813) UID: 0 2024-04-24T12:35:07.386 INFO:teuthology.orchestra.run.smithi123.stdout:generic/522 5756s 2024-04-24T12:35:07.483 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/522 2024-04-24T12:35:07.483 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:07.519 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:07.520 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/523) 2024-04-24T12:35:08.654 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:08.668 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:08.668 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:09.811 INFO:teuthology.orchestra.run.smithi123.stdout:generic/523 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T12:35:09.823 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/523 2024-04-24T12:35:09.824 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/523 2024-04-24T12:35:09.824 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:09.833 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:09.865 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:09.866 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/524) 2024-04-24T12:35:10.340 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:10.353 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:10.806 INFO:teuthology.orchestra.run.smithi123.stdout:generic/524 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T12:35:10.817 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/524 2024-04-24T12:35:10.817 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/524 2024-04-24T12:35:10.817 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:10.825 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:10.851 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:10.851 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/525) 2024-04-24T12:35:11.447 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:11.458 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:11.459 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:generic/525 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T12:35:11.843 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/525 2024-04-24T12:35:11.844 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/525 2024-04-24T12:35:11.844 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:11.856 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:11.891 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:11.891 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/526) 2024-04-24T12:35:12.395 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:12.407 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:12.407 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:13.035 INFO:teuthology.orchestra.run.smithi123.stdout:generic/526 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T12:35:13.048 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/526 2024-04-24T12:35:13.048 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/526 2024-04-24T12:35:13.049 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:13.090 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:13.090 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/527) 2024-04-24T12:35:13.572 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:13.584 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:13.585 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:13.978 INFO:teuthology.orchestra.run.smithi123.stdout:generic/527 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T12:35:13.988 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/527 2024-04-24T12:35:13.988 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/527 2024-04-24T12:35:13.988 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:13.997 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:14.026 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:14.026 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/528) 2024-04-24T12:35:14.526 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:14.539 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:14.540 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:15.113 INFO:teuthology.orchestra.run.smithi123.stdout:generic/528 [not run] inode creation time not supported by this filesystem 2024-04-24T12:35:15.125 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/528 2024-04-24T12:35:15.125 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/528 2024-04-24T12:35:15.125 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:15.133 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:15.161 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:15.162 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/529) 2024-04-24T12:35:15.655 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:15.667 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:15.667 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:16.100 INFO:teuthology.orchestra.run.smithi123.stdout:generic/529 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-24T12:35:16.109 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/529 2024-04-24T12:35:16.109 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/529 2024-04-24T12:35:16.109 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:16.143 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:16.143 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/530) 2024-04-24T12:35:16.648 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:16.661 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:17.027 INFO:teuthology.orchestra.run.smithi123.stdout:generic/530 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T12:35:17.038 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/530 2024-04-24T12:35:17.038 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/530 2024-04-24T12:35:17.038 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:17.073 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:17.073 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/531) 2024-04-24T12:35:17.535 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:17.546 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:17.546 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:17.941 INFO:teuthology.orchestra.run.smithi123.stdout:generic/531 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T12:35:17.951 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/531 2024-04-24T12:35:17.951 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/531 2024-04-24T12:35:17.951 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:17.986 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:17.986 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/532) 2024-04-24T12:35:18.459 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:18.472 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:18.472 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:18.972 INFO:teuthology.orchestra.run.smithi123.stdout:generic/532 0s 2024-04-24T12:35:18.986 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/532 2024-04-24T12:35:18.986 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:18.994 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:19.020 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:19.020 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/533) 2024-04-24T12:35:19.479 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:19.491 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:19.491 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:20.333 INFO:teuthology.orchestra.run.smithi123.stdout:generic/533 1s 2024-04-24T12:35:20.347 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/533 2024-04-24T12:35:20.347 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:20.356 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:20.384 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:20.384 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/534) 2024-04-24T12:35:20.899 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:20.912 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:20.913 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:21.689 INFO:teuthology.orchestra.run.smithi123.stdout:generic/534 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T12:35:21.703 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/534 2024-04-24T12:35:21.703 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/534 2024-04-24T12:35:21.703 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:21.712 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:21.742 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:21.742 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/535) 2024-04-24T12:35:22.247 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:22.261 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:22.261 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:22.717 INFO:teuthology.orchestra.run.smithi123.stdout:generic/535 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T12:35:22.727 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/535 2024-04-24T12:35:22.727 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/535 2024-04-24T12:35:22.727 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:22.736 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:22.763 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:22.763 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/536) 2024-04-24T12:35:23.222 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:23.233 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:23.234 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:23.565 INFO:teuthology.orchestra.run.smithi123.stdout:generic/536 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T12:35:23.575 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/536 2024-04-24T12:35:23.575 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/536 2024-04-24T12:35:23.575 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:23.584 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:23.609 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:23.609 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/537) 2024-04-24T12:35:24.084 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:24.094 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:24.094 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:24.449 INFO:teuthology.orchestra.run.smithi123.stdout:generic/537 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T12:35:24.460 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/537 2024-04-24T12:35:24.460 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/537 2024-04-24T12:35:24.461 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T12:35:24.468 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:35:24.496 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T12:35:24.496 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/538) 2024-04-24T12:35:24.964 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T12:35:24.976 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T12:35:24.976 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T12:37:27.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:37:27.333 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:37:57.515 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:37:57.524 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:38:27.646 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:38:27.654 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:38:57.785 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:38:57.794 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:39:27.924 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:39:27.933 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:39:58.055 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:39:58.063 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:40:28.184 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:40:28.193 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:40:58.324 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:40:58.333 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:41:28.443 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:41:28.452 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:41:58.576 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:41:58.584 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:42:28.724 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:42:28.734 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:42:58.853 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:42:58.861 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:43:28.980 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:43:29.013 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:43:59.151 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:43:59.160 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:44:29.273 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:44:29.282 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:44:59.409 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:44:59.418 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:45:29.536 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:45:29.545 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:45:59.654 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:45:59.666 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:46:29.919 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:46:29.929 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:47:00.049 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:47:00.057 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:47:30.172 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:47:30.183 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:48:00.570 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:48:00.579 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:48:30.691 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:48:30.700 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:49:00.830 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:49:00.839 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:49:30.984 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:49:30.993 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:50:01.151 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:50:01.159 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:50:31.388 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:50:31.398 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:51:01.518 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:51:01.527 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:51:31.917 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:51:31.926 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:52:02.096 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:52:02.104 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:52:32.227 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:52:32.235 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:53:02.348 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:53:02.357 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:53:32.470 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:53:32.479 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:54:02.603 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:54:02.611 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:54:32.723 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:54:32.732 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:55:02.870 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:55:02.878 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:55:32.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:55:33.004 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:56:03.114 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:56:03.123 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:56:33.230 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:56:33.236 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:57:03.360 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:57:03.376 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:57:33.492 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:57:33.502 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:58:03.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:58:03.716 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:58:33.834 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:58:33.843 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:59:03.964 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:59:03.973 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:59:34.099 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T12:59:34.159 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:00:04.285 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:00:04.295 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:00:34.600 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:00:34.609 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:01:04.740 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:01:04.749 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:01:34.868 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:01:34.877 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:02:05.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:02:05.011 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:02:35.154 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:02:35.162 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:03:05.303 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:03:05.312 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:03:35.444 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:03:35.462 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:04:05.587 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:04:05.597 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:04:35.726 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:04:35.734 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:05:05.860 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:05:05.871 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:05:36.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:05:36.014 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:06:06.130 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:06:06.140 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:06:36.701 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:06:36.713 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:07:06.829 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:07:06.839 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:07:37.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:07:37.011 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:08:07.132 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:08:07.142 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:08:37.278 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:08:37.287 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:09:07.393 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:09:07.850 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:09:37.965 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:09:37.974 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:10:08.126 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:10:08.137 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:10:38.278 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:10:38.287 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:11:08.630 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:11:08.641 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:11:38.759 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:11:38.769 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:12:08.923 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:12:08.934 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:12:39.057 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:12:39.066 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:13:09.189 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:13:09.199 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:13:39.336 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:13:39.345 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:14:09.474 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:14:09.483 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:14:39.604 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:14:39.613 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:15:09.735 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:15:09.746 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:15:39.883 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:15:39.890 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:16:10.022 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:16:10.043 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:16:40.169 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:16:40.182 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:17:10.324 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:17:10.333 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:17:40.514 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:17:40.523 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:18:10.663 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:18:10.671 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:18:40.782 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:18:40.793 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:19:10.940 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:19:10.948 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:19:41.069 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:19:41.078 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:20:11.225 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:20:11.234 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:20:41.364 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:20:41.373 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:21:11.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:21:11.508 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:21:41.624 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:21:41.635 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:22:12.048 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:22:12.057 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:22:42.203 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:22:42.212 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:23:12.341 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:23:12.350 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:23:42.464 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:23:42.472 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:24:12.594 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:24:12.603 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:24:42.775 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:24:42.784 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:25:12.905 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:25:12.914 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:25:43.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:25:43.045 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:26:13.202 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:26:13.211 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:26:43.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:26:43.334 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:27:13.445 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:27:13.454 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:27:43.568 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:27:43.577 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:28:13.690 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:28:13.701 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:28:43.838 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:28:43.846 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:29:13.986 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:29:13.996 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:29:44.114 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:29:44.121 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:30:14.270 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:30:14.279 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:30:44.402 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:30:44.412 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:31:14.531 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:31:14.540 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:31:44.653 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:31:44.664 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:32:14.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:32:14.820 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:32:44.990 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:32:44.999 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:33:15.108 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:33:15.120 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:33:45.279 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:33:45.289 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:34:15.412 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:34:15.422 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:34:45.783 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:34:45.793 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:35:15.915 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:35:15.925 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:35:46.061 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:35:46.070 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:36:16.722 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:36:16.731 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:36:46.853 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:36:46.861 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:37:17.470 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:37:17.476 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:37:47.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:37:47.605 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:38:17.721 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:38:17.730 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:38:47.992 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:38:48.001 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:39:18.118 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:39:18.130 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:39:48.246 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:39:48.256 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:40:18.393 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:40:18.402 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:40:48.524 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:40:48.533 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:41:18.662 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:41:18.672 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:41:48.809 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:41:48.817 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:41:48.933 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T13:41:48.934+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 55527) UID: 0 2024-04-24T13:41:48.934 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T13:41:48.934+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 55527) UID: 0 2024-04-24T13:41:48.934 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T13:41:48.934+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 55527) UID: 0 2024-04-24T13:41:48.934 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T13:41:48.934+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 55527) UID: 0 2024-04-24T13:44:47.909 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:44:47.920 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:44:48.055 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T13:44:48.056+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 55551) UID: 0 2024-04-24T13:44:48.055 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T13:44:48.056+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 55551) UID: 0 2024-04-24T13:44:48.056 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T13:44:48.056+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 55551) UID: 0 2024-04-24T13:44:48.056 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T13:44:48.056+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 55551) UID: 0 2024-04-24T13:47:50.477 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:47:50.487 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:48:20.797 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:48:20.806 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:48:50.917 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:48:50.925 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:49:21.034 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:49:21.043 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:49:51.172 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:49:51.181 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:50:21.301 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:50:21.310 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:50:51.433 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:50:51.442 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:51:21.554 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:51:21.564 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:51:51.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:51:51.686 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:52:21.825 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:52:21.835 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:52:52.039 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:52:52.048 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:53:22.189 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:53:22.198 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:53:52.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:53:52.332 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:54:22.462 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:54:22.471 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:54:52.644 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:54:52.654 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:55:22.783 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:55:22.792 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:55:52.947 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:55:52.956 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:56:23.068 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:56:23.077 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:56:53.206 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:56:53.216 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:57:23.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:57:23.341 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:57:53.600 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:57:53.609 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:58:23.791 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:58:23.801 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:58:53.952 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:58:53.961 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:59:24.107 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:59:24.118 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:59:54.305 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T13:59:54.314 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:00:24.439 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:00:24.448 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:00:54.570 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:00:54.579 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:01:24.707 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:01:24.716 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:01:54.829 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:01:54.838 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:02:25.260 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:02:25.270 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:02:55.392 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:02:55.408 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:03:25.536 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:03:25.545 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:03:56.076 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:03:56.085 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:04:26.204 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:04:26.213 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:04:56.341 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:04:56.350 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:05:26.490 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:05:26.500 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:05:56.620 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:05:56.629 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:06:26.741 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:06:26.751 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:06:56.872 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:06:56.882 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:07:27.020 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:07:27.030 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:07:57.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:07:57.156 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:08:27.289 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:08:27.298 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:08:57.420 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:08:57.429 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:09:27.577 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:09:27.586 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:09:57.704 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:09:57.713 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:10:27.869 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:10:27.878 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:10:57.991 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:10:58.000 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:11:28.130 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:11:28.139 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:11:58.266 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:11:58.277 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:12:28.382 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:12:28.390 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:12:58.513 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:12:58.522 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:13:28.651 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:13:28.662 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:13:58.790 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:13:58.799 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:14:29.046 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:14:29.055 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:14:59.170 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:14:59.184 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:15:29.306 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:15:29.316 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:15:59.557 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:15:59.566 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:16:29.729 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:16:29.739 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:16:59.866 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:16:59.875 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:17:29.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:17:30.004 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:18:00.184 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:18:00.193 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:18:30.434 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:18:30.443 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:19:00.564 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:19:00.572 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:19:30.682 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:19:30.692 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:20:00.817 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:20:00.826 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:20:30.940 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:20:30.952 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:21:01.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:21:01.250 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:21:31.371 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:21:31.381 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:21:41.895 INFO:teuthology.orchestra.run.smithi123.stdout:generic/538 6377s 2024-04-24T14:21:41.932 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/538 2024-04-24T14:21:41.932 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:21:41.942 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:41.984 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:21:41.984 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/539) 2024-04-24T14:21:43.238 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:21:43.253 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:21:43.253 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:49.661 INFO:teuthology.orchestra.run.smithi123.stdout:generic/539 6s 2024-04-24T14:21:49.678 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/539 2024-04-24T14:21:49.679 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:21:49.687 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:49.728 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:21:49.728 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/540) 2024-04-24T14:21:50.433 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:21:50.444 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:21:50.445 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:50.923 INFO:teuthology.orchestra.run.smithi123.stdout:generic/540 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:21:50.936 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/540 2024-04-24T14:21:50.936 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/540 2024-04-24T14:21:50.937 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:21:50.948 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:50.978 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:21:50.979 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/541) 2024-04-24T14:21:51.521 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:21:51.534 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:21:51.535 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:51.973 INFO:teuthology.orchestra.run.smithi123.stdout:generic/541 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:21:51.986 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/541 2024-04-24T14:21:51.986 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/541 2024-04-24T14:21:51.986 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:21:51.995 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:52.027 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:21:52.027 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/542) 2024-04-24T14:21:52.578 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:21:52.590 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:21:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:53.033 INFO:teuthology.orchestra.run.smithi123.stdout:generic/542 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:21:53.045 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/542 2024-04-24T14:21:53.045 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/542 2024-04-24T14:21:53.045 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:21:53.054 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:53.086 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:21:53.087 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/543) 2024-04-24T14:21:53.637 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:21:53.650 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:21:53.650 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:54.032 INFO:teuthology.orchestra.run.smithi123.stdout:generic/543 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:21:54.043 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/543 2024-04-24T14:21:54.043 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/543 2024-04-24T14:21:54.043 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:21:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:54.081 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:21:54.081 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/544) 2024-04-24T14:21:54.586 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:21:54.598 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:21:54.598 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:55.009 INFO:teuthology.orchestra.run.smithi123.stdout:generic/544 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:21:55.020 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/544 2024-04-24T14:21:55.020 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/544 2024-04-24T14:21:55.020 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:21:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:55.060 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:21:55.060 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/545) 2024-04-24T14:21:55.627 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:21:55.641 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:21:55.641 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:56.231 INFO:teuthology.orchestra.run.smithi123.stdout:generic/545 [not run] file system doesn't support chattr +i 2024-04-24T14:21:56.242 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/545 2024-04-24T14:21:56.242 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/545 2024-04-24T14:21:56.243 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:21:56.252 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:56.284 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:21:56.284 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/546) 2024-04-24T14:21:56.817 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:21:56.830 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:21:56.830 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:57.409 INFO:teuthology.orchestra.run.smithi123.stdout:generic/546 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:21:57.423 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/546 2024-04-24T14:21:57.423 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/546 2024-04-24T14:21:57.423 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:21:57.434 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:57.465 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:21:57.465 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/547) 2024-04-24T14:21:57.997 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:21:58.010 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:21:58.010 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:58.494 INFO:teuthology.orchestra.run.smithi123.stdout:generic/547 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:21:58.507 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/547 2024-04-24T14:21:58.508 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/547 2024-04-24T14:21:58.508 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:21:58.518 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:58.547 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:21:58.548 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/548) 2024-04-24T14:21:59.081 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:21:59.095 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:21:59.095 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:59.678 INFO:teuthology.orchestra.run.smithi123.stdout:generic/548 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:21:59.690 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/548 2024-04-24T14:21:59.690 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/548 2024-04-24T14:21:59.690 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:21:59.700 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:21:59.731 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:21:59.731 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/549) 2024-04-24T14:22:00.258 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:00.272 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:00.273 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:00.762 INFO:teuthology.orchestra.run.smithi123.stdout:generic/549 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:00.773 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/549 2024-04-24T14:22:00.773 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/549 2024-04-24T14:22:00.773 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:00.782 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:00.810 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:00.810 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/550) 2024-04-24T14:22:01.331 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:01.344 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:01.344 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:01.492 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:22:01.498 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:22:01.847 INFO:teuthology.orchestra.run.smithi123.stdout:generic/550 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:01.860 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/550 2024-04-24T14:22:01.860 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/550 2024-04-24T14:22:01.860 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:01.869 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:01.897 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:01.897 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/551) 2024-04-24T14:22:02.399 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:02.412 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:02.412 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:02.783 INFO:teuthology.orchestra.run.smithi123.stdout:generic/551 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:02.792 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/551 2024-04-24T14:22:02.792 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/551 2024-04-24T14:22:02.792 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:02.799 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:02.823 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:02.823 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/552) 2024-04-24T14:22:03.316 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:03.327 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:03.906 INFO:teuthology.orchestra.run.smithi123.stdout:generic/552 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:03.917 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/552 2024-04-24T14:22:03.918 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/552 2024-04-24T14:22:03.918 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:03.926 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:03.953 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:03.953 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/553) 2024-04-24T14:22:04.457 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:04.470 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:04.470 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:05.027 INFO:teuthology.orchestra.run.smithi123.stdout:generic/553 [not run] xfs_io chattr +i failed (old kernel/wrong fs/bad args?) 2024-04-24T14:22:05.038 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/553 2024-04-24T14:22:05.038 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/553 2024-04-24T14:22:05.038 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:05.046 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:05.085 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:05.085 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/554) 2024-04-24T14:22:05.768 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:05.780 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:05.780 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:06.190 INFO:teuthology.orchestra.run.smithi123.stdout:generic/554 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:06.199 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/554 2024-04-24T14:22:06.199 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/554 2024-04-24T14:22:06.199 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:06.207 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:06.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:06.233 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/555) 2024-04-24T14:22:06.748 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:06.762 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:07.286 INFO:teuthology.orchestra.run.smithi123.stdout:generic/555 [not run] xfs_io chattr +ia failed (old kernel/wrong fs/bad args?) 2024-04-24T14:22:07.301 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/555 2024-04-24T14:22:07.302 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/555 2024-04-24T14:22:07.302 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:07.312 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:07.346 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:07.346 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/556) 2024-04-24T14:22:07.995 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:08.008 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:08.392 INFO:teuthology.orchestra.run.smithi123.stdout:generic/556 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:08.402 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/556 2024-04-24T14:22:08.402 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/556 2024-04-24T14:22:08.402 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:08.409 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:08.435 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:08.436 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/557) 2024-04-24T14:22:08.949 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:08.963 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:08.964 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:09.472 INFO:teuthology.orchestra.run.smithi123.stdout:generic/557 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:09.484 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/557 2024-04-24T14:22:09.485 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/557 2024-04-24T14:22:09.485 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:09.496 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:09.525 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:09.526 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/558) 2024-04-24T14:22:10.139 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:10.153 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:10.153 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:10.644 INFO:teuthology.orchestra.run.smithi123.stdout:generic/558 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:10.656 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/558 2024-04-24T14:22:10.656 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/558 2024-04-24T14:22:10.656 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:10.666 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:10.698 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:10.699 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/559) 2024-04-24T14:22:11.257 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:11.270 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:11.271 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:11.720 INFO:teuthology.orchestra.run.smithi123.stdout:generic/559 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:11.732 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/559 2024-04-24T14:22:11.732 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/559 2024-04-24T14:22:11.732 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:11.741 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:11.772 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:11.772 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/560) 2024-04-24T14:22:12.339 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:12.351 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:12.351 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:12.761 INFO:teuthology.orchestra.run.smithi123.stdout:generic/560 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:12.775 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/560 2024-04-24T14:22:12.775 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/560 2024-04-24T14:22:12.775 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:12.787 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:12.822 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:12.822 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/561) 2024-04-24T14:22:13.350 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:13.364 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:13.365 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:13.811 INFO:teuthology.orchestra.run.smithi123.stdout:generic/561 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:13.824 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/561 2024-04-24T14:22:13.824 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/561 2024-04-24T14:22:13.824 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:13.833 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:13.862 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:13.862 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/562) 2024-04-24T14:22:14.370 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:14.384 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:14.384 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:14.784 INFO:teuthology.orchestra.run.smithi123.stdout:generic/562 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:14.799 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/562 2024-04-24T14:22:14.799 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/562 2024-04-24T14:22:14.799 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:14.811 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:14.841 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:14.842 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/563) 2024-04-24T14:22:15.370 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:15.384 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:15.384 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:15.836 INFO:teuthology.orchestra.run.smithi123.stdout:generic/563 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:15.851 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/563 2024-04-24T14:22:15.851 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/563 2024-04-24T14:22:15.851 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:15.862 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:15.896 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:15.896 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/564) 2024-04-24T14:22:16.406 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:16.420 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:16.420 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:17.608 INFO:teuthology.orchestra.run.smithi123.stdout:generic/564 1s 2024-04-24T14:22:17.625 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/564 2024-04-24T14:22:17.625 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:17.634 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:17.682 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:17.683 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/565) 2024-04-24T14:22:18.389 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:18.400 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:18.401 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:18.801 INFO:teuthology.orchestra.run.smithi123.stdout:generic/565 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:18.813 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/565 2024-04-24T14:22:18.813 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/565 2024-04-24T14:22:18.813 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:18.822 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:18.851 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:18.851 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/566) 2024-04-24T14:22:19.376 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:19.390 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:19.390 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:19.811 INFO:teuthology.orchestra.run.smithi123.stdout:generic/566 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:19.824 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/566 2024-04-24T14:22:19.824 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/566 2024-04-24T14:22:19.824 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:19.836 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:19.873 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:19.873 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/567) 2024-04-24T14:22:20.413 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:20.426 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:20.426 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:20.849 INFO:teuthology.orchestra.run.smithi123.stdout:generic/567 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:20.859 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/567 2024-04-24T14:22:20.859 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/567 2024-04-24T14:22:20.859 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:20.867 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:20.891 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:20.892 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/568) 2024-04-24T14:22:21.438 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:21.451 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:21.451 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:22.109 INFO:teuthology.orchestra.run.smithi123.stdout:generic/568 1s 2024-04-24T14:22:22.147 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/568 2024-04-24T14:22:22.147 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:22.163 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:22.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:22.213 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/569) 2024-04-24T14:22:22.787 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:22.800 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:22.800 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:23.343 INFO:teuthology.orchestra.run.smithi123.stdout:generic/569 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:23.356 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/569 2024-04-24T14:22:23.356 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/569 2024-04-24T14:22:23.356 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:23.365 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:23.397 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:23.398 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/570) 2024-04-24T14:22:23.885 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:23.898 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:23.899 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:24.299 INFO:teuthology.orchestra.run.smithi123.stdout:generic/570 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:24.313 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/570 2024-04-24T14:22:24.314 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/570 2024-04-24T14:22:24.314 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:24.359 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:24.359 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/571) 2024-04-24T14:22:24.885 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:24.898 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:24.898 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:31.590 INFO:teuthology.orchestra.run.smithi123.stdout:generic/571 7s 2024-04-24T14:22:31.606 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/571 2024-04-24T14:22:31.606 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:31.615 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:31.642 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:31.642 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/572) 2024-04-24T14:22:31.657 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:22:31.704 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:22:32.336 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:32.351 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:32.351 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:32.746 INFO:teuthology.orchestra.run.smithi123.stdout:generic/572 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:32.759 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/572 2024-04-24T14:22:32.759 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/572 2024-04-24T14:22:32.759 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:32.769 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:32.796 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:32.797 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/573) 2024-04-24T14:22:33.334 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:33.349 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:33.349 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:33.776 INFO:teuthology.orchestra.run.smithi123.stdout:generic/573 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:33.787 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/573 2024-04-24T14:22:33.787 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/573 2024-04-24T14:22:33.787 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:33.796 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:33.826 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:33.826 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/574) 2024-04-24T14:22:34.362 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:34.375 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:34.375 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:34.788 INFO:teuthology.orchestra.run.smithi123.stdout:generic/574 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:34.797 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/574 2024-04-24T14:22:34.798 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/574 2024-04-24T14:22:34.798 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:34.806 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:34.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:34.832 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/575) 2024-04-24T14:22:35.366 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:35.379 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:35.379 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:35.810 INFO:teuthology.orchestra.run.smithi123.stdout:generic/575 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:35.822 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/575 2024-04-24T14:22:35.822 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/575 2024-04-24T14:22:35.822 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:35.832 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:35.862 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:35.863 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/576) 2024-04-24T14:22:36.383 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:36.398 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:36.398 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:36.936 INFO:teuthology.orchestra.run.smithi123.stdout:generic/576 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:36.951 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/576 2024-04-24T14:22:36.951 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/576 2024-04-24T14:22:36.951 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:36.962 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:36.996 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:36.997 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/577) 2024-04-24T14:22:37.526 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:37.539 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:37.539 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:37.940 INFO:teuthology.orchestra.run.smithi123.stdout:generic/577 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:37.953 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/577 2024-04-24T14:22:37.953 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/577 2024-04-24T14:22:37.954 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:37.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:37.994 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/578) 2024-04-24T14:22:38.519 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:38.531 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:38.531 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:39.055 INFO:teuthology.orchestra.run.smithi123.stdout:generic/578 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-24T14:22:39.071 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/578 2024-04-24T14:22:39.071 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/578 2024-04-24T14:22:39.071 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:39.084 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:39.120 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:39.120 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/579) 2024-04-24T14:22:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:39.638 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:39.638 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:40.096 INFO:teuthology.orchestra.run.smithi123.stdout:generic/579 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:40.108 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/579 2024-04-24T14:22:40.108 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/579 2024-04-24T14:22:40.108 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:40.117 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:40.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:40.147 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/580) 2024-04-24T14:22:40.677 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:40.689 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:40.690 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:41.194 INFO:teuthology.orchestra.run.smithi123.stdout:generic/580 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:41.204 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/580 2024-04-24T14:22:41.204 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/580 2024-04-24T14:22:41.205 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:41.214 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:41.246 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:41.247 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/581) 2024-04-24T14:22:41.771 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:41.784 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:41.784 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:43.046 INFO:teuthology.orchestra.run.smithi123.stdout:generic/581 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:43.057 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/581 2024-04-24T14:22:43.058 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/581 2024-04-24T14:22:43.058 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:43.067 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:43.099 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:43.099 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/582) 2024-04-24T14:22:43.622 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:43.634 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:43.635 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:44.159 INFO:teuthology.orchestra.run.smithi123.stdout:generic/582 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:44.171 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/582 2024-04-24T14:22:44.171 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/582 2024-04-24T14:22:44.171 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:44.182 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:44.212 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:44.212 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/583) 2024-04-24T14:22:44.749 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:44.763 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:44.763 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:45.267 INFO:teuthology.orchestra.run.smithi123.stdout:generic/583 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:45.279 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/583 2024-04-24T14:22:45.279 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/583 2024-04-24T14:22:45.279 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:45.289 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:45.321 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:45.321 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/584) 2024-04-24T14:22:45.869 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:45.883 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:45.883 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:46.405 INFO:teuthology.orchestra.run.smithi123.stdout:generic/584 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:46.419 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/584 2024-04-24T14:22:46.420 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/584 2024-04-24T14:22:46.420 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:46.431 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:46.464 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:46.464 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/585) 2024-04-24T14:22:46.994 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:47.006 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:47.007 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:47.452 INFO:teuthology.orchestra.run.smithi123.stdout:generic/585 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:22:47.467 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/585 2024-04-24T14:22:47.467 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/585 2024-04-24T14:22:47.467 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:22:47.480 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:22:47.511 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:22:47.512 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/586) 2024-04-24T14:22:48.043 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:22:48.058 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:22:48.058 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:01.903 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:23:01.914 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:23:32.025 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:23:32.035 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:23:37.652 INFO:teuthology.orchestra.run.smithi123.stdout:generic/586 49s 2024-04-24T14:23:37.669 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/586 2024-04-24T14:23:37.669 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:37.678 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:37.714 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:37.714 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/587) 2024-04-24T14:23:38.402 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:38.414 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:39.481 INFO:teuthology.orchestra.run.smithi123.stdout:generic/587 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-24T14:23:39.496 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/587 2024-04-24T14:23:39.496 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/587 2024-04-24T14:23:39.496 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:39.507 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:39.540 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:39.540 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/588) 2024-04-24T14:23:40.058 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:40.070 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:40.070 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:40.542 INFO:teuthology.orchestra.run.smithi123.stdout:generic/588 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:23:40.556 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/588 2024-04-24T14:23:40.557 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/588 2024-04-24T14:23:40.557 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:40.569 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:40.599 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:40.599 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/589) 2024-04-24T14:23:41.134 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:41.148 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:41.148 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:41.586 INFO:teuthology.orchestra.run.smithi123.stdout:generic/589 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:23:41.597 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/589 2024-04-24T14:23:41.598 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/589 2024-04-24T14:23:41.598 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:41.606 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:41.633 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:41.634 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/590) 2024-04-24T14:23:42.153 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:42.165 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:42.165 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:42.562 INFO:teuthology.orchestra.run.smithi123.stdout:generic/590 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:23:42.574 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/590 2024-04-24T14:23:42.574 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/590 2024-04-24T14:23:42.574 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:42.583 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:42.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:42.613 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/591) 2024-04-24T14:23:43.148 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:43.161 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:43.161 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:44.017 INFO:teuthology.orchestra.run.smithi123.stdout:generic/591 1s 2024-04-24T14:23:44.034 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/591 2024-04-24T14:23:44.034 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:44.079 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:44.079 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/592) 2024-04-24T14:23:44.766 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:44.777 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:44.777 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:45.251 INFO:teuthology.orchestra.run.smithi123.stdout:generic/592 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:23:45.266 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/592 2024-04-24T14:23:45.266 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/592 2024-04-24T14:23:45.266 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:45.279 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:45.310 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:45.310 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/593) 2024-04-24T14:23:45.843 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:45.855 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:45.856 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:46.361 INFO:teuthology.orchestra.run.smithi123.stdout:generic/593 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:23:46.373 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/593 2024-04-24T14:23:46.374 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/593 2024-04-24T14:23:46.374 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:46.386 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:46.420 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:46.420 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/594) 2024-04-24T14:23:46.943 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:46.956 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:46.957 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:47.379 INFO:teuthology.orchestra.run.smithi123.stdout:generic/594 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:23:47.393 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/594 2024-04-24T14:23:47.393 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/594 2024-04-24T14:23:47.393 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:47.405 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:47.439 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:47.440 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/595) 2024-04-24T14:23:47.990 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:48.003 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:48.004 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:48.513 INFO:teuthology.orchestra.run.smithi123.stdout:generic/595 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:23:48.524 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/595 2024-04-24T14:23:48.524 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/595 2024-04-24T14:23:48.524 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:48.535 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:48.566 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:48.566 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/596) 2024-04-24T14:23:49.087 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:49.099 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:49.099 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:49.646 INFO:teuthology.orchestra.run.smithi123.stdout:generic/596 [not run] file system doesn't support chattr +S 2024-04-24T14:23:49.655 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/596 2024-04-24T14:23:49.656 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/596 2024-04-24T14:23:49.656 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:49.663 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:49.688 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:49.689 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/597) 2024-04-24T14:23:50.215 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:50.229 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:50.230 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:52.076 INFO:teuthology.orchestra.run.smithi123.stdout:generic/597 - output mismatch (see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/597.out.bad) 2024-04-24T14:23:52.083 INFO:teuthology.orchestra.run.smithi123.stdout: --- tests/generic/597.out 2024-04-24 05:47:53.317731013 +0000 2024-04-24T14:23:52.084 INFO:teuthology.orchestra.run.smithi123.stdout: +++ /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/597.out.bad 2024-04-24 14:23:52.068413900 +0000 2024-04-24T14:23:52.084 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -4,11 +4,11 @@ 2024-04-24T14:23:52.084 INFO:teuthology.orchestra.run.smithi123.stdout: fs.protected_symlinks = 0 2024-04-24T14:23:52.084 INFO:teuthology.orchestra.run.smithi123.stdout: successfully followed symlink 2024-04-24T14:23:52.084 INFO:teuthology.orchestra.run.smithi123.stdout: fs.protected_symlinks = 1 2024-04-24T14:23:52.084 INFO:teuthology.orchestra.run.smithi123.stdout: -Permission denied 2024-04-24T14:23:52.084 INFO:teuthology.orchestra.run.smithi123.stdout: +successfully followed symlink 2024-04-24T14:23:52.084 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:52.084 INFO:teuthology.orchestra.run.smithi123.stdout: == Test hardlink create protection when 2024-04-24T14:23:52.084 INFO:teuthology.orchestra.run.smithi123.stdout: == process != target owner and process cannot read target 2024-04-24T14:23:52.084 INFO:teuthology.orchestra.run.smithi123.stdout: fs.protected_hardlinks = 0 2024-04-24T14:23:52.084 INFO:teuthology.orchestra.run.smithi123.stdout: -successfully created hardlink 2024-04-24T14:23:52.085 INFO:teuthology.orchestra.run.smithi123.stdout: +Permission denied 2024-04-24T14:23:52.085 INFO:teuthology.orchestra.run.smithi123.stdout: fs.protected_hardlinks = 1 2024-04-24T14:23:52.085 INFO:teuthology.orchestra.run.smithi123.stdout: -ln: failed to create hard link 'TEST_DIR/597/sticky_dir/hardlink' => 'TEST_DIR/597/target': Operation not permitted 2024-04-24T14:23:52.085 INFO:teuthology.orchestra.run.smithi123.stdout: +Permission denied 2024-04-24T14:23:52.091 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/597 2024-04-24T14:23:52.091 INFO:teuthology.orchestra.run.smithi123.stdout:Failures: generic/597 2024-04-24T14:23:52.091 INFO:teuthology.orchestra.run.smithi123.stdout:Failed 1 of 1 tests 2024-04-24T14:23:52.099 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:52.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T14:23:52.124 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-24T14:23:52.125 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-24T14:23:52.125 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/598) 2024-04-24T14:23:52.872 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:52.884 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:52.884 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:54.737 INFO:teuthology.orchestra.run.smithi123.stdout:generic/598 - output mismatch (see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/598.out.bad) 2024-04-24T14:23:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: --- tests/generic/598.out 2024-04-24 05:47:53.317731013 +0000 2024-04-24T14:23:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: +++ /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/598.out.bad 2024-04-24 14:23:54.739350795 +0000 2024-04-24T14:23:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -5,13 +5,10 @@ 2024-04-24T14:23:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: = only group writable dir 2024-04-24T14:23:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: fs.protected_regular = 1 2024-04-24T14:23:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: = group & world writable dir 2024-04-24T14:23:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: -Permission denied 2024-04-24T14:23:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: = only group writable dir 2024-04-24T14:23:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: fs.protected_regular = 2 2024-04-24T14:23:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: = group & world writable dir 2024-04-24T14:23:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: -Permission denied 2024-04-24T14:23:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: = only group writable dir 2024-04-24T14:23:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: -Permission denied 2024-04-24T14:23:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: == Test fifo open when owned by another and fifo owner != dir owner 2024-04-24T14:23:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: fs.protected_fifos = 0 2024-04-24T14:23:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -19,10 +16,7 @@ 2024-04-24T14:23:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: = only group writable dir 2024-04-24T14:23:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: fs.protected_fifos = 1 2024-04-24T14:23:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: = group & world writable dir 2024-04-24T14:23:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: -Permission denied 2024-04-24T14:23:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: = only group writable dir 2024-04-24T14:23:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: fs.protected_fifos = 2 2024-04-24T14:23:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: = group & world writable dir 2024-04-24T14:23:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: -Permission denied 2024-04-24T14:23:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: = only group writable dir 2024-04-24T14:23:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: -Permission denied 2024-04-24T14:23:54.752 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/598 2024-04-24T14:23:54.752 INFO:teuthology.orchestra.run.smithi123.stdout:Failures: generic/598 2024-04-24T14:23:54.752 INFO:teuthology.orchestra.run.smithi123.stdout:Failed 1 of 1 tests 2024-04-24T14:23:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:54.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T14:23:54.793 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-24T14:23:54.794 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-24T14:23:54.794 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/599) 2024-04-24T14:23:55.499 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:55.512 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:55.512 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:55.969 INFO:teuthology.orchestra.run.smithi123.stdout:generic/599 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:23:55.983 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/599 2024-04-24T14:23:55.984 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/599 2024-04-24T14:23:55.984 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:55.994 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:56.028 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:56.028 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/600) 2024-04-24T14:23:56.590 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:56.604 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:56.604 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:57.045 INFO:teuthology.orchestra.run.smithi123.stdout:generic/600 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:23:57.059 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/600 2024-04-24T14:23:57.060 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/600 2024-04-24T14:23:57.060 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:57.070 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:57.107 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:57.108 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/601) 2024-04-24T14:23:57.645 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:57.657 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:57.657 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:58.068 INFO:teuthology.orchestra.run.smithi123.stdout:generic/601 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:23:58.080 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/601 2024-04-24T14:23:58.080 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/601 2024-04-24T14:23:58.080 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:58.089 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:58.118 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:58.118 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/602) 2024-04-24T14:23:58.635 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:58.647 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:58.647 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:59.173 INFO:teuthology.orchestra.run.smithi123.stdout:generic/602 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:23:59.184 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/602 2024-04-24T14:23:59.184 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/602 2024-04-24T14:23:59.184 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:23:59.193 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:23:59.222 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:23:59.222 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/603) 2024-04-24T14:23:59.753 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:23:59.765 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:23:59.765 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:00.206 INFO:teuthology.orchestra.run.smithi123.stdout:generic/603 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:00.216 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/603 2024-04-24T14:24:00.216 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/603 2024-04-24T14:24:00.216 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:00.225 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:00.255 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:00.255 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/604) 2024-04-24T14:24:00.780 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:00.794 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:00.795 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:01.233 INFO:teuthology.orchestra.run.smithi123.stdout:generic/604 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:01.245 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/604 2024-04-24T14:24:01.245 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/604 2024-04-24T14:24:01.245 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:01.257 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:01.290 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:01.290 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/605) 2024-04-24T14:24:01.831 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:01.845 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:01.845 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:02.157 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:24:02.164 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:24:02.282 INFO:teuthology.orchestra.run.smithi123.stdout:generic/605 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:02.292 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/605 2024-04-24T14:24:02.292 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/605 2024-04-24T14:24:02.292 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:02.301 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:02.329 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:02.330 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/606) 2024-04-24T14:24:02.906 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:02.920 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:02.920 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:03.325 INFO:teuthology.orchestra.run.smithi123.stdout:generic/606 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:03.335 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/606 2024-04-24T14:24:03.336 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/606 2024-04-24T14:24:03.336 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:03.378 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:03.378 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/607) 2024-04-24T14:24:03.878 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:03.890 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:03.890 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:04.302 INFO:teuthology.orchestra.run.smithi123.stdout:generic/607 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:04.316 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/607 2024-04-24T14:24:04.316 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/607 2024-04-24T14:24:04.317 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:04.328 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:04.363 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:04.363 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/608) 2024-04-24T14:24:04.884 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:04.898 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:04.899 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:05.324 INFO:teuthology.orchestra.run.smithi123.stdout:generic/608 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:05.336 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/608 2024-04-24T14:24:05.336 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/608 2024-04-24T14:24:05.336 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:05.346 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:05.378 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:05.378 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/609) 2024-04-24T14:24:05.917 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:05.928 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:05.928 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:07.009 INFO:teuthology.orchestra.run.smithi123.stdout:generic/609 2s 2024-04-24T14:24:07.026 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/609 2024-04-24T14:24:07.026 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:07.034 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:07.087 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:07.087 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/610) 2024-04-24T14:24:07.793 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:07.808 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:07.808 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:08.267 INFO:teuthology.orchestra.run.smithi123.stdout:generic/610 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:08.279 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/610 2024-04-24T14:24:08.279 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/610 2024-04-24T14:24:08.279 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:08.291 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:08.323 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:08.324 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/611) 2024-04-24T14:24:08.878 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:08.891 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:08.892 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:09.298 INFO:teuthology.orchestra.run.smithi123.stdout:generic/611 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:09.309 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/611 2024-04-24T14:24:09.310 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/611 2024-04-24T14:24:09.310 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:09.319 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:09.348 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:09.349 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/612) 2024-04-24T14:24:09.910 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:09.923 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:09.924 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:10.489 INFO:teuthology.orchestra.run.smithi123.stdout:generic/612 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T14:24:10.502 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/612 2024-04-24T14:24:10.502 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/612 2024-04-24T14:24:10.502 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:10.513 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:10.562 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:10.562 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/613) 2024-04-24T14:24:11.248 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:11.260 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:11.260 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:11.820 INFO:teuthology.orchestra.run.smithi123.stdout:generic/613 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:11.832 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/613 2024-04-24T14:24:11.833 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/613 2024-04-24T14:24:11.833 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:11.843 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:11.873 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:11.873 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/614) 2024-04-24T14:24:12.422 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:12.435 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:12.878 INFO:teuthology.orchestra.run.smithi123.stdout:generic/614 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:12.891 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/614 2024-04-24T14:24:12.892 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/614 2024-04-24T14:24:12.892 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:12.903 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:12.939 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:12.939 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/615) 2024-04-24T14:24:13.463 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:13.476 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:13.476 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:13.870 INFO:teuthology.orchestra.run.smithi123.stdout:generic/615 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:13.880 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/615 2024-04-24T14:24:13.880 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/615 2024-04-24T14:24:13.880 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:13.888 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:13.914 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:13.915 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/616) 2024-04-24T14:24:14.439 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:14.455 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:14.455 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:14.946 INFO:teuthology.orchestra.run.smithi123.stdout:generic/616 [not run] kernel does not support IO_URING 2024-04-24T14:24:14.960 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/616 2024-04-24T14:24:14.960 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/616 2024-04-24T14:24:14.960 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:14.970 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:15.001 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:15.001 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/617) 2024-04-24T14:24:15.528 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:15.540 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:16.060 INFO:teuthology.orchestra.run.smithi123.stdout:generic/617 [not run] kernel does not support IO_URING 2024-04-24T14:24:16.072 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/617 2024-04-24T14:24:16.072 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/617 2024-04-24T14:24:16.072 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:16.083 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:16.116 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:16.116 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/618) 2024-04-24T14:24:16.631 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:16.645 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:16.645 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:17.039 INFO:teuthology.orchestra.run.smithi123.stdout:generic/618 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:17.051 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/618 2024-04-24T14:24:17.051 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/618 2024-04-24T14:24:17.051 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:17.092 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:17.093 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/619) 2024-04-24T14:24:17.620 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:17.633 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:17.633 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:18.045 INFO:teuthology.orchestra.run.smithi123.stdout:generic/619 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:18.057 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/619 2024-04-24T14:24:18.057 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/619 2024-04-24T14:24:18.057 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:18.065 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:18.096 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:18.097 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/620) 2024-04-24T14:24:18.622 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:18.636 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:18.636 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:19.134 INFO:teuthology.orchestra.run.smithi123.stdout:generic/620 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:19.148 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/620 2024-04-24T14:24:19.149 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/620 2024-04-24T14:24:19.149 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:19.164 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:19.194 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:19.194 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/621) 2024-04-24T14:24:19.711 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:19.725 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:19.726 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:20.224 INFO:teuthology.orchestra.run.smithi123.stdout:generic/621 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:20.239 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/621 2024-04-24T14:24:20.239 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/621 2024-04-24T14:24:20.239 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:20.251 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:20.288 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:20.288 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/622) 2024-04-24T14:24:20.831 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:20.844 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:20.845 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:21.250 INFO:teuthology.orchestra.run.smithi123.stdout:generic/622 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:21.262 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/622 2024-04-24T14:24:21.262 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/622 2024-04-24T14:24:21.262 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:21.272 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:21.300 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:21.301 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/623) 2024-04-24T14:24:21.824 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:21.839 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:21.839 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:22.247 INFO:teuthology.orchestra.run.smithi123.stdout:generic/623 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:22.258 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/623 2024-04-24T14:24:22.258 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/623 2024-04-24T14:24:22.258 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:22.268 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:22.300 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:22.300 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/624) 2024-04-24T14:24:22.834 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:22.849 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:22.849 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:23.252 INFO:teuthology.orchestra.run.smithi123.stdout:generic/624 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:23.263 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/624 2024-04-24T14:24:23.263 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/624 2024-04-24T14:24:23.263 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:23.273 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:23.301 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:23.301 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/625) 2024-04-24T14:24:23.834 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:23.847 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:23.847 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:24.264 INFO:teuthology.orchestra.run.smithi123.stdout:generic/625 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:24.275 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/625 2024-04-24T14:24:24.275 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/625 2024-04-24T14:24:24.276 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:24.312 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:24.313 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/626) 2024-04-24T14:24:24.861 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:24.875 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:24.875 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:25.315 INFO:teuthology.orchestra.run.smithi123.stdout:generic/626 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:25.327 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/626 2024-04-24T14:24:25.327 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/626 2024-04-24T14:24:25.327 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:25.335 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:25.365 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:25.366 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/627) 2024-04-24T14:24:25.909 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:25.922 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:25.922 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:26.357 INFO:teuthology.orchestra.run.smithi123.stdout:generic/627 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:26.369 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/627 2024-04-24T14:24:26.369 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/627 2024-04-24T14:24:26.369 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:26.378 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:26.409 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:26.409 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/628) 2024-04-24T14:24:26.934 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:26.947 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:26.947 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:27.418 INFO:teuthology.orchestra.run.smithi123.stdout:generic/628 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:27.432 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/628 2024-04-24T14:24:27.432 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/628 2024-04-24T14:24:27.432 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:27.443 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:27.475 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:27.476 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/629) 2024-04-24T14:24:27.998 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:28.009 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:28.010 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:28.439 INFO:teuthology.orchestra.run.smithi123.stdout:generic/629 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:28.451 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/629 2024-04-24T14:24:28.451 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/629 2024-04-24T14:24:28.451 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:28.461 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:28.493 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:28.493 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/630) 2024-04-24T14:24:28.989 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:29.002 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:29.002 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:29.409 INFO:teuthology.orchestra.run.smithi123.stdout:generic/630 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:29.420 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/630 2024-04-24T14:24:29.420 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/630 2024-04-24T14:24:29.421 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:29.430 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:29.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:29.462 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/631) 2024-04-24T14:24:30.006 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:30.020 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:30.021 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:30.508 INFO:teuthology.orchestra.run.smithi123.stdout:generic/631 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:30.520 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/631 2024-04-24T14:24:30.520 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/631 2024-04-24T14:24:30.521 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:30.531 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:30.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:30.561 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/632) 2024-04-24T14:24:31.092 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:31.105 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:32.286 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:24:32.295 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:24:51.621 INFO:teuthology.orchestra.run.smithi123.stdout:generic/632 20s 2024-04-24T14:24:51.638 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/632 2024-04-24T14:24:51.638 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:51.647 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:51.679 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:51.680 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/633) 2024-04-24T14:24:52.205 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:52.218 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:53.150 INFO:teuthology.orchestra.run.smithi123.stdout:generic/633 [failed, exit status 1]- output mismatch (see /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/633.out.bad) 2024-04-24T14:24:53.179 INFO:teuthology.orchestra.run.smithi123.stdout: --- tests/generic/633.out 2024-04-24 05:47:53.323730872 +0000 2024-04-24T14:24:53.179 INFO:teuthology.orchestra.run.smithi123.stdout: +++ /tmp/tmp.ejcLBa8fOwxfstests-dev/results//generic/633.out.bad 2024-04-24 14:24:53.150970710 +0000 2024-04-24T14:24:53.179 INFO:teuthology.orchestra.run.smithi123.stdout: @@ -1,2 +1,5 @@ 2024-04-24T14:24:53.179 INFO:teuthology.orchestra.run.smithi123.stdout: QA output created by 633 2024-04-24T14:24:53.179 INFO:teuthology.orchestra.run.smithi123.stdout: Silence is golden 2024-04-24T14:24:53.180 INFO:teuthology.orchestra.run.smithi123.stdout: +vfstest.c: 1164: protected_symlinks - Success - failure: openat 2024-04-24T14:24:53.180 INFO:teuthology.orchestra.run.smithi123.stdout: +vfstest.c: 1171: protected_symlinks - Success - failure: wait_for_pid 2024-04-24T14:24:53.180 INFO:teuthology.orchestra.run.smithi123.stdout: +vfstest.c: 2418: run_test - Success - failure: following protected symlinks on regular mounts 2024-04-24T14:24:53.189 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/633 2024-04-24T14:24:53.190 INFO:teuthology.orchestra.run.smithi123.stdout:Failures: generic/633 2024-04-24T14:24:53.190 INFO:teuthology.orchestra.run.smithi123.stdout:Failed 1 of 1 tests 2024-04-24T14:24:53.208 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:53.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T14:24:53.260 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-24T14:24:53.260 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-24T14:24:53.260 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/634) 2024-04-24T14:24:53.842 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:53.856 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:54.274 INFO:teuthology.orchestra.run.smithi123.stdout:generic/634 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:54.285 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/634 2024-04-24T14:24:54.286 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/634 2024-04-24T14:24:54.286 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:54.325 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:54.325 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/635) 2024-04-24T14:24:54.857 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:54.870 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:55.283 INFO:teuthology.orchestra.run.smithi123.stdout:generic/635 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:55.293 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/635 2024-04-24T14:24:55.294 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/635 2024-04-24T14:24:55.294 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:55.302 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:55.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:55.332 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/636) 2024-04-24T14:24:55.855 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:55.867 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:55.867 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:56.278 INFO:teuthology.orchestra.run.smithi123.stdout:generic/636 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:24:56.289 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/636 2024-04-24T14:24:56.289 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/636 2024-04-24T14:24:56.289 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:24:56.298 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:24:56.329 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:24:56.330 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/637) 2024-04-24T14:24:56.876 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:24:56.890 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:24:56.890 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:00.202 INFO:teuthology.orchestra.run.smithi123.stdout:generic/637 4s 2024-04-24T14:25:00.222 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/637 2024-04-24T14:25:00.222 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:25:00.232 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:00.265 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:25:00.266 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/638) 2024-04-24T14:25:00.993 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:25:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:25:01.005 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:01.675 INFO:teuthology.orchestra.run.smithi123.stdout:generic/638 0s 2024-04-24T14:25:01.692 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/638 2024-04-24T14:25:01.692 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:25:01.701 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:01.731 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:25:01.733 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/639) 2024-04-24T14:25:02.256 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:25:02.270 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:25:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:02.434 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:25:02.441 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:25:07.392 INFO:teuthology.orchestra.run.smithi123.stdout:generic/639 5s 2024-04-24T14:25:07.406 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/639 2024-04-24T14:25:07.406 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:25:07.414 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:07.440 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:25:07.440 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/640) 2024-04-24T14:25:07.986 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:25:07.999 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:25:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:08.450 INFO:teuthology.orchestra.run.smithi123.stdout:generic/640 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:25:08.462 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/640 2024-04-24T14:25:08.462 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/640 2024-04-24T14:25:08.462 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:25:08.472 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:08.500 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:25:08.501 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/641) 2024-04-24T14:25:09.034 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:25:09.047 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:25:09.047 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:09.501 INFO:teuthology.orchestra.run.smithi123.stdout:generic/641 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:25:09.513 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/641 2024-04-24T14:25:09.513 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/641 2024-04-24T14:25:09.513 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:25:09.522 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:09.551 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:25:09.551 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/642) 2024-04-24T14:25:10.080 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:25:10.093 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:25:10.093 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:10.550 INFO:teuthology.orchestra.run.smithi123.stdout:generic/642 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:25:10.564 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/642 2024-04-24T14:25:10.564 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/642 2024-04-24T14:25:10.564 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:25:10.576 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:10.609 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:25:10.610 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/643) 2024-04-24T14:25:11.152 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:25:11.166 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:25:11.166 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:11.599 INFO:teuthology.orchestra.run.smithi123.stdout:generic/643 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:25:11.610 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/643 2024-04-24T14:25:11.610 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/643 2024-04-24T14:25:11.610 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:25:11.618 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:11.646 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:25:11.647 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/644) 2024-04-24T14:25:12.174 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:25:12.188 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:25:12.189 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:12.646 INFO:teuthology.orchestra.run.smithi123.stdout:generic/644 [not run] vfstest not support by ceph-fuse 2024-04-24T14:25:12.657 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/644 2024-04-24T14:25:12.657 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/644 2024-04-24T14:25:12.657 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:25:12.667 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:12.700 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:25:12.700 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/645) 2024-04-24T14:25:13.247 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:25:13.259 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:25:13.260 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:13.734 INFO:teuthology.orchestra.run.smithi123.stdout:generic/645 [not run] vfstest not support by ceph-fuse 2024-04-24T14:25:13.745 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/645 2024-04-24T14:25:13.745 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/645 2024-04-24T14:25:13.745 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:25:13.754 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:13.784 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:25:13.785 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/646) 2024-04-24T14:25:14.317 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:25:14.331 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:25:14.331 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:14.756 INFO:teuthology.orchestra.run.smithi123.stdout:generic/646 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:25:14.766 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/646 2024-04-24T14:25:14.766 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/646 2024-04-24T14:25:14.767 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:25:14.774 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:14.803 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:25:14.803 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/647) 2024-04-24T14:25:15.332 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:25:15.347 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:25:15.347 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:16.195 INFO:teuthology.orchestra.run.smithi123.stdout:generic/647 1s 2024-04-24T14:25:16.213 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/647 2024-04-24T14:25:16.213 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:25:16.223 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:16.271 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:25:16.274 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/648) 2024-04-24T14:25:16.981 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:25:16.995 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:25:16.995 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:17.451 INFO:teuthology.orchestra.run.smithi123.stdout:generic/648 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T14:25:17.464 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/648 2024-04-24T14:25:17.464 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/648 2024-04-24T14:25:17.464 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:25:17.476 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:17.511 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:25:17.512 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/649) 2024-04-24T14:25:18.036 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:25:18.050 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:25:18.050 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:18.625 INFO:teuthology.orchestra.run.smithi123.stdout:generic/649 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T14:25:18.636 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/649 2024-04-24T14:25:18.636 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/649 2024-04-24T14:25:18.636 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T14:25:18.644 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:18.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T14:25:18.693 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/650) 2024-04-24T14:25:19.377 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T14:25:19.389 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T14:25:19.389 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T14:25:32.631 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:25:32.642 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:26:02.832 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:26:02.840 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:26:33.589 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:26:33.598 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:27:03.851 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:27:03.864 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:27:34.045 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:27:34.057 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:28:04.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:28:04.517 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:28:35.057 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:28:35.066 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:28:35.170 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T14:28:35.176+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57114) UID: 0 2024-04-24T14:28:35.171 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T14:28:35.176+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57114) UID: 0 2024-04-24T14:28:35.172 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T14:28:35.178+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57114) UID: 0 2024-04-24T14:28:35.184 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T14:28:35.176+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57114) UID: 0 2024-04-24T14:32:15.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:32:15.509 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:32:15.613 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T14:32:15.618+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57143) UID: 0 2024-04-24T14:32:15.613 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T14:32:15.618+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57143) UID: 0 2024-04-24T14:32:15.614 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T14:32:15.619+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57143) UID: 0 2024-04-24T14:32:15.614 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T14:32:15.619+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57143) UID: 0 2024-04-24T14:36:11.709 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:36:11.718 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:36:11.910 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T14:36:11.916+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 505005) UID: 0 2024-04-24T14:36:11.911 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T14:36:11.917+0000 7fed5d000640 -1 Fail to open '/proc/505005/cmdline' error = (2) No such file or directory 2024-04-24T14:36:11.911 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T14:36:11.917+0000 7fed5d000640 -1 received signal: Hangup from (PID: 505005) UID: 0 2024-04-24T14:36:11.912 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T14:36:11.914+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 505005) UID: 0 2024-04-24T14:36:11.914 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T14:36:11.915+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 505005) UID: 0 2024-04-24T14:41:44.293 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:41:44.305 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:41:44.458 INFO:tasks.ceph.mds.a.smithi123.stderr:2024-04-24T14:41:44.463+0000 7f7cdcc00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 506178) UID: 0 2024-04-24T14:41:44.459 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-24T14:41:44.465+0000 7f20c3400640 -1 received signal: Hangup from (PID: 506178) UID: 0 2024-04-24T14:47:35.925 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T14:47:35.930+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 507309) UID: 0 2024-04-24T14:47:35.926 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T14:47:35.930+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 507309) UID: 0 2024-04-24T14:47:35.928 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T14:47:35.934+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 507309) UID: 0 2024-04-24T14:47:35.932 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T14:47:35.935+0000 7f388d600640 -1 received signal: Hangup from (PID: 507309) UID: 0 2024-04-24T14:59:54.616 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:59:54.622 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T14:59:54.718 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T14:59:54.718+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 509579) UID: 0 2024-04-24T14:59:54.719 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T14:59:54.719+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 509579) UID: 0 2024-04-24T14:59:54.719 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T14:59:54.720+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 509579) UID: 0 2024-04-24T14:59:54.720 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T14:59:54.720+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 509579) UID: 0 2024-04-24T15:06:39.574 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T15:06:39.583 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T15:06:39.750 INFO:tasks.ceph.mds.a.smithi123.stderr:2024-04-24T15:06:39.748+0000 7f7cdcc00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 510893) UID: 0 2024-04-24T15:06:39.750 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-24T15:06:39.749+0000 7f20c3400640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 510893) UID: 0 2024-04-24T15:12:28.641 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T15:12:28.650 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T15:12:28.833 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T15:12:28.832+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57313) UID: 0 2024-04-24T15:12:28.834 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T15:12:28.832+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57313) UID: 0 2024-04-24T15:12:28.834 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T15:12:28.832+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57313) UID: 0 2024-04-24T15:12:28.834 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T15:12:28.833+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57313) UID: 0 2024-04-24T15:24:04.303 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T15:24:04.308 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T15:24:04.477 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T15:24:04.476+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57358) UID: 0 2024-04-24T15:24:04.478 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T15:24:04.476+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57358) UID: 0 2024-04-24T15:24:04.478 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T15:24:04.476+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57358) UID: 0 2024-04-24T15:24:04.478 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T15:24:04.476+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57358) UID: 0 2024-04-24T15:25:59.992 INFO:teuthology.orchestra.run.smithi123.stdout:generic/650 3640s 2024-04-24T15:26:00.044 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/650 2024-04-24T15:26:00.045 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:00.052 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:00.080 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:00.080 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/651) 2024-04-24T15:26:00.950 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:00.964 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:00.964 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:01.433 INFO:teuthology.orchestra.run.smithi123.stdout:generic/651 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:01.445 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/651 2024-04-24T15:26:01.445 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/651 2024-04-24T15:26:01.445 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:01.453 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:01.486 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:01.486 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/652) 2024-04-24T15:26:02.098 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:02.119 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:02.119 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:02.567 INFO:teuthology.orchestra.run.smithi123.stdout:generic/652 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:02.581 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/652 2024-04-24T15:26:02.581 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/652 2024-04-24T15:26:02.581 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:02.591 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:02.618 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:02.619 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/653) 2024-04-24T15:26:03.220 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:03.237 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:03.660 INFO:teuthology.orchestra.run.smithi123.stdout:generic/653 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:03.674 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/653 2024-04-24T15:26:03.674 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/653 2024-04-24T15:26:03.675 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:03.686 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:03.728 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:03.728 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/654) 2024-04-24T15:26:04.295 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:04.308 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:04.308 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:04.829 INFO:teuthology.orchestra.run.smithi123.stdout:generic/654 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:04.840 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/654 2024-04-24T15:26:04.841 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/654 2024-04-24T15:26:04.841 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:04.852 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:04.887 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:04.888 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/655) 2024-04-24T15:26:05.445 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:05.460 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:05.460 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:05.851 INFO:teuthology.orchestra.run.smithi123.stdout:generic/655 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:05.864 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/655 2024-04-24T15:26:05.865 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/655 2024-04-24T15:26:05.865 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:05.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:05.903 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/656) 2024-04-24T15:26:06.440 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:06.453 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:06.910 INFO:teuthology.orchestra.run.smithi123.stdout:generic/656 [not run] vfstest not support by ceph-fuse 2024-04-24T15:26:06.923 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/656 2024-04-24T15:26:06.924 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/656 2024-04-24T15:26:06.924 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:06.934 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:06.962 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:06.962 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/657) 2024-04-24T15:26:07.527 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:07.541 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:07.944 INFO:teuthology.orchestra.run.smithi123.stdout:generic/657 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:07.957 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/657 2024-04-24T15:26:07.957 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/657 2024-04-24T15:26:07.957 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:07.996 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:07.996 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/658) 2024-04-24T15:26:08.549 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:08.561 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:08.561 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:08.931 INFO:teuthology.orchestra.run.smithi123.stdout:generic/658 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:08.943 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/658 2024-04-24T15:26:08.943 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/658 2024-04-24T15:26:08.943 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:08.950 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:08.978 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:08.978 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/659) 2024-04-24T15:26:09.524 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:09.538 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:09.539 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:10.186 INFO:teuthology.orchestra.run.smithi123.stdout:generic/659 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:10.200 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/659 2024-04-24T15:26:10.200 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/659 2024-04-24T15:26:10.200 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:10.209 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:10.248 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:10.248 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/660) 2024-04-24T15:26:10.819 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:10.838 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:11.379 INFO:teuthology.orchestra.run.smithi123.stdout:generic/660 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:11.392 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/660 2024-04-24T15:26:11.392 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/660 2024-04-24T15:26:11.393 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:11.400 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:11.434 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:11.434 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/661) 2024-04-24T15:26:11.995 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:12.008 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:12.008 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:12.434 INFO:teuthology.orchestra.run.smithi123.stdout:generic/661 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/661 2024-04-24T15:26:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/661 2024-04-24T15:26:12.446 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:12.454 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:12.481 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:12.481 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/662) 2024-04-24T15:26:13.050 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:13.067 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:13.068 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:13.448 INFO:teuthology.orchestra.run.smithi123.stdout:generic/662 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:13.459 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/662 2024-04-24T15:26:13.459 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/662 2024-04-24T15:26:13.459 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:13.467 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:13.496 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:13.497 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/663) 2024-04-24T15:26:14.061 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:14.077 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:14.078 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:14.472 INFO:teuthology.orchestra.run.smithi123.stdout:generic/663 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:14.484 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/663 2024-04-24T15:26:14.485 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/663 2024-04-24T15:26:14.485 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:14.493 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:14.528 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:14.528 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/664) 2024-04-24T15:26:15.082 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:15.097 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:15.097 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:15.464 INFO:teuthology.orchestra.run.smithi123.stdout:generic/664 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:15.476 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/664 2024-04-24T15:26:15.476 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/664 2024-04-24T15:26:15.476 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:15.484 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:15.512 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:15.512 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/665) 2024-04-24T15:26:16.079 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:16.093 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:16.093 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:16.460 INFO:teuthology.orchestra.run.smithi123.stdout:generic/665 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:16.471 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/665 2024-04-24T15:26:16.471 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/665 2024-04-24T15:26:16.471 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:16.512 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:16.512 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/666) 2024-04-24T15:26:17.056 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:17.072 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:17.465 INFO:teuthology.orchestra.run.smithi123.stdout:generic/666 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:17.474 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/666 2024-04-24T15:26:17.474 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/666 2024-04-24T15:26:17.474 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:17.481 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:17.511 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:17.511 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/667) 2024-04-24T15:26:18.080 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:18.096 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:18.096 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:18.464 INFO:teuthology.orchestra.run.smithi123.stdout:generic/667 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:18.476 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/667 2024-04-24T15:26:18.476 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/667 2024-04-24T15:26:18.476 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:18.487 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:18.515 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:18.515 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/668) 2024-04-24T15:26:19.061 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:19.074 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:19.074 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:19.456 INFO:teuthology.orchestra.run.smithi123.stdout:generic/668 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:19.474 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/668 2024-04-24T15:26:19.474 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/668 2024-04-24T15:26:19.474 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:19.481 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:19.513 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:19.513 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/669) 2024-04-24T15:26:20.074 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:20.089 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:20.089 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:20.479 INFO:teuthology.orchestra.run.smithi123.stdout:generic/669 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:20.490 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/669 2024-04-24T15:26:20.491 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/669 2024-04-24T15:26:20.491 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:20.499 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:20.526 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:20.526 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/670) 2024-04-24T15:26:21.095 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:21.107 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:21.107 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:21.479 INFO:teuthology.orchestra.run.smithi123.stdout:generic/670 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:21.491 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/670 2024-04-24T15:26:21.491 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/670 2024-04-24T15:26:21.491 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:21.500 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:21.531 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:21.531 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/671) 2024-04-24T15:26:22.070 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:22.084 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:22.084 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:22.471 INFO:teuthology.orchestra.run.smithi123.stdout:generic/671 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:22.480 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/671 2024-04-24T15:26:22.480 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/671 2024-04-24T15:26:22.480 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:22.489 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:22.514 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:22.515 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/672) 2024-04-24T15:26:23.071 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:23.087 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:23.087 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:23.590 INFO:teuthology.orchestra.run.smithi123.stdout:generic/672 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:23.603 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/672 2024-04-24T15:26:23.603 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/672 2024-04-24T15:26:23.604 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:23.611 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:23.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:23.635 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/673) 2024-04-24T15:26:24.185 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:24.199 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:24.199 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:25.222 INFO:teuthology.orchestra.run.smithi123.stdout:generic/673 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:25.233 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/673 2024-04-24T15:26:25.233 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/673 2024-04-24T15:26:25.234 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:25.244 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:25.280 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:25.280 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/674) 2024-04-24T15:26:25.825 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:25.838 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:25.839 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:26.277 INFO:teuthology.orchestra.run.smithi123.stdout:generic/674 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:26.289 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/674 2024-04-24T15:26:26.290 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/674 2024-04-24T15:26:26.290 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:26.298 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:26.328 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:26.329 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/675) 2024-04-24T15:26:26.858 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:26.874 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:26.874 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:27.339 INFO:teuthology.orchestra.run.smithi123.stdout:generic/675 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:26:27.350 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/675 2024-04-24T15:26:27.350 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/675 2024-04-24T15:26:27.350 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:26:27.359 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:26:27.390 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:26:27.390 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/676) 2024-04-24T15:26:27.944 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:26:27.957 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:26:27.958 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:28:36.072 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T15:28:36.077 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T15:28:36.187 INFO:tasks.ceph.mds.a.smithi123.stderr:2024-04-24T15:28:36.186+0000 7f7cdcc00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 525219) UID: 0 2024-04-24T15:28:36.188 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-24T15:28:36.186+0000 7f20c3400640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 525219) UID: 0 2024-04-24T15:37:40.211 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T15:37:40.211+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 525229) UID: 0 2024-04-24T15:37:40.212 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T15:37:40.211+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 525229) UID: 0 2024-04-24T15:37:40.212 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T15:37:40.211+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 525229) UID: 0 2024-04-24T15:37:40.213 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T15:37:40.211+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 525229) UID: 0 2024-04-24T15:53:41.961 INFO:teuthology.orchestra.run.smithi123.stdout:generic/676 1634s 2024-04-24T15:53:42.023 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/676 2024-04-24T15:53:42.023 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:53:42.032 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:53:42.073 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:53:42.074 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/677) 2024-04-24T15:53:43.681 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:53:43.691 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:53:43.691 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:53:44.298 INFO:teuthology.orchestra.run.smithi123.stdout:generic/677 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:53:44.313 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/677 2024-04-24T15:53:44.313 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/677 2024-04-24T15:53:44.313 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:53:44.325 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:53:44.364 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:53:44.364 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/678) 2024-04-24T15:53:45.009 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:53:45.025 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:53:45.025 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:53:45.618 INFO:teuthology.orchestra.run.smithi123.stdout:generic/678 [not run] kernel does not support IO_URING 2024-04-24T15:53:45.631 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/678 2024-04-24T15:53:45.632 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/678 2024-04-24T15:53:45.632 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:53:45.648 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:53:45.684 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:53:45.684 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/679) 2024-04-24T15:53:46.328 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:53:46.346 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:53:46.346 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:53:47.275 INFO:teuthology.orchestra.run.smithi123.stdout:generic/679 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:53:47.287 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/679 2024-04-24T15:53:47.287 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/679 2024-04-24T15:53:47.287 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:53:47.296 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:53:47.327 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:53:47.327 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/680) 2024-04-24T15:53:47.908 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:53:47.921 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:53:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:53:56.832 INFO:teuthology.orchestra.run.smithi123.stdout:generic/680 9s 2024-04-24T15:53:56.855 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/680 2024-04-24T15:53:56.855 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:53:56.866 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:53:56.911 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:53:56.911 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/681) 2024-04-24T15:53:57.756 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:53:57.770 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:53:57.770 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:53:58.372 INFO:teuthology.orchestra.run.smithi123.stdout:generic/681 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-24T15:53:58.383 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/681 2024-04-24T15:53:58.384 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/681 2024-04-24T15:53:58.384 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:53:58.396 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:53:58.436 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:53:58.436 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/682) 2024-04-24T15:53:59.100 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:53:59.116 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:53:59.116 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:53:59.553 INFO:teuthology.orchestra.run.smithi123.stdout:generic/682 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-24T15:53:59.568 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/682 2024-04-24T15:53:59.568 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/682 2024-04-24T15:53:59.568 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:53:59.579 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:53:59.619 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:53:59.619 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/683) 2024-04-24T15:54:00.222 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:00.236 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:00.236 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:01.552 INFO:teuthology.orchestra.run.smithi123.stdout:generic/683 [not run] /tmp/tmp.CHueoLyoRxtest: file alloc unit 4194304 larger than op length 65536 2024-04-24T15:54:01.566 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/683 2024-04-24T15:54:01.566 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/683 2024-04-24T15:54:01.566 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:01.612 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:01.613 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/684) 2024-04-24T15:54:02.256 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:02.277 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:05.092 INFO:teuthology.orchestra.run.smithi123.stdout:generic/684 [not run] /tmp/tmp.CHueoLyoRxtest: file alloc unit 4194304 larger than op length 65536 2024-04-24T15:54:05.106 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/684 2024-04-24T15:54:05.107 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/684 2024-04-24T15:54:05.107 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:05.117 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:05.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:05.150 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/685) 2024-04-24T15:54:05.730 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:05.751 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:05.752 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:08.764 INFO:teuthology.orchestra.run.smithi123.stdout:generic/685 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-24T15:54:08.777 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/685 2024-04-24T15:54:08.777 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/685 2024-04-24T15:54:08.778 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:08.789 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:08.827 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:08.827 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/686) 2024-04-24T15:54:09.475 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:09.496 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:09.497 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:12.436 INFO:teuthology.orchestra.run.smithi123.stdout:generic/686 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-24T15:54:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/686 2024-04-24T15:54:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/686 2024-04-24T15:54:12.450 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:12.464 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:12.501 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:12.501 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/687) 2024-04-24T15:54:13.068 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:13.083 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:15.932 INFO:teuthology.orchestra.run.smithi123.stdout:generic/687 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-04-24T15:54:15.943 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/687 2024-04-24T15:54:15.944 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/687 2024-04-24T15:54:15.944 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:15.953 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:15.988 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:15.988 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/688) 2024-04-24T15:54:16.561 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:16.583 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:16.583 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:17.584 INFO:teuthology.orchestra.run.smithi123.stdout:generic/688 [not run] /tmp/tmp.CHueoLyoRxtest: file alloc unit 4194304 larger than op length 65536 2024-04-24T15:54:17.596 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/688 2024-04-24T15:54:17.596 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/688 2024-04-24T15:54:17.596 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:17.604 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:17.635 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:17.635 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/689) 2024-04-24T15:54:18.242 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:18.257 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:18.257 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:18.856 INFO:teuthology.orchestra.run.smithi123.stdout:generic/689 [not run] vfstest not support by ceph-fuse 2024-04-24T15:54:18.869 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/689 2024-04-24T15:54:18.870 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/689 2024-04-24T15:54:18.871 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:18.883 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:18.918 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:18.918 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/690) 2024-04-24T15:54:19.609 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:19.623 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:19.623 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:20.295 INFO:teuthology.orchestra.run.smithi123.stdout:generic/690 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:54:20.309 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/690 2024-04-24T15:54:20.310 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/690 2024-04-24T15:54:20.310 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:20.321 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:20.360 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:20.360 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/691) 2024-04-24T15:54:21.042 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:21.058 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:21.058 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:21.584 INFO:teuthology.orchestra.run.smithi123.stdout:generic/691 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:54:21.596 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/691 2024-04-24T15:54:21.597 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/691 2024-04-24T15:54:21.597 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:21.605 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:21.643 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:21.643 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/692) 2024-04-24T15:54:22.278 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:22.292 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:22.292 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:22.799 INFO:teuthology.orchestra.run.smithi123.stdout:generic/692 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:54:22.812 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/692 2024-04-24T15:54:22.812 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/692 2024-04-24T15:54:22.813 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:22.822 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:22.865 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:22.865 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/693) 2024-04-24T15:54:23.487 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:23.502 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:23.502 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:24.098 INFO:teuthology.orchestra.run.smithi123.stdout:generic/693 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:54:24.111 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/693 2024-04-24T15:54:24.111 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/693 2024-04-24T15:54:24.111 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:24.123 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:24.156 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:24.156 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/694) 2024-04-24T15:54:24.767 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:24.785 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:24.785 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:26.014 INFO:teuthology.orchestra.run.smithi123.stdout:generic/694 2s 2024-04-24T15:54:26.033 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/694 2024-04-24T15:54:26.033 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:26.042 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:26.079 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:26.079 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/695) 2024-04-24T15:54:26.712 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:26.730 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:26.730 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:27.286 INFO:teuthology.orchestra.run.smithi123.stdout:generic/695 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:54:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/695 2024-04-24T15:54:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/695 2024-04-24T15:54:27.300 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:27.309 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:27.343 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:27.344 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/696) 2024-04-24T15:54:27.921 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:27.937 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:27.937 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:28.394 INFO:teuthology.orchestra.run.smithi123.stdout:generic/696 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:54:28.406 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/696 2024-04-24T15:54:28.407 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/696 2024-04-24T15:54:28.407 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:28.414 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:28.450 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:28.450 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/697) 2024-04-24T15:54:29.102 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:29.115 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:29.666 INFO:teuthology.orchestra.run.smithi123.stdout:generic/697 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-24T15:54:29.677 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/697 2024-04-24T15:54:29.677 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/697 2024-04-24T15:54:29.678 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:29.688 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:29.722 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:29.722 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/698) 2024-04-24T15:54:30.374 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:30.392 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:30.392 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:30.985 INFO:teuthology.orchestra.run.smithi123.stdout:generic/698 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:54:30.996 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/698 2024-04-24T15:54:30.996 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/698 2024-04-24T15:54:30.996 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:31.006 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:31.032 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:31.033 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/699) 2024-04-24T15:54:31.653 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:31.668 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:31.669 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:32.445 INFO:teuthology.orchestra.run.smithi123.stdout:generic/699 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:54:32.460 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/699 2024-04-24T15:54:32.460 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/699 2024-04-24T15:54:32.461 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:32.471 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:32.502 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:32.503 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/700) 2024-04-24T15:54:33.074 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:33.091 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:33.091 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:33.528 INFO:teuthology.orchestra.run.smithi123.stdout:generic/700 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:54:33.539 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/700 2024-04-24T15:54:33.539 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/700 2024-04-24T15:54:33.539 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:33.548 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:33.578 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:33.578 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/701) 2024-04-24T15:54:34.194 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:34.212 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:34.212 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:35.141 INFO:teuthology.orchestra.run.smithi123.stdout:generic/701 1s 2024-04-24T15:54:35.164 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/701 2024-04-24T15:54:35.164 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:35.173 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:35.210 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:35.210 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/702) 2024-04-24T15:54:36.102 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:36.122 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:36.122 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:36.557 INFO:teuthology.orchestra.run.smithi123.stdout:generic/702 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:54:36.571 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/702 2024-04-24T15:54:36.571 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/702 2024-04-24T15:54:36.571 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:36.584 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:36.622 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:36.622 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/703) 2024-04-24T15:54:37.244 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:37.262 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:37.262 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:37.731 INFO:teuthology.orchestra.run.smithi123.stdout:generic/703 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:54:37.744 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/703 2024-04-24T15:54:37.744 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/703 2024-04-24T15:54:37.744 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:37.754 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:37.791 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:37.791 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/704) 2024-04-24T15:54:38.418 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:38.437 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:39.007 INFO:teuthology.orchestra.run.smithi123.stdout:generic/704 [not run] require ceph-fuse to be valid block disk 2024-04-24T15:54:39.020 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/704 2024-04-24T15:54:39.020 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/704 2024-04-24T15:54:39.020 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:39.027 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:39.061 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:39.061 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/705) 2024-04-24T15:54:39.837 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:39.850 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:39.850 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:40.484 INFO:teuthology.orchestra.run.smithi123.stdout:generic/705 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:54:40.495 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/705 2024-04-24T15:54:40.495 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/705 2024-04-24T15:54:40.495 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:40.505 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:40.540 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:40.541 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/706) 2024-04-24T15:54:41.110 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:41.125 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:41.125 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:43.539 INFO:teuthology.orchestra.run.smithi123.stdout:generic/706 2s 2024-04-24T15:54:43.557 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/706 2024-04-24T15:54:43.558 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:43.566 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:43.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:43.597 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/707) 2024-04-24T15:54:44.435 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:44.452 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:44.452 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:44.871 INFO:teuthology.orchestra.run.smithi123.stdout:generic/707 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:54:44.885 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/707 2024-04-24T15:54:44.885 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/707 2024-04-24T15:54:44.885 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:44.895 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:44.929 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:44.929 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/708) 2024-04-24T15:54:45.571 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:45.585 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:45.585 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:46.671 INFO:teuthology.orchestra.run.smithi123.stdout:generic/708 1s 2024-04-24T15:54:46.694 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/708 2024-04-24T15:54:46.695 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:46.706 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:46.743 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:46.743 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/709) 2024-04-24T15:54:47.518 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:47.552 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:48.067 INFO:teuthology.orchestra.run.smithi123.stdout:generic/709 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-24T15:54:48.078 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/709 2024-04-24T15:54:48.078 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/709 2024-04-24T15:54:48.078 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:48.118 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:48.118 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/710) 2024-04-24T15:54:48.903 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:48.918 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:48.918 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:49.419 INFO:teuthology.orchestra.run.smithi123.stdout:generic/710 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-24T15:54:49.430 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/710 2024-04-24T15:54:49.431 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/710 2024-04-24T15:54:49.431 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:49.440 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:49.488 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:49.488 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/711) 2024-04-24T15:54:50.262 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:50.275 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:50.275 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:50.808 INFO:teuthology.orchestra.run.smithi123.stdout:generic/711 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-24T15:54:50.819 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/711 2024-04-24T15:54:50.819 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/711 2024-04-24T15:54:50.819 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:50.827 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:50.875 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:50.876 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/712) 2024-04-24T15:54:51.642 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:51.657 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:51.657 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:52.157 INFO:teuthology.orchestra.run.smithi123.stdout:generic/712 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-24T15:54:52.168 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/712 2024-04-24T15:54:52.169 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/712 2024-04-24T15:54:52.169 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:52.231 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:52.231 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/713) 2024-04-24T15:54:52.986 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:53.003 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:53.003 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:53.481 INFO:teuthology.orchestra.run.smithi123.stdout:generic/713 [not run] xfs_io swapext -v exchrange -s 64k -l 64k not supported on ceph-fuse 2024-04-24T15:54:53.494 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/713 2024-04-24T15:54:53.494 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/713 2024-04-24T15:54:53.494 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:53.505 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:53.542 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:53.543 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/714) 2024-04-24T15:54:54.341 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:54.356 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:54.833 INFO:teuthology.orchestra.run.smithi123.stdout:generic/714 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-24T15:54:54.844 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/714 2024-04-24T15:54:54.844 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/714 2024-04-24T15:54:54.844 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:54.882 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:54.882 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/715) 2024-04-24T15:54:55.679 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:55.695 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:55.695 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:56.193 INFO:teuthology.orchestra.run.smithi123.stdout:generic/715 [not run] xfs_io swapext -v exchrange -s 64k -l 64k not supported on ceph-fuse 2024-04-24T15:54:56.209 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/715 2024-04-24T15:54:56.209 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/715 2024-04-24T15:54:56.209 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:56.215 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:56.251 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:56.251 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/716) 2024-04-24T15:54:57.041 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:57.054 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:57.054 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:57.576 INFO:teuthology.orchestra.run.smithi123.stdout:generic/716 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-24T15:54:57.590 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/716 2024-04-24T15:54:57.590 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/716 2024-04-24T15:54:57.590 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:57.598 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:57.629 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:57.630 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/717) 2024-04-24T15:54:58.389 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:58.407 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:58.407 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:58.880 INFO:teuthology.orchestra.run.smithi123.stdout:generic/717 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-24T15:54:58.894 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/717 2024-04-24T15:54:58.894 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/717 2024-04-24T15:54:58.894 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:54:58.900 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:54:58.929 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:54:58.929 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/718) 2024-04-24T15:54:59.715 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:54:59.730 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:54:59.730 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:00.237 INFO:teuthology.orchestra.run.smithi123.stdout:generic/718 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-24T15:55:00.251 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/718 2024-04-24T15:55:00.252 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/718 2024-04-24T15:55:00.252 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:00.260 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:00.300 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:00.300 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/719) 2024-04-24T15:55:01.051 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:01.065 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:01.065 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:01.613 INFO:teuthology.orchestra.run.smithi123.stdout:generic/719 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-24T15:55:01.626 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/719 2024-04-24T15:55:01.626 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/719 2024-04-24T15:55:01.626 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:01.635 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:01.658 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:01.658 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/720) 2024-04-24T15:55:02.420 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:02.436 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:02.437 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:02.939 INFO:teuthology.orchestra.run.smithi123.stdout:generic/720 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-24T15:55:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/720 2024-04-24T15:55:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/720 2024-04-24T15:55:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:02.961 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:02.993 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:02.993 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/721) 2024-04-24T15:55:03.745 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:03.763 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:03.763 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:04.237 INFO:teuthology.orchestra.run.smithi123.stdout:generic/721 [not run] xfs_io startupdate support is missing 2024-04-24T15:55:04.249 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/721 2024-04-24T15:55:04.250 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/721 2024-04-24T15:55:04.250 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:04.259 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:04.295 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:04.295 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/722) 2024-04-24T15:55:04.840 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:04.854 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:04.854 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:05.345 INFO:teuthology.orchestra.run.smithi123.stdout:generic/722 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-24T15:55:05.359 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/722 2024-04-24T15:55:05.359 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/722 2024-04-24T15:55:05.360 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:05.404 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:05.404 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/723) 2024-04-24T15:55:06.189 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:06.204 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:06.204 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:06.802 INFO:teuthology.orchestra.run.smithi123.stdout:generic/723 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-24T15:55:06.813 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/723 2024-04-24T15:55:06.814 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/723 2024-04-24T15:55:06.814 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:06.823 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:06.850 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:06.850 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/724) 2024-04-24T15:55:07.660 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:07.677 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:07.677 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:08.361 INFO:teuthology.orchestra.run.smithi123.stdout:generic/724 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-24T15:55:08.375 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/724 2024-04-24T15:55:08.375 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/724 2024-04-24T15:55:08.375 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:08.384 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:08.428 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:08.428 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/725) 2024-04-24T15:55:09.213 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:09.229 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:09.734 INFO:teuthology.orchestra.run.smithi123.stdout:generic/725 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-24T15:55:09.748 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/725 2024-04-24T15:55:09.748 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/725 2024-04-24T15:55:09.748 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:09.756 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:09.787 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:09.787 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/726) 2024-04-24T15:55:10.558 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:10.571 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:10.571 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:10.682 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T15:55:10.687 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T15:55:10.802 INFO:tasks.ceph.mds.a.smithi123.stderr:2024-04-24T15:55:10.804+0000 7f7cdcc00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 547327) UID: 0 2024-04-24T15:55:10.803 INFO:tasks.ceph.mds.c.smithi123.stderr:2024-04-24T15:55:10.804+0000 7f20c3400640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 547327) UID: 0 2024-04-24T15:55:10.861 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T15:55:10.863+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57436) UID: 0 2024-04-24T15:55:10.862 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T15:55:10.863+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57436) UID: 0 2024-04-24T15:55:10.862 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T15:55:10.863+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57436) UID: 0 2024-04-24T15:55:10.862 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T15:55:10.863+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57436) UID: 0 2024-04-24T15:55:12.268 INFO:teuthology.orchestra.run.smithi123.stdout:generic/726 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-24T15:55:12.284 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/726 2024-04-24T15:55:12.284 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/726 2024-04-24T15:55:12.285 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:12.294 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:12.337 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:12.337 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/727) 2024-04-24T15:55:13.189 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:13.205 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:13.205 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:13.862 INFO:teuthology.orchestra.run.smithi123.stdout:generic/727 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-24T15:55:13.878 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/727 2024-04-24T15:55:13.878 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/727 2024-04-24T15:55:13.878 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:13.893 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:13.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:13.933 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/728) 2024-04-24T15:55:14.903 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:14.924 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:14.925 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:19.678 INFO:teuthology.orchestra.run.smithi123.stdout:generic/728 5s 2024-04-24T15:55:19.703 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/728 2024-04-24T15:55:19.703 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:19.712 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:19.748 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:19.749 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/729) 2024-04-24T15:55:20.377 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:20.392 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:20.392 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:21.359 INFO:teuthology.orchestra.run.smithi123.stdout:generic/729 1s 2024-04-24T15:55:21.386 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/729 2024-04-24T15:55:21.386 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:21.398 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:21.445 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:21.446 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/730) 2024-04-24T15:55:22.294 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:22.310 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:22.310 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:22.774 INFO:teuthology.orchestra.run.smithi123.stdout:generic/730 [not run] require ceph-fuse to be valid block disk 2024-04-24T15:55:22.787 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/730 2024-04-24T15:55:22.787 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/730 2024-04-24T15:55:22.787 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:22.797 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:22.833 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:22.834 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/731) 2024-04-24T15:55:23.443 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:23.462 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:23.462 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:23.920 INFO:teuthology.orchestra.run.smithi123.stdout:generic/731 [not run] require ceph-fuse to be valid block disk 2024-04-24T15:55:23.933 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/731 2024-04-24T15:55:23.933 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/731 2024-04-24T15:55:23.933 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:23.946 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:23.979 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:23.979 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/732) 2024-04-24T15:55:24.602 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:24.614 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:24.614 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:25.070 INFO:teuthology.orchestra.run.smithi123.stdout:generic/732 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:55:25.081 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/732 2024-04-24T15:55:25.082 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/732 2024-04-24T15:55:25.082 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:25.089 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:25.122 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:25.123 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/733) 2024-04-24T15:55:25.786 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:25.803 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:26.241 INFO:teuthology.orchestra.run.smithi123.stdout:generic/733 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:55:26.251 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/733 2024-04-24T15:55:26.251 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/733 2024-04-24T15:55:26.251 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:26.264 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:26.296 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:26.296 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/734) 2024-04-24T15:55:26.897 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:26.914 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:26.914 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:27.480 INFO:teuthology.orchestra.run.smithi123.stdout:generic/734 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-24T15:55:27.491 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/734 2024-04-24T15:55:27.492 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/734 2024-04-24T15:55:27.492 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:27.502 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:27.534 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:27.535 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/735) 2024-04-24T15:55:28.373 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:28.389 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:28.389 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:31.281 INFO:teuthology.orchestra.run.smithi123.stdout:generic/735 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-24T15:55:31.292 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/735 2024-04-24T15:55:31.292 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/735 2024-04-24T15:55:31.292 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:55:31.302 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:55:31.336 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:55:31.336 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/736) 2024-04-24T15:55:32.213 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:55:32.232 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:55:32.232 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:59:10.800 INFO:teuthology.orchestra.run.smithi123.stdout:generic/736 218s 2024-04-24T15:59:10.819 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/736 2024-04-24T15:59:10.819 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:59:10.830 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:59:10.874 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:59:10.875 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/737) 2024-04-24T15:59:11.757 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:59:11.774 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:59:11.774 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:59:12.187 INFO:teuthology.orchestra.run.smithi123.stdout:generic/737 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:59:12.199 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/737 2024-04-24T15:59:12.199 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/737 2024-04-24T15:59:12.199 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:59:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:59:12.247 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:59:12.247 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/738) 2024-04-24T15:59:12.900 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:59:12.916 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:59:12.916 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:59:13.385 INFO:teuthology.orchestra.run.smithi123.stdout:generic/738 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:59:13.399 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/738 2024-04-24T15:59:13.399 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/738 2024-04-24T15:59:13.399 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:59:13.409 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:59:13.442 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:59:13.442 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/739) 2024-04-24T15:59:14.068 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:59:14.081 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:59:14.082 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:59:14.627 INFO:teuthology.orchestra.run.smithi123.stdout:generic/739 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:59:14.641 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/739 2024-04-24T15:59:14.641 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/739 2024-04-24T15:59:14.641 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:59:14.648 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:59:14.683 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:59:14.683 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/741) 2024-04-24T15:59:15.349 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:59:15.364 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:59:15.364 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:59:18.641 INFO:teuthology.orchestra.run.smithi123.stdout:generic/741 [not run] this test requires a valid $SCRATCH_DEV 2024-04-24T15:59:18.654 INFO:teuthology.orchestra.run.smithi123.stdout:Ran: generic/741 2024-04-24T15:59:18.655 INFO:teuthology.orchestra.run.smithi123.stdout:Not run: generic/741 2024-04-24T15:59:18.655 INFO:teuthology.orchestra.run.smithi123.stdout:Passed all 1 tests 2024-04-24T15:59:18.666 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T15:59:18.699 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-24T15:59:18.699 DEBUG:teuthology.orchestra.run.smithi123:> (cd /tmp/tmp.ejcLBa8fOwxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/742) 2024-04-24T15:59:19.271 INFO:teuthology.orchestra.run.smithi123.stdout:FSTYP -- ceph-fuse 2024-04-24T15:59:19.287 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM -- Linux/x86_64 smithi123 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-24T15:59:19.288 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-24T16:09:23.566 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T16:09:23.566+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 554418) UID: 0 2024-04-24T16:09:23.566 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T16:09:23.566+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 554418) UID: 0 2024-04-24T16:09:23.567 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T16:09:23.566+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 554418) UID: 0 2024-04-24T16:09:23.567 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T16:09:23.567+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 554418) UID: 0 2024-04-24T16:21:29.832 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:21:29.838 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:21:30.104 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T16:21:30.106+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57881) UID: 0 2024-04-24T16:21:30.104 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T16:21:30.106+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57881) UID: 0 2024-04-24T16:21:30.105 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T16:21:30.106+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57881) UID: 0 2024-04-24T16:21:30.105 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T16:21:30.106+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 57881) UID: 0 2024-04-24T16:21:30.444 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T16:21:30.446+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 554533) UID: 0 2024-04-24T16:21:30.445 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T16:21:30.446+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 554533) UID: 0 2024-04-24T16:21:30.445 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T16:21:30.447+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 554533) UID: 0 2024-04-24T16:21:30.446 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T16:21:30.448+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 554533) UID: 0 2024-04-24T16:37:04.424 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:37:04.430 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:37:04.561 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-24T16:37:04.561+0000 7ff33a200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 554785) UID: 0 2024-04-24T16:37:04.561 INFO:tasks.ceph.osd.0.smithi123.stderr:2024-04-24T16:37:04.561+0000 7f3e5dc00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 554785) UID: 0 2024-04-24T16:37:04.562 INFO:tasks.ceph.osd.1.smithi123.stderr:2024-04-24T16:37:04.563+0000 7fed5d000640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 554785) UID: 0 2024-04-24T16:37:04.563 INFO:tasks.ceph.osd.3.smithi123.stderr:2024-04-24T16:37:04.563+0000 7f388d600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 554785) UID: 0 2024-04-24T16:37:04.609 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-04-24T16:37:04.611+0000 7fcdba200640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 58085) UID: 0 2024-04-24T16:37:04.610 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-04-24T16:37:04.611+0000 7ff1d3600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 58085) UID: 0 2024-04-24T16:37:04.610 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-04-24T16:37:04.611+0000 7f92cc400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 58085) UID: 0 2024-04-24T16:37:04.610 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-04-24T16:37:04.611+0000 7f80e8800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 58085) UID: 0 2024-04-24T16:43:00.012 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:43:00.049 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:43:30.173 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:43:30.182 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:44:00.305 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:44:00.314 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:44:30.503 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:44:30.512 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:45:00.632 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:45:00.641 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:45:30.803 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:45:30.813 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:46:00.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:46:00.960 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:46:31.141 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:46:31.150 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:47:01.292 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:47:01.301 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:47:32.243 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:47:32.253 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:48:02.374 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:48:02.384 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:48:32.563 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:48:32.571 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:49:02.735 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:49:02.744 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:49:32.881 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:49:32.890 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:50:03.104 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:50:03.112 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:50:33.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:50:33.361 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:51:03.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:51:03.487 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:51:33.687 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:51:33.697 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:52:03.860 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:52:03.870 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:52:34.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:52:34.014 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:53:04.203 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:53:04.212 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:53:34.358 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:53:34.367 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:54:04.606 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:54:04.615 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:54:34.786 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:54:34.795 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:55:04.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:55:04.961 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:55:35.147 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:55:35.156 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:56:05.340 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:56:05.350 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:56:35.543 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:56:35.553 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:57:05.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:57:05.715 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:57:35.938 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:57:35.948 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:58:06.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:58:06.085 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:58:36.265 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:58:36.275 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:59:06.419 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:59:06.428 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:59:36.583 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T16:59:36.593 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:00:06.797 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:00:06.806 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:00:36.938 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:00:36.947 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:01:07.082 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:01:07.092 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:01:37.272 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:01:37.282 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:02:07.486 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:02:07.496 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:02:37.625 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:02:37.635 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:03:07.830 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:03:07.838 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:03:38.061 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:03:38.070 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:04:08.274 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:04:08.283 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:04:38.406 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:04:38.415 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:05:08.606 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:05:08.615 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:05:38.799 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:05:38.809 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:06:08.980 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:06:08.989 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:06:39.169 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:06:39.182 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:07:09.358 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:07:09.368 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:07:39.504 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:07:39.513 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:08:09.645 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:08:09.653 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:08:40.517 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:08:40.526 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:09:10.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:09:10.680 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:09:40.834 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:09:40.843 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:10:11.034 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:10:11.042 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:10:41.248 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:10:41.257 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:11:11.395 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:11:11.482 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:11:41.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:11:41.643 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:12:11.755 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:12:11.764 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:12:41.887 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:12:41.896 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:13:12.008 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:13:12.018 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:13:42.155 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:13:42.164 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:14:12.328 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:14:12.336 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:14:42.542 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:14:42.552 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:15:12.739 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:15:12.748 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:15:42.937 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:15:42.946 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:16:13.068 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:16:13.076 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:16:43.222 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:16:43.231 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:17:13.462 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:17:13.473 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:17:43.595 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:17:43.607 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:18:13.763 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:18:13.772 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:18:44.282 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:18:44.293 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:19:14.412 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:19:14.422 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:19:44.613 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:19:44.622 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:20:14.819 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:20:14.831 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:20:45.025 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:20:45.035 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:21:15.172 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:21:15.181 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:21:45.387 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:21:45.396 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:22:15.592 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:22:15.601 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:22:45.798 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:22:45.807 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:23:15.936 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:23:15.945 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:23:46.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:23:46.143 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:24:16.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:24:16.323 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:24:46.486 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:24:46.495 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:25:16.690 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:25:16.699 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:25:46.847 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:25:46.856 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:26:16.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:26:17.003 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:26:47.115 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:26:47.124 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:27:17.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:27:17.422 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:27:47.601 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:27:47.611 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:28:17.807 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:28:17.816 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:28:47.937 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:28:47.945 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:29:18.947 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:29:18.956 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:29:49.098 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:29:49.108 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:30:19.279 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:30:19.287 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:30:49.485 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:30:49.494 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:31:19.682 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:31:19.692 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:31:49.862 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:31:49.871 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:32:20.042 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:32:20.051 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:32:50.273 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:32:50.282 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:33:20.454 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:33:20.465 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:33:50.584 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:33:50.593 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-24T17:34:07.867 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-04-24T17:34:07.867 DEBUG:teuthology.task.console_log:Killing console logger for smithi123 2024-04-24T17:34:07.868 DEBUG:teuthology.task.console_log:Killing console logger for smithi155 2024-04-24T17:34:07.868 DEBUG:teuthology.exit:Finished running handlers